In this video, we'll be talking about simple linear regression and multiple linear regression. Linear regression will refer to one independent variable to make a prediction. Multiple linear regression will refer to multiple independent variables to make a prediction. Simple linear regression or SLR is a method to help us understand the relationship between two variables. The predictor independent variable x and the target dependent variable y. We would like to come up with a linear relationship between the variables shown here. The parameter b_0 is the intercept. The parameter b_1 is the slope. When we fit or train the model, we will come up with these parameters. This step requires lots of math, so we will not focus on this part. Let's clarify the prediction step. It's hard to figure out how much a car costs, but the highway miles per gallon is in the owner's manual. If we assume there is a linear relationship between these variables, we can use this relationship to formulate a model to determine the price of the car. If the highway miles per gallon is 20, we can input this value into the model to obtain a prediction of $22,000. In order to determine the line, we take data points from our data set marked in red here. We then use these training points to fit our model. The results of the training points are the parameters. We usually store the data points into data frame or numpy arrays. The value we would like to predict is called the target that we store in the array y. We store the dependent variable in the data frame or array x. Each sample corresponds to a different row in each data frame or array. In many cases, many factors influence how much people pay for a car. For example, make or how old the car is. In this model, this uncertainty is taken into account by assuming a small random value is added to the point on the line. This is called noise. The figure on the left shows the distribution of the noise. The vertical axis shows the value added and the horizontal axis illustrates the probability that the value will be added. Usually a small positive value is added or a small negative value. Sometimes, large values are added. But for the most part, the values added are near zero. We can summarize the process like this. We have a set of training points. We use these training points to fit or train the model and get parameters. We then use these parameters in the model. We now have a model. We use the hat on the y to denote the model is an estimate. We can use this model to predict values that we haven't seen. For example, we have no car with 20 highway miles per gallon. We can use our model to make a prediction for the price of this car. But don't forget our model is not always correct. We can see this by comparing the predicted value to the actual value. We have a sample for 10 highway miles per gallon but the predictive value does not match the actual value. If the linear assumption is correct, this error is due to the noise. But there can be other reasons. To fit the model in Python, first we import linear model from sklearn then create a linear regression object using the constructor. We define the predictor variable and target variable then use the method fit to fit the model and find the parameters b_0 and b_1. The input the features and the targets. We can obtain prediction using the method predict. The output is an array. The array has the same number of samples as the input x. The intercept b_0 is an attribute of the object lm. The slope b_1 is also an attribute of the object lm. The relationship between price and highway miles per gallon is given by this equation in bold, price equals 38,423.31 minus 821.73 times highway miles per gallon, like the equation we discussed before. Multiple linear regression is used to explain the relationship between one continuous target y variable and two or more predictor x variables. If we have for example 4 predictor variables then b_0 intercept x equal zero b _1 the coefficient or parameter of x_1, b_2 the coefficient of parameter x_2 and so on. If there are only two variables then we can visualize the values. Consider the following function. The variables x_1 and x_2 can be visualized on a 2D plane. Let's do an example on the next slide. The table contains different values of the predictor variables x_1 and x_2. The position of each point is placed on the 2D plane color-coded accordingly. Each value of the predictor variables x_1 and x_2 will be mapped to a new value y, y hat. The new values of y y hat are mapped in the vertical direction with height proportional to the value that y hat takes. We can fit the multiple linear regression as follows. We can extract the four predictor variables and store them in the variable z then train the model as before using the method train with the features or dependent variables and the targets color. We can also obtain a prediction using the method predict. In this case, the input is an array or data frame with four columns. The number of rows corresponds to the number of samples. The output is an array with the same number of elements as number of samples. The intercept is an attribute of the object and the coefficients are also attributes. It is helpful to visualize the equation, replacing the dependent variable names with actual names. This is identical to the form we discussed earlier.
