Table of Contents

Multiple linear regression by hand is a step-by-step process to calculate the coefficients of multiple independent variables used to predict a dependent variable. It involves estimating the parameters of a linear equation, which requires finding the mean, variance, and covariance of the variables, calculating the regression coefficients, and testing the significance of the model. This process follows a number of steps, such as forming the linear equation, taking the partial derivatives of the cost function, setting the derivatives to zero, and solving to find the coefficients.

Multiple linear regression is a method we can use to quantify the relationship between two or more predictor variables and a response variable.

This tutorial explains how to perform multiple linear regression by hand.

**Example: Multiple Linear Regression by Hand**

Suppose we have the following dataset with one response variable *y* and two predictor variables X_{1} and X_{2}:

Use the following steps to fit a multiple linear regression model to this dataset.

**Step 1: Calculate X _{1}^{2}, X_{2}^{2}, X_{1}y, X_{2}y and X_{1}X_{2}.**

**Step 2: Calculate Regression Sums.**

Next, make the following regression sum calculations:

- Σx
_{1}^{2 }= ΣX_{1}^{2 }– (ΣX_{1})^{2}/ n = 38,767 – (555)^{2}/ 8 =**263.875** - Σx
_{2}^{2 }= ΣX_{2}^{2 }– (ΣX_{2})^{2}/ n = 2,823 – (145)^{2}/ 8 =**194.875** - Σx
_{1}y = ΣX_{1}y – (ΣX_{1}Σy) / n = 101,895 – (555*1,452) / 8 =**1,162.5** - Σx
_{2}y = ΣX_{2}y – (ΣX_{2}Σy) / n = 25,364 – (145*1,452) / 8 =**-953.5** - Σx
_{1}x_{2}= ΣX_{1}X_{2}– (ΣX_{1}ΣX_{2}) / n = 9,859 – (555*145) / 8 =**-200.375**

**Step 3: Calculate b _{0}, b_{1}, and b_{2}.**

The formula to calculate b_{1 }is: [(Σx_{2}^{2})(Σx_{1}y) – (Σx_{1}x_{2})(Σx_{2}y)] / [(Σx_{1}^{2}) (Σx_{2}^{2}) – (Σx_{1}x_{2})^{2}]

Thus, **b _{1 }**= [(194.875)(1162.5) – (-200.375)(-953.5)] / [(263.875) (194.875) – (-200.375)

^{2}] =

**3.148**

The formula to calculate b_{2 }is: [(Σx_{1}^{2})(Σx_{2}y) – (Σx_{1}x_{2})(Σx_{1}y)] / [(Σx_{1}^{2}) (Σx_{2}^{2}) – (Σx_{1}x_{2})^{2}]

Thus, **b _{2 }**= [(263.875)(-953.5) – (-200.375)(1152.5)] / [(263.875) (194.875) – (-200.375)

^{2}] =

**-1.656**

Thus, **b _{0 }**= 181.5 – 3.148(69.375) – (-1.656)(18.125) =

**-6.867**

**Step 5: Place b _{0}, b_{1}, and b_{2} in the estimated linear regression equation.**

The estimated linear regression equation is: ŷ = b_{0} + b_{1}*x_{1} + b_{2}*x_{2}

In our example, it is **ŷ = -6.867 + 3.148x _{1} – 1.656x_{2}**

**How to Interpret a Multiple Linear Regression Equation**

Here is how to interpret this estimated linear regression equation: ŷ = -6.867 + 3.148x_{1} – 1.656x_{2}

**b _{0} = -6.867**. When both predictor variables are equal to zero, the mean value for y is -6.867.

**b _{1 }= 3.148**. A one unit increase in x

_{1 }is associated with a 3.148 unit increase in y, on average, assuming x

_{2 }is held constant.

**b _{2 }= -1.656**. A one unit increase in x

_{2 }is associated with a 1.656 unit decrease in y, on average, assuming x

_{1 }is held constant.

An Introduction to Multiple Linear Regression

How to Perform Simple Linear Regression by Hand