In statistics, prediction error refers to the difference between the predicted values made by some model and the actual values.

Prediction error is often used in two settings:

1. Linear regression: Used to predict the value of some continuous response variable.

We typically measure the prediction error of a linear regression model with a metric known as , which stands for root mean squared error. 

It is calculated as:

RMSE = √Σ(ŷ_i – y_i)² / n

where:

• Σ is a symbol that means “sum”
• ŷ_i is the predicted value for the i^th observation
• y_i is the observed value for the i^th observation
• n is the sample size

2. Logistic Regression: Used to predict the value of some binary response variable.

One common way to measure the prediction error of a logistic regression model is with a metric known as the total misclassification rate.

It is calculated as:

Total misclassification rate = (# incorrect predictions / # total predictions)

The lower the value for the misclassification rate, the better the model is able to predict the outcomes of the response variable.

The following examples show how to calculate prediction error for both a linear regression model and a logistic regression model in practice.

Example 1: Calculating Prediction Error in Linear Regression

Suppose we use a regression model to predict the number of points that 10 players will score in a basketball game.

The following table shows the predicted points from the model vs. the actual points the players scored:

We would calculate the root mean squared error (RMSE) as:

• RMSE = √Σ(ŷ_i – y_i)² / n
• RMSE = √(((14-12)²+(15-15)²+(18-20)²+(19-16)²+(25-20)²+(18-19)²+(12-16)²+(12-20)²+(15-16)²+(22-16)²) / 10)
• RMSE = 4

The root mean squared error is 4. This tells us that the average deviation between the predicted points scored and the actual points scored is 4.

Related:

Example 2: Calculating Prediction Error in Logistic Regression

Suppose we use a logistic regression model to predict whether or not 10 college basketball players will get drafted into the NBA.

The following table shows the predicted outcome for each player vs. the actual outcome (1 = Drafted, 0 = Not Drafted):

We would calculate the total misclassification rate as:

• Total misclassification rate = (# incorrect predictions / # total predictions)
• Total misclassification rate = 4/10
• Total misclassification rate = 40%

The total misclassification rate is 40%.

This value is quite high, which indicates that the model doesn’t do a very good job of predicting whether or not a player will get drafted.

Additional Resources

The following tutorials provide an introduction to different types of regression methods: