Table of Contents
Prediction error in statistics is the difference between the observed value of a given variable and its predicted value based on a statistical model. It is a measure of how well a model accurately predicts a value and is used to compare different models.
In statistics, prediction error refers to the difference between the predicted values made by some model and the actual values.
Prediction error is often used in two settings:
1. Linear regression: Used to predict the value of some continuous response variable.
We typically measure the prediction error of a linear regression model with a metric known as , which stands for root mean squared error.
It is calculated as:
RMSE = √Σ(ŷi – yi)2 / n
where:
- Σ is a symbol that means “sum”
- ŷi is the predicted value for the ith observation
- yi is the observed value for the ith observation
- n is the sample size
2. Logistic Regression: Used to predict the value of some binary response variable.
One common way to measure the prediction error of a logistic regression model is with a metric known as the total misclassification rate.
It is calculated as:
Total misclassification rate = (# incorrect predictions / # total predictions)
The lower the value for the misclassification rate, the better the model is able to predict the outcomes of the response variable.
The following examples show how to calculate prediction error for both a linear regression model and a logistic regression model in practice.
Example 1: Calculating Prediction Error in Linear Regression
Suppose we use a regression model to predict the number of points that 10 players will score in a basketball game.
The following table shows the predicted points from the model vs. the actual points the players scored:
We would calculate the root mean squared error (RMSE) as:
- RMSE = √Σ(ŷi – yi)2 / n
- RMSE = √(((14-12)2+(15-15)2+(18-20)2+(19-16)2+(25-20)2+(18-19)2+(12-16)2+(12-20)2+(15-16)2+(22-16)2) / 10)
- RMSE = 4
The root mean squared error is 4. This tells us that the average deviation between the predicted points scored and the actual points scored is 4.
Related:
Example 2: Calculating Prediction Error in Logistic Regression
Suppose we use a logistic regression model to predict whether or not 10 college basketball players will get drafted into the NBA.
The following table shows the predicted outcome for each player vs. the actual outcome (1 = Drafted, 0 = Not Drafted):
We would calculate the total misclassification rate as:
- Total misclassification rate = (# incorrect predictions / # total predictions)
- Total misclassification rate = 4/10
- Total misclassification rate = 40%
The total misclassification rate is 40%.
This value is quite high, which indicates that the model doesn’t do a very good job of predicting whether or not a player will get drafted.
The following tutorials provide an introduction to different types of regression methods: