Table of Contents
One of the most common metrics used to measure the forecasting accuracy of a model is MAPE, which stands for mean absolute percentage error.
The formula to calculate MAPE is as follows:
MAPE = (1/n) * Σ(|actual – forecast| / |actual|) * 100
where:
- Σ – a fancy symbol that means “sum”
- n – sample size
- actual – the actual data value
- forecast – the forecasted data value
MAPE is commonly used because it’s easy to interpret and explain. For example, a MAPE value of 8% means that the average difference between the forecasted value and the actual value is 8%.
One of the most common questions people have when using this metric is:
What is a good value for MAPE?
The unsatisfying answer: It depends.
Obviously the lower the value for MAPE the better, but there is no specific value that you can call “good” or “bad.” It depends on a couple factors:
- The type of industry
- The MAPE value compared to a simple forecasting model
Let’s explore these two factors in depth.
MAPE Varies by Industry
Often companies create forecasts for demand of their products and then use MAPE as a way to measure the accuracy of the forecasts.
Unfortunately, there is no “standard” MAPE value because it can vary so much by the type of company.
For example, a company that rarely changes their pricing will likely have steady and predictable demand, which means they may have a model that produces a very low MAPE, perhaps under 3%.
For other companies that constantly run promotions and specials, their demand will vary greatly over time and thus a forecasting model will likely have a harder time predicting demand as accurately which means the models may have a higher value for MAPE.
Compare MAPE to a Simple Forecasting Model
Rather than trying to compare the MAPE of your model with some arbitrary “good” value, you should instead compare it to the MAPE of simple forecasting models.
There are two well-known simple forecasting models:
1. The average forecasting method.
This type of forecast model simply predicts the value for the next upcoming period to be the average of all prior periods. Although this method seems overly simplistic, it actually tends to perform well in practice.
2. The naïve forecasting method.
This type of forecast model predicts the value for the next upcoming period to be equal to the prior period. Again, although this method is quite simple it tends to work surprisingly well.
When developing a new forecasting model, you should compare the MAPE of that model to the MAPE of these two simple forecasting methods.
If the MAPE of your new model is not significantly better than these two methods, then you shouldn’t consider it to be useful.
Closing Thoughts
Although MAPE is widely used and easy to interpret, there are a couple potential drawbacks to using it:
1. Since the formula to calculate absolute percent error is |actual-forecast| / |actual| this means that it will be undefined if any of the actual values are zero.
2. MAPE should not be used with low volume data. For example, if the actual demand for some item is 2 and the forecast is 1, the value for the absolute percent error will be |2-1| / |2| = 50%, which makes it seem like the forecast error is quite high, despite the forecast only being off by one unit.
Potential alternatives to MAPE include mean absolute deviation and root mean squared error.