How to Detrend Data (With Examples)

The Critical Need for Detrending: Achieving Stationarity

When analyzing time-dependent data, the concept of stationarity is paramount. A time series is considered stationary if its statistical properties—like the mean, variance, and autocorrelation structure—do not change over time. Many powerful statistical models, such as ARIMA, rely on the assumption that the data being analyzed is stationary.

Unfortunately, real-world data often exhibits non-stationary characteristics due to underlying trends (e.g., population growth, inflation, technological advancements). If a dataset displays a clear upward or downward trajectory, it means the mean is changing over time, violating the stationarity assumption. This is where detrending becomes essential.

Detrending transforms a non-stationary series into one that is approximately stationary, enabling the accurate identification of complex relationships, seasonality, or short-run shocks that might otherwise be obscured by the overpowering linear trend. The successful removal of this trend allows for much clearer interpretation of residual variance and improved model accuracy.

Visualizing Hidden Cyclical Patterns

To illustrate the importance of detrending, consider a practical scenario involving sales data. Imagine we have observed the total sales figures for a company over 20 consecutive periods. If these sales are consistently growing due to market expansion or general economic uplift, the raw data plot will show a strong upward trajectory.

For example, consider the following time series data that represents the total sales for some company during 20 consecutive periods:

While it is clear that the sales are trending upwards over time—a significant trend is visually apparent—there also appears to be a smaller, superimposed cyclical or seasonal pattern. This pattern, visible as tiny “hills” or oscillations within the overall growth curve, is difficult to analyze quantitatively when the dominant upward movement remains.

To gain a clearer and more measurable view of this cyclical trend, we must employ detrending. This process isolates the short-term fluctuations by subtracting the long-term upward movement, resulting in a series that highlights only the cyclical behavior.

The Result of Trend Removal

When the overall upward trend is successfully removed from the sales data, the resulting detrended series reveals the cyclical component in isolation. The visualization below shows the data after this transformation has been applied, making the periodic structure obvious.

As demonstrated, the residual series now clearly highlights the periodic fluctuations. This new, stationary series is much more valuable for advanced forecasting models that rely on predictable cyclicality or seasonality. Successfully detrending is the prerequisite for reliable predictive modeling. There are two primary, widely-used methods for achieving this crucial transformation:

There are two common methods used to detrend time series data:

1. Detrend by Differencing: A simple mathematical transformation based on changes.
2. Detrend by Model Fitting: A statistical approach utilizing regression analysis to isolate the trend component.

Method 1: Detrending by Differencing

Differencing is arguably the simplest and most common technique for detrending time series data, especially when the trend is approximately linear. This method involves creating a new dataset where each observation is calculated as the difference between the current observation and the previous one (i.e., the first-order difference).

Mathematically, if $Y_t$ is the value at time $t$, the detrended value $D_t$ is calculated as $D_t = Y_t – Y_{t-1}$. This effectively removes the constant increase or decrease (the linear component) because the output represents the rate of change rather than the absolute value.

This technique is powerful for achieving stationarity. If the trend is non-linear (e.g., exponential), higher-order differencing (taking the difference of the differences) may be required to achieve stationarity, which is often characteristic of integrated time series.

Practical Application of First-Order Differencing

We can apply the concept of first-order differencing to a simple numerical series to see how the trend is eliminated. This procedure transforms the original sequence of absolute values into a sequence representing incremental changes between periods.

For example, the following image shows how to use differencing to detrend a data series:

To obtain the first value of the detrended time series data, we calculate 13 – 8 = 5. Then to obtain the next value we calculate 18 – 13 = 5, and so on. Note that the first value (at time t=1) of the differenced series is undefined, as there is no preceding value (t=0) to subtract from.

If we return to the full sales example used earlier, the comparison between the original (non-stationary) series and the differenced series highlights the power of this method. The differenced plot (Figure 2) shows how much easier it is to identify the periodic movement when the long-term trend is eliminated.

It is significantly easier to identify and analyze the seasonal or cyclical trend in the resulting detrended series because the overall upward trajectory has been mathematically eliminated, leaving behind only the short-term variability.

Method 2: Detrending by Model Fitting

The second robust method for detrending involves fitting a statistical model, typically a linear regression model, to capture the deterministic trend component of the data. Once the trend line is established, the detrended series is calculated by finding the residuals—the difference between the actual observed values and the values predicted by the fitted model.

This technique assumes that the underlying trend can be adequately represented by a mathematical function. The residuals ($e_t$) represent the random noise and the remaining cyclical/seasonal components after the systematic trend component ($hat{Y}_t$) is removed from the actual observation ($Y_t$): $e_t = Y_t – hat{Y}_t$.

This method is often preferred when the trend is assumed to be deterministic and we want to preserve the actual magnitude of the residual variations, unlike differencing which transforms the data into rates of change.

Applying Linear Regression for Trend Estimation

Let’s apply this method to our original dataset. In this approach, the time period index ($t$) is used as the independent variable to predict the observed sales figure ($Y_t$).

For example, suppose we have the same dataset where we observe Sales (Y) over Time (t):

By fitting a linear regression model to this data, we can derive the equation for the best-fit trend line. This equation allows us to obtain a predicted value ($hat{Y}_t$) for each time period ($t$) in the dataset, representing the expected value solely based on the underlying trend.

We then calculate the difference between the actual value ($Y_t$) and the predicted trend value ($hat{Y}_t$) for each observation. These differences, known as the residuals, constitute the detrended data series.

The resulting plot of these residuals provides a clear visual representation of the cyclical patterns, stripped of the overall linear trend component.

If we create a plot of the detrended data (the residuals), we can visualize the seasonal or cyclical trend in the data much more easily:

Choosing the Right Detrending Technique

Deciding between differencing and model fitting hinges on the nature of the trend and the ultimate purpose of the analysis. Differencing is suitable for random walk processes where the trend is stochastic (random) and the data is non-stationary in variance or mean. It is computationally simple and often preferred for making a series stationary for use in ARIMA models.

Model fitting, on the other hand, is best utilized when the trend is known to be deterministic (i.e., fixed and predictable over time). It provides residuals that retain the original units of measurement, which can be advantageous if the amplitude of the fluctuations needs to be interpreted directly.

Note that we used linear regression in this example, but it is possible to use a more complex method, such as polynomial regression or exponential smoothing, if there is an exponentially increasing or decreasing pattern in the data. Selecting the correct model complexity ensures that the trend is fully absorbed by the fit, maximizing the stationarity of the resulting residuals.