Table of Contents
Pandas can find the difference between two rows using the diff() function. This function will return the difference between two rows in a DataFrame. The result will be a DataFrame with the same index as the original DataFrame, and the same column names, but with new values that represent the difference between the two rows. This can be very useful for comparing the values from two different rows. It can be used to identify trends or patterns over time, which can be very helpful when analyzing data.
You can use the DataFrame.diff() function to find the difference between two rows in a pandas DataFrame.
This function uses the following syntax:
DataFrame.diff(periods=1, axis=0)
where:
- periods: The number of previous rows for calculating the difference.
- axis: Find difference over rows (0) or columns (1).
The following examples show how to use this function in practice.
Example 1: Find Difference Between Each Previous Row
Suppose we have the following pandas DataFrame:
import pandas as pd #create DataFrame df = pd.DataFrame({'period': [1, 2, 3, 4, 5, 6, 7, 8], 'sales': [12, 14, 15, 15, 18, 20, 19, 24], 'returns': [2, 2, 3, 3, 5, 4, 4, 6]}) #view DataFrame df period sales returns 0 1 12 2 1 2 14 2 2 3 15 3 3 4 15 3 4 5 18 5 5 6 20 4 6 7 19 4 7 8 24 6
The following code shows how to find the difference between every current row in a DataFrame and the previous row:
#add new column to represent sales differences between each row df['sales_diff'] = df['sales'].diff() #view DataFrame df period sales returns sales_diff 0 1 12 2 NaN 1 2 14 2 2.0 2 3 15 3 1.0 3 4 15 3 0.0 4 5 18 5 3.0 5 6 20 4 2.0 6 7 19 4 -1.0 7 8 24 6 5.0
Note that we can also find the difference between several rows prior. For example, the following code shows how to find the difference between each current row and the row that occurred three rows earlier:
#add new column to represent sales differences between current row and 3 rows earlier df['sales_diff'] = df['sales'].diff(periods=3) #view DataFrame df period sales returns sales_diff 0 1 12 2 NaN 1 2 14 2 NaN 2 3 15 3 NaN 3 4 15 3 3.0 4 5 18 5 4.0 5 6 20 4 5.0 6 7 19 4 4.0 7 8 24 6 6.0
Example 2: Find Difference Based on Condition
We can also filter the DataFrame to show rows where the difference between the current row and the previous row is less than or greater than some value.
For example, the following code returns only the rows where the value in the current row is less than the value in the previous row:
import pandas as pd #create DataFrame df = pd.DataFrame({'period': [1, 2, 3, 4, 5, 6, 7, 8], 'sales': [12, 14, 15, 13, 18, 20, 19, 24], 'returns': [2, 2, 3, 3, 5, 4, 4, 6]}) #find difference between each current row and the previous row df['sales_diff'] = df['sales'].diff() #filter for rows where difference is less than zero df = df[df['sales_diff']<0] #view DataFrame df period sales returns sales_diff 3 4 13 3 -2.0 6 7 19 4 -1.0