Table of Contents
Normalizing columns in a Pandas Dataframe involves transforming the values of each column using a function so that all of the columns have a similar scale. This can be done by subtracting each value in the column by the mean of the column and dividing by the standard deviation of the column. This will make the data points have a mean of 0 and a standard deviation of 1 for the normalized column.
Often you may want to normalize the data values of one or more columns in a pandas DataFrame.
This tutorial explains two ways to do so:
1. Min-Max Normalization
- Objective: Converts each data value to a value between 0 and 1.
- Formula: New value = (value – min) / (max – min)
2. Mean Normalization
- Objective: Scales values such that the mean of all values is 0 and std. dev. is 1.
- Formula: New value = (value – mean) / (standard deviation)
Let’s check out an example of how to use each method on a pandas DataFrame.
Example 1: Min-Max Normalization
Suppose we have the following pandas DataFrame:
import pandas as pd #create DataFrame df = pd.DataFrame({'points': [25, 12, 15, 14, 19], 'assists': [5, 7, 7, 9, 12], 'rebounds': [11, 8, 10, 6, 6]}) #view DataFrame print(df) points assists rebounds 0 25 5 11 1 12 7 8 2 15 7 10 3 14 9 6 4 19 12 6
We can use the following code to apply a min-max normalization to each column in the DataFrame:
(df-df.min())/(df.max()-df.min()) points assists rebounds 0 1.000000 0.000000 1.0 1 0.000000 0.285714 0.4 2 0.230769 0.285714 0.8 3 0.153846 0.571429 0.0 4 0.538462 1.000000 0.0
The max value in each column is now equal to 1 and the min value in each column is now equal to 0, with all other values ranging between 0 and 1.
Example 2: Mean Normalization
Once again suppose we have the following pandas DataFrame:
import pandas as pd #create DataFrame df = pd.DataFrame({'points': [25, 12, 15, 14, 19], 'assists': [5, 7, 7, 9, 12], 'rebounds': [11, 8, 10, 6, 6]}) #view DataFrame print(df) points assists rebounds 0 25 5 11 1 12 7 8 2 15 7 10 3 14 9 6 4 19 12 6
We can use the following code to apply a mean normalization to each column in the DataFrame:
(df-df.mean())/df.std() points assists rebounds 0 1.554057 -1.133893 1.227881 1 -0.971286 -0.377964 -0.087706 2 -0.388514 -0.377964 0.789352 3 -0.582772 0.377964 -0.964764 4 0.388514 1.511858 -0.964764
The values in each column are now normalized such that the mean of the values in each column is 0 and the standard deviation of values in each column is 1.
If a particular data point has a normalized value greater than 0, it’s an indication that the data point is greater than the mean of its column. Conversely, a normalized value less than 0 is an indication that the data point is less than the mean of its column.
Pandas: How to Group and Aggregate by Multiple Columns
How to Filter a Pandas DataFrame on Multiple Conditions
How to Count Missing Values in a Pandas DataFrame