How can I drop rows in a Pandas DataFrame based on a condition?

How can I drop rows in a Pandas DataFrame based on a condition?

The process of dropping rows in a Pandas DataFrame based on a condition involves using the .drop() method to remove rows that do not meet a specified criteria. This can be achieved by using conditional statements or by using the .loc[] function to select rows based on a specific condition. By dropping rows, the DataFrame can be filtered to only include the desired data, making it easier to work with and analyze. This method is useful for cleaning and organizing data in a DataFrame.

Drop Rows in Pandas DataFrame Based on Condition


We can use the following syntax to drop rows in a pandas DataFrame based on condition:

Method 1: Drop Rows Based on One Condition

df = df[df.col1 > 8]

Method 2: Drop Rows Based on Multiple Conditions

df = df[(df.col1 > 8) & (df.col2 != 'A')]

Note: We can also use the drop() function to drop rows from a DataFrame, but this function has been shown to be much slower than just assigning the DataFrame to a filtered version of itself.

The following examples show how to use this syntax in practice with the following pandas DataFrame:

import pandas as pd

#create DataFrame
df = pd.DataFrame({'team': ['A', 'A', 'A', 'A', 'B', 'B', 'B', 'B'],
                   'pos': ['G', 'G', 'F', 'F', 'G', 'G', 'F', 'F'],
                   'assists': [5, 7, 7, 9, 12, 9, 9, 4],
                   'rebounds': [11, 8, 10, 6, 6, 5, 9, 12]})

#view DataFrame
df

	team	pos	assists	rebounds
0	A	G	5	11
1	A	G	7	8
2	A	F	7	10
3	A	F	9	6
4	B	G	12	6
5	B	G	9	5
6	B	F	9	9
7	B	F	4	12

Method 1: Drop Rows Based on One Condition

The following code shows how to drop rows in the DataFrame based on one condition:

#drop rows where value in 'assists' column is less than or equal to 8
df = df[df.assists > 8] 

#view updated DataFrame
df

	team	pos	assists	rebounds
3	A	F	9	6
4	B	G	12	6
5	B	G	9	5
6	B	F	9	9

Any row that had a value less than or equal to 8 in the ‘assists’ column was dropped from the DataFrame.

Method 2: Drop Rows Based on Multiple Conditions

The following code shows how to drop rows in the DataFrame based on multiple conditions:

#only keep rows where 'assists' is greater than 8 and rebounds is greater than 5
df = df[(df.assists > 8) & (df.rebounds > 5)]

#view updated DataFrame
df

	team	pos	assists	rebounds
3	A	F	9	6
4	B	G	12	6
5	B	G	9	5
6	B	F	9	9

The only rows that we kept in the DataFrame were the ones where the assists value was greater than 8 and the rebounds value was greater than 5.

Note that we can also use the | operator to apply an “or” filter:

#only keep rows where 'assists' is greater than 8 or rebounds is greater than 10
df = df[(df.assists > 8) | (df.rebounds > 10)]

#view updated DataFrame
df

	team	pos	assists	rebounds
0	A	G	5	11
3	A	F	9	6
4	B	G	12	6
5	B	G	9	5
6	B	F	9	9
7	B	F	4	12

The only rows that we kept in the DataFrame were the ones where the assists value was greater than 8 or the rebounds value was greater than 10.

Any rows that didn’t meet one of these conditions was dropped.

Additional Resources

The following tutorials explain how to perform other common operations in pandas:

Cite this article

stats writer (2024). How can I drop rows in a Pandas DataFrame based on a condition?. PSYCHOLOGICAL SCALES. Retrieved from https://scales.arabpsychology.com/stats/how-can-i-drop-rows-in-a-pandas-dataframe-based-on-a-condition/

stats writer. "How can I drop rows in a Pandas DataFrame based on a condition?." PSYCHOLOGICAL SCALES, 2 Jul. 2024, https://scales.arabpsychology.com/stats/how-can-i-drop-rows-in-a-pandas-dataframe-based-on-a-condition/.

stats writer. "How can I drop rows in a Pandas DataFrame based on a condition?." PSYCHOLOGICAL SCALES, 2024. https://scales.arabpsychology.com/stats/how-can-i-drop-rows-in-a-pandas-dataframe-based-on-a-condition/.

stats writer (2024) 'How can I drop rows in a Pandas DataFrame based on a condition?', PSYCHOLOGICAL SCALES. Available at: https://scales.arabpsychology.com/stats/how-can-i-drop-rows-in-a-pandas-dataframe-based-on-a-condition/.

[1] stats writer, "How can I drop rows in a Pandas DataFrame based on a condition?," PSYCHOLOGICAL SCALES, vol. X, no. Y, ص Z-Z, July, 2024.

stats writer. How can I drop rows in a Pandas DataFrame based on a condition?. PSYCHOLOGICAL SCALES. 2024;vol(issue):pages.

Download Post (.PDF)
Slide Up
x
PDF
Scroll to Top