How can I select rows in a Pandas DataFrame that do not contain any NaN values?

How can I select rows in a Pandas DataFrame that do not contain any NaN values?

One can use the “dropna()” function in Pandas to select rows in a DataFrame that do not contain any missing or NaN values. This function can be applied to the entire DataFrame or to specific columns by specifying the axis parameter. This method allows for the removal of incomplete or missing data, resulting in a clean and complete dataset for analysis.

Select Rows without NaN Values in Pandas


You can use the following methods to select rows without NaN values in pandas:

Method 1: Select Rows without NaN Values in All Columns

df[~df.isnull().any(axis=1)]

Method 2: Select Rows without NaN Values in Specific Column

df[~df['this_column'].isna()]

The following examples show how to use each method in practice with the following pandas DataFrame:

import pandas as pd
import numpy as np

#create DataFrame
df = pd.DataFrame({'team': ['A', 'B', 'C', 'D', 'E', 'F', 'G'],
                   'points': [np.nan, 12, 15, 25, np.nan, 22, 30],
                   'assists': [4, np.nan, 5, 9, 12, 14, 10]})

#view DataFrame
print(df)

  team  points  assists
0    A     NaN      4.0
1    B    12.0      NaN
2    C    15.0      5.0
3    D    25.0      9.0
4    E     NaN     12.0
5    F    22.0     14.0
6    G    30.0     10.0

Example 1: Select Rows without NaN Values in All Columns

We can use the following syntax to select rows without NaN values in every column of the DataFrame:

#create new DataFrame that only contains rows without NaNs
no_nans = df[~df.isnull().any(axis=1)]

#view results
print(no_nans)

  team  points  assists
2    C    15.0      5.0
3    D    25.0      9.0
5    F    22.0     14.0
6    G    30.0     10.0   

Notice that each row in the resulting DataFrame contains no NaN values in any column.

Example 2: Select Rows without NaN Values in Specific Column

We can use the following syntax to select rows without NaN values in the points column of the DataFrame:

#create new DataFrame that only contains rows without NaNs in points column
no_points_nans = df[~df['points'].isna()]

#view results
print(no_points_nans)

  team  points  assists
1    B    12.0      NaN
2    C    15.0      5.0
3    D    25.0      9.0
5    F    22.0     14.0
6    G    30.0     10.0

Notice that each row in the resulting DataFrame contains no NaN values in the points column.

There is one row with a NaN value in the assists column, but the row is kept in the DataFrame since the value in the points column of that row is not NaN.

Cite this article

stats writer (2024). How can I select rows in a Pandas DataFrame that do not contain any NaN values?. PSYCHOLOGICAL SCALES. Retrieved from https://scales.arabpsychology.com/stats/how-can-i-select-rows-in-a-pandas-dataframe-that-do-not-contain-any-nan-values/

stats writer. "How can I select rows in a Pandas DataFrame that do not contain any NaN values?." PSYCHOLOGICAL SCALES, 26 Jun. 2024, https://scales.arabpsychology.com/stats/how-can-i-select-rows-in-a-pandas-dataframe-that-do-not-contain-any-nan-values/.

stats writer. "How can I select rows in a Pandas DataFrame that do not contain any NaN values?." PSYCHOLOGICAL SCALES, 2024. https://scales.arabpsychology.com/stats/how-can-i-select-rows-in-a-pandas-dataframe-that-do-not-contain-any-nan-values/.

stats writer (2024) 'How can I select rows in a Pandas DataFrame that do not contain any NaN values?', PSYCHOLOGICAL SCALES. Available at: https://scales.arabpsychology.com/stats/how-can-i-select-rows-in-a-pandas-dataframe-that-do-not-contain-any-nan-values/.

[1] stats writer, "How can I select rows in a Pandas DataFrame that do not contain any NaN values?," PSYCHOLOGICAL SCALES, vol. X, no. Y, ص Z-Z, June, 2024.

stats writer. How can I select rows in a Pandas DataFrame that do not contain any NaN values?. PSYCHOLOGICAL SCALES. 2024;vol(issue):pages.

Download Post (.PDF)
Slide Up
x
PDF
Scroll to Top