Table of Contents
In SAS, one can use the PROC SORT statement with the NODUPKEY or NODUPRECS options to remove any duplicate records from the dataset. This will create a dataset that only contains unique observations and no duplicated records. Additionally, the DISTINCT option in PROC SQL can also be used to eliminate any duplicate observations.
You can use PROC SORT in SAS with NODUPKEY to order the observations in a dataset by one or more variables and remove any duplicates.
The following example shows how to use this procedure with the following dataset in SAS:
/*create dataset*/
data original_data;
input team $ points rebounds;
datalines;
A 12 8
A 12 8
A 12 8
A 23 9
A 20 12
A 14 7
A 14 7
B 20 2
B 20 5
B 29 4
B 14 7
B 20 2
B 20 2
B 20 5
;
run;
/*view dataset*/
proc print data=original_data;
Example: Using PROC SORT with NODUPKEY in SAS
Suppose we simply use proc sort to sort the observations in the dataset in ascending order (smallest to largest) based on the value in the points column:
/*sort by points ascending*/
proc sort data=original_data out=data2;
by points;
run;
/*view sorted dataset*/
proc print data=data2;
Notice that the observations are sorted in ascending order based on the value in the points column.
However, there are several observations that are duplicates.
To sort the observations based on the values in the points column and remove all duplicates, we can add nodupkey after the proc sort statement:
/*sort by points ascending and remove duplicates*/
proc sort data=original_data out=data3 nodupkey;
by points;
run;
/*view sorted dataset*/
proc print data=data3;
The observations are now sorted in ascending order based on the value in the points column and all duplicate observations have been removed.
Note that we can also add the argument descending to instead sort the observations based on the value in the points column in descending order and remove all duplicates:
/*sort by points descending and remove duplicates*/
proc sort data=original_data out=data4 nodupkey;
by descending points;
run;
/*view sorted dataset*/
proc print data=data4;
The following tutorials explain how to perform other common tasks in SAS: