Example

A researcher is trying to develop a new, less expensive, test to detect a particular chemical in soil samples. The old test correlates to the criterion (i.e., gold standard measurement) at r = 0.89.  The new test correlates to the criterion at r = 0.76.  Two research assistants are asked to collect soil samples to compare the old test with the new test. The first research assistant collects 42 independent soil samples; the second research assistant collects 47 samples. Each runs a power analysis to determine the observed power.

Next, the research assistants are asked to calculate the number of independent samples necessary to detect a difference between 0.89 and 0.76 for power values of .7, .8 and .9.

Prelude to the Power Analysis

There are two different aspects of power analysis. One is to calculate the observed power for a specified sample size as in the first part of the example. The other aspect is to calculate the necessary sample size when given a specific power as in the second part of the example. Technically, power is the probability of rejecting the null hypothesis when the specific alternative hypothesis is true. For all examples, we will assume alpha = 0.05.

Power Analysis

In SAS, it is fairly straightforward to perform power analysis for comparing correlations. For example, we can use SAS’s proc power for our calculation as shown below. We first specify the onecorr statement and that the distribution is Fisher’s r to z (fisherz). Next we use the nullcorr statement and provide the correlation of the old test to the criterion. Next, we use the corr statement and provide the correlation between the new test and the criterion. We use the ntotal statement to indicate the number of samples collected, and we have a period (.) on the power statement to indicate that we want SAS to calculate the power. The first research assistant collected 42 soil samples, so we specify 42 in the ntotal statement. We will run the analysis a second time with the value of 47 on the ntotal statement, the number of samples the second research assistant collected.

proc power;
onecorr dist=fisherz
nullcorr = 0.89
corr = 0.76
ntotal = 42
power = .;
run;


The POWER Procedure
Fisher's z Test for Pearson Correlation

                      Fixed Scenario Elements

Distribution                          Fisher's z transformation of r
Method                                          Normal approximation
Null Correlation                                                0.89
Correlation                                                     0.76
Total Sample Size                                                 42
Number of Sides                                                    2
Nominal Alpha                                                   0.05
Number of Variables Partialled Out                                 0


 Computed Power

 Actual
  Alpha    Power

 0.0488    0.761

proc power;
onecorr dist=fisherz
nullcorr = 0.89
corr = 0.76
ntotal = 47
power = .;
run;

The POWER Procedure
Fisher's z Test for Pearson Correlation

                      Fixed Scenario Elements

Distribution                          Fisher's z transformation of r
Method                                          Normal approximation
Null Correlation                                                0.89
Correlation                                                     0.76
Total Sample Size                                                 47
Number of Sides                                                    2
Nominal Alpha                                                   0.05
Number of Variables Partialled Out                                 0


 Computed Power

Actual
 Alpha    Power

 0.049    0.809

We can see that the first research assistant observed a power of approximately 0.76, while the second research assistant observed a power of approximately 0.81.

Now the research assistants need to calculate the necessary sample sizes for power values of .7, .8 and .9. In SAS, we could run three separate power analyses, or we could run one analysis and get the results for all three levels of power in a single table. Let’s do that.

proc power;
onecorr dist=fisherz
nullcorr = 0.89
corr = 0.76
ntotal = .
power = .7 .8 .9;
run;

The POWER Procedure
Fisher's z Test for Pearson Correlation

                      Fixed Scenario Elements

Distribution                          Fisher's z transformation of r
Method                                          Normal approximation
Null Correlation                                                0.89
Correlation                                                     0.76
Number of Sides                                                    2
Nominal Alpha                                                   0.05
Number of Variables Partialled Out                                 0


                 Computed N Total

            Nominal     Actual    Actual        N
   Index      Power      Alpha     Power    Total

       1        0.7     0.0487     0.703       37
       2        0.8     0.0490     0.800       46
       3        0.9     0.0492     0.902       61

The required sample size for a power of .7 is 37. The required sample size for a power of .8 is 46, and the required sample size for a power of .9 is 61. This makes sense, because as power increases, the sample size must increase, assuming that alpha and the effect size are held constant.