Stata FAQ: How can I do post-hoc pairwise comparisons of adjusted means in Stata?

This FAQ will cover doing pairwise comparisons for adjusted means and will make use of
the margins and pwcompare commands.

We will demonstrate the pairwise comparisons of adjusted means using the
hsbdemo dataset with prog (program type) as the categorical variable
and read as the continuous covariate.

We will begin by running a model without the covariate so that we can see the differences in the unadjusted means.

use https://stats.idre.ucla.edu/stat/data/hsbdemo, clear

tabstat write, by(prog) stat(n mean sd)

Summary for variables: write
     by categories of: prog (type of program)

    prog |         N      mean        sd
---------+------------------------------
 general |        45  51.33333  9.397775
academic |       105  56.25714  7.943343
vocation |        50     46.76  9.318754
---------+------------------------------
   Total |       200    52.775  9.478586
----------------------------------------

 anova write prog

                           Number of obs =     200     R-squared     =  0.1776
                           Root MSE      = 8.63918     Adj R-squared =  0.1693

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |  3175.69786     2  1587.84893      21.27     0.0000
                         |
                    prog |  3175.69786     2  1587.84893      21.27     0.0000
                         |
                Residual |  14703.1771   197   74.635417   
              -----------+----------------------------------------------------
                   Total |   17878.875   199   89.843593 

margins prog  // view cell means

Adjusted predictions                              Number of obs   =        200

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prog |
          1  |   51.33333   1.287853    39.86   0.000     48.80919    53.85748
          2  |   56.25714   .8430976    66.73   0.000      54.6047    57.90958
          3  |      46.76   1.221764    38.27   0.000     44.36539    49.15461
------------------------------------------------------------------------------

pwcompare prog, mcompare(tukey) effects  // pairwise comparisons

Pairwise comparisons of marginal linear predictions

Margins      : asbalanced

---------------------------
             |    Number of
             |  Comparisons
-------------+-------------
        prog |            3
---------------------------

------------------------------------------------------------------------------
             |                              Tukey                Tukey
             |   Contrast   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prog |
     2 vs 1  |    4.92381   1.539279     3.20   0.005     1.288703    8.558916
     3 vs 1  |  -4.573333   1.775183    -2.58   0.029    -8.765543   -.3811241
     3 vs 2  |  -9.497143   1.484426    -6.40   0.000    -13.00271   -5.991573
------------------------------------------------------------------------------
Note: The tukey method requires balanced data for proper level coverage. A
      factor was found to be unbalanced.

With unbalanced data the tukey option in pwcompare produces the
Tukey-Kramer solution.

Next, we add the covariate to the model. The margins and pwcompare
commands take into account the covariate in the model to produces the cell means
and the pairwise differences.

anova write prog c.read
                           Number of obs =     200     R-squared     =  0.3925
                           Root MSE      = 7.44408     Adj R-squared =  0.3832

                  Source |  Partial SS    df       MS           F     Prob > F
              -----------+----------------------------------------------------
                   Model |  7017.68123     3  2339.22708      42.21     0.0000
                         |
                    prog |  650.259965     2  325.129983       5.87     0.0034
                    read |  3841.98338     1  3841.98338      69.33     0.0000
                         |
                Residual |  10861.1938   196  55.4142539   
              -----------+----------------------------------------------------
                   Total |   17878.875   199   89.843593  
                   
margins prog, asbalanced

Predictive margins                                Number of obs   =        200

Expression   : Linear prediction, predict()

------------------------------------------------------------------------------
             |            Delta-method
             |     Margin   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prog |
          1  |   52.50272   1.118549    46.94   0.000     50.31041    54.69504
          2  |   54.39898   .7599706    71.58   0.000     52.90946    55.88849
          3  |    49.6097   1.106984    44.82   0.000     47.44005    51.77934
------------------------------------------------------------------------------

pwcompare prog, mcompare(tukey) effects

Pairwise comparisons of marginal linear predictions

Margins      : asbalanced

---------------------------
             |    Number of
             |  Comparisons
-------------+-------------
        prog |            3
---------------------------

------------------------------------------------------------------------------
             |                              Tukey                Tukey
             |   Contrast   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
        prog |
     2 vs 1  |   1.896256   1.375278     1.38   0.354    -1.351677     5.14419
     3 vs 1  |  -2.893026   1.542866    -1.88   0.149    -6.536747    .7506945
     3 vs 2  |  -4.789282    1.39847    -3.42   0.002    -8.091988   -1.486576
------------------------------------------------------------------------------
Note: The tukey method requires balanced data for proper level coverage. A
      factor was found to be unbalanced.

Once again, with unbalanced data the tukey option in pwcompare produces the Tukey-Kramer solution.