Multiple Comparison Methods for Means
Abstract
Multiple comparison methods (MCMs) are used to investigate differences between pairs of population means or, more generally, between subsets of population means using sample data. Although several such methods are commonly available in statistical software packages, users may be poorly informed about the appropriate method(s) to use and/or the correct way to interpret the results. This paper classifies the MCMs and presents the important methods for each class. Both simulated and real data are used to compare methods, and emphasis is placed on correct application and interpretation. We include suggestions for choosing the best method.
Mathematica programs developed by the authors are used to compare MCMs. By taking advantage of Mathematica's notebook structure, an interested student can use these programs to explore the subject more deeply. The programs and examples used in the article are available at http://www.cs.gasou.edu/faculty/rafter/MCMM/.
[1] , Statistics with Mathematica®, Academic Press Inc., 1999xiv+632, With 1 CD‐ROM (Macintosh, Windows, DOS and UNIX) [ISBN: 0‐12‐041555‐0] 99m:62006
[2] , Closure of the Newman‐Keuls multiple comparisons procedure, J. Amer. Statist. Assoc., 76 (1981), 241–245 82h:62119
[3] , The ANOVA and multiple comparisons for data with heterogeneous variances. (French summary), Biometrics, 30 (1974), 719–724 50:11648
[4] , Multiple range tests for correlated and heteroscedastic means, Biometrics, 13 (1957), pp. 164–176. bmt BIOMB6 0006-341X Biometrics
[5] , Estimation of the means of dependent variables, Ann. Math. Statist., 29 (1958), 1095–1111 21:399
[6] , A multiple comparison procedure for comparing several treatments with a control, J. Amer. Statist. Assoc., 50 (1955), pp. 1096–1121. jsk JSTNAL 0003-1291 J. Am. Stat. Assoc.
[7] , Pairwise multiple comparisons in the homogeneous variance, unequal sample size case, J. Amer. Statist. Assoc., 75 (1980), pp. 789–795. b3g JSTNAL 0162-1459 J. Am. Stat. Assoc.
[8] , Pairwise multiple comparisons in the unequal variance case, J. Amer. Statist. Assoc., 75 (1980), pp. 796–800. b3g JSTNAL 0162-1459 J. Am. Stat. Assoc.
[9] , A step‐up multiple test procedure, J. Amer. Statist. Assoc., 87 (1992), 162–170 1158635
[10] , A study of the powers of several methods of multiple comparisons, J. Amer. Statist. Assoc., 70 (1975), pp. 574–583. b3g JSTNAL 0162-1459 J. Am. Stat. Assoc.
[11] R. A. Fisher, The Design of Experiments, Oliver & Boyd, Edinburgh, 1935.
[12] , On the relation between union intersection and likelihood ratio tests. , Univ. of North Carolina Press, Chapel Hill, N.C., 1970, 251–266 42:8607
[13] , Pairwise multiple comparison procedures with unequal N’s and/or variances: A Monte Carlo study, J. Educ. Statist., 1 (1976), pp. 113–125. c7g ZZZZZZ 0362-9791 J. Educ. Stat.
[14] , A proof of the conjecture that the Tukey‐Kramer multiple comparisons procedure is conservative, Ann. Statist., 12 (1984), 61–75 85g:62124
[15] , A sharper Bonferroni procedure for multiple tests of significance, Biometrika, 75 (1988), 800–802 90d:62037
[16] , Multiple comparison procedures, Wiley Series in Probability and Mathematical Statistics: Applied Probability and Statistics, John Wiley & Sons Inc., 1987xxiv+450 89c:62125
[17] , Multiple comparisons, Chapman & Hall, 1996xiv+277, Theory and methods 99g:62102
[18] J. Neter, M. H. Kutner, C. J. Nachtshein, and W. Wasserman, Applied Linear Statistical Models, 4th ed., Irwin, Chicago, 1996.
[19] , Significance tests for multiple comparison of proportions, variances and order statistics, Psychol. Bull., 57 (1960), pp. 318–328 pbu PSBUAI 0033-2909 Psychol. Bull.
[20] L. E. Toothaker, Multiple Comparisons for Researchers, Sage, Newbury Park, CA, 1991.
[21] , Stepwise multiple comparison procedures, J. Amer. Statist. Assoc., 72 (1977), 566–575 56:7044
