13: Additional ANOVA Topics Post hoc comparisonsÍ Least squared differenceÍ The multiple comparisons problemÍ Bonferroni ANOVA assumptionsÍ Assessing equal varianceÍ When assumptions are severely violatedÍ Kruskal-Wallis Summary
Illustrative Examplep. 13.1 Pigment.sav - skin pigment levels in 4 families ofsame “race” (n 5 in each group) Mean pigment levelsÍÍÍÍGroup 1: 38.6Group 2: 46.0Group 3: 46.4Group 4: 52.4 EDA (right) ANOVA H0: :1 :2 :3 :4Í p . .000Í Reject nullÍ Which means differ?
Post Hoc Comparisonsp. 13.2 Which means differ?Í Are all four groups different from each other?Í Is there “one odd main out?” After the fact (post hoc) comparisonsÍ Apriori contrasts 6 pre-plannedÍ Posteriori 6 not planned Different philosophies for each
Illustrative Example (pigment.sav)p. 13.2 How may post hoc comparisons?In testing k groups, there are kC2 pairwise comparisons For illustrative example, k 4There are 4C2 4! / (2!)(4-2!) 6 pairwise comparisonsTest 1: H0: :1 :2Test 2: H0: :1 :3Test 3: H0: :1 :4Test 4: H0: :2 :3Test 5: H0: :2 :4Test 6: H0: :3 :4
Least Square Difference (LSD) Method(p. 13.2) Illustrative data, test 1: H 0: :1 :2 DataÍ xbar1 38.6Í xbar2 46.0Í Pooled estimate of variance s2W MSw (from ANOVA table) 12.350Í dfw (also from ANOVA table) N!k 20!4 16Convert to pdfw 20!4 16p .0042
LSD Output from SPSSp. 13.3Post hoc button during ANOVA procedure LSD check box
The Problem of Multiplicityp. 13.4 A man and a woman who sits and deals out adeck of cards . . . (John Tukey, p. 13.4) ConsiderÍ " .05 6 probability of correct retention .95Í In testing three correct null hypotheses at " .05, Pr(threecorrect retentions) .95 .95 .95 . .86Í Pr(at least one false rejection) 1!.86 .14Í This is the family-wise error rate Family-wise error rate increases with eachadditional testÍ For m tests, family-wise error rate at " .05 is 1!.95mÍ e.g., 20 tests, family-wise error rate 1!.9520 .65
Dealing w/ The Problem of Multiplicityp. 13.4 Depends on purpose of test For planned comparisonsÍ No adjustment necessaryÍ Proceed with LSD method For unplanned comparisonsÍ Make adjustments so family-wise error rate kept in checkÍ Many adjustment methods (see post hoc button in SPSS)Í We cover “Bonferroni”
Bonferroni’s Methodp. 13.4 - 13.6 LetÍ m number of comparisons Recall: m kC2 k! / 2!(k!2)!Í p p-value from LSD test pBonf p m Illustrative example, test 1 H0: :1 :2Í LSD derived p .0042Í There are four groups and six comparisonsÍ pBonf .0042 6 .025
SPSS Bonferroni Outputp. 13.5
ANOVA Assumptionsp. 13.6 Validity assumptionsÍ good selection (random representation of populations;related to “independence” assumption below)Í information accurate (no information bias)Í comparability of group in factors other than that whichidentifies groups (no confounding) Distributional assumptionsÍ independence (random samples from k populations)Í normality (of sampling distribution of means – centrallimit theorem)Í equal variance (homoscedasticity)
CommentsNot in Reader notes Validity assumptionsÍ Difficult to assess (counterfactual)Í Are of utmost importanceÍ Trump distributional assumptions Distributional assumptionsÍ Can be assessed via dataÍ Often talked aboutÍ Pale in importance compared to validity assumptions Moral dilemmasÍÍÍÍÍDo we pretend validity assumptions do not exist?Do we use limited time to fretting over distributional?Does expediency trump validity?Do we bother to defend distributional assumptions?Do we make the best of the situation?
Assessing Equal Variancep. 13.7 Compare standard deviations ( 2-fold difference in s?) Side-by-side boxplots (2-fold difference in hingespread?) F-ratio (2 groups) or Levene’s test (k groups)(A)Equal meansEqual variances(B)Unequal meansEqual variances(C)(D)Equal meansUnequal meansUnequal variances Unequal variances
Illustrative DataSmall samples!
Levene’s testp. 13.6SPSS One-way ANOVA Option (check “homogeneity ofvariance”) Illustrative example (pigment.sav)H0: F²1 F²2 F²3 F²4F 1.49 with 3 and 16 degrees of freedom (p .25)No significant evidence of unequal varianceBut then again, no evidence of equal variance either
Options When Assumptions areViolated Severelyp. 13.8 Descriptive analysis only Use ANOVA anyway Use more robust test (e.g., unequal variance ttests) Transform data (covered in Chap 15) Nonparametric testing
Kruskal-WallisÍ Nonparametric analogue to ANOVAÍ H0: population medians are equal vs. H1: H0 falseÍ Use SPSS Analyze Non-Parametric Tests kIndependent SamplesÍ Output chi-square stat, df, p valueÍ Interpret as other tests
Kruskal-Wallis Testpp. 13.8 - 13.9 Illustrative example: airsamples.savÍ Boxplot (right)Í F ratio test p .00018Í Conclude: variances unequal Kruskal-WallisÍ Does not require equal varianceÍ H0: population medians are equalÍ SPSS derives P²K-W 0.40 df 1 p .53Í Conclusion: no significant difference inmedians
The multiple comparisons problem Bonferroni ANOVA assumptions Assessing equal variance When assumptions are severely violated Kruskal-Wallis Summary. Illustrative Example p. 13.1 Pigment.sav - skin pigment levels in 4 families of same “race” (n