One Way ANOVA Post-Hoc Test

In the previous tutorial on One Way ANOVA, we learnt how to know whether the mean difference between more than two groups is statistically significant.
However, until we run the Post-Hoc tests, we won't be able to tell, which two groups in specific within multiple groups had their means differed.
In SPSS, We can run Post-Hoc Tests using the following steps:

Step 1. identify the quantitative dependent variable and the categorical independent variable/factor (with more than two groups).

In this sample data, Job category is an independent variable or factor and salary is a dependent variable. So, I want to look which job categories differ in their mean salary.

we can see that there are three categories for job i.e. Clerical, Custodial and Manager.

Step 2:  Go to analyze, compare means and click on One way ANOVA

Step 3. Then you see the following dialog box.

Step 4. As shown in the above picture, move the current salary (which is a dependent variable) to the dependent list and employment category (which is an independent variable) to the Factor box.
After that, click on Post HoC

Step 5. Then you see the following dialog box with so many options:

Let me each of those options briefly:
Equal Variance Assumed:

·        LSD. Uses t tests to perform all pairwise comparisons between group means. No adjustment is made to the error rate for multiple comparisons.

·         Bonferroni. Uses t tests to perform pairwise comparisons between group means, but controls overall error rate by setting the error rate for each test to the experimentwise error rate divided by the total number of tests. Hence, the observed significance level is adjusted for the fact that multiple comparisons are being made.

·         Sidak. Pairwise multiple comparison test based on a t statistic. Sidak adjusts the significance level for multiple comparisons and provides tighter bounds than Bonferroni.

·         Scheffe. Performs simultaneous joint pairwise comparisons for all possible pairwise combinations of means. Uses the F sampling distribution. Can be used to examine all possible linear combinations of group means, not just pairwise comparisons.

·         R-E-G-W F. Ryan-Einot-Gabriel-Welsch multiple stepdown procedure based on an F test.

·         R-E-G-W Q. Ryan-Einot-Gabriel-Welsch multiple stepdown procedure based on the Studentized range.

·         S-N-K. Makes all pairwise comparisons between means using the Studentized range distribution. With equal sample sizes, it also compares pairs of means within homogeneous subsets, using a stepwise procedure. Means are ordered from highest to lowest, and extreme differences are tested first.

·         Tukey. Uses the Studentized range statistic to make all of the pairwise comparisons between groups. Sets the experimentwise error rate at the error rate for the collection for all pairwise comparisons.

·         Tukey's b. Uses the Studentized range distribution to make pairwise comparisons between groups. The critical value is the average of the corresponding value for the Tukey's honestly significant difference test and the Student-Newman-Keuls.

·         Duncan. Makes pairwise comparisons using a stepwise order of comparisons identical to the order used by the Student-Newman-Keuls test, but sets a protection level for the error rate for the collection of tests, rather than an error rate for individual tests. Uses the Studentized range statistic.

·         Hochberg's GT2. Multiple comparison and range test that uses the Studentized maximum modulus. Similar to Tukey's honestly significant difference test.

·         Gabriel. Pairwise comparison test that used the Studentized maximum modulus and is generally more powerful than Hochberg's GT2 when the cell sizes are unequal. Gabriel's test may become liberal when the cell sizes vary greatly.

·         Waller-Duncan. Multiple comparison test based on a t statistic; uses a Bayesian approach.

·         Dunnett. Pairwise multiple comparison t test that compares a set of treatments against a single control mean. When you select Dunnett, you will have two options: first category and last category. The last category is the default control category. Alternatively, you can choose the first category.

Equal variances not Assumed:

·         Tamhane's T2. Conservative pairwise comparisons test based on a t test. This test is appropriate when the variances are unequal.

·         Dunnett's T3. Pairwise comparison test based on the Studentized maximum modulus. This test is appropriate when the variances are unequal.

·         Games-Howell. Pairwise comparison test that is sometimes liberal. This test is appropriate when the variances are unequal.

·         Dunnett's C. Pairwise comparison test based on the Studentized range. This test is appropriate when the variances are unequal.

..................................................................................................

Step 6: So, selection of an option depends on your assumptions. You can test for homogenity for variance as well before applying it (see step 8 to see the option to test homogenity for variance). Let me chose Tukey in this example to show how it generates the output. You can also set your desired significance level. By default, it is set to 0.05

Step 7: click continue and you get the following dialog box.

Step 8. Click on Options, and you get the following dialog box.

Step 9: Click on Descriptives and means plot (if you want to show the descriptive statistics and means plot). You can even click on Homogenity of variance test to test whether variances can be assumed equal across groups. Click continue and click ok. You see the following output.

From the output, we can see that the mean difference in salary between the groups is statistically significant. The table on Post-Hoc Tests shows the double redundant table in which the mean difference between the salary of Clerical and Custodial is not statistically significant at 5% level of significance (P value=0.277). 

However, the mean difference between the salary of Clerical and manager is statistically significant (P=0.000) and so is for the Custodial and Manager (P=0.000).

The means plot also provisionally tells us that the mean salary of Clerical and Custodial categories don't significantly differ while the mean salaries between the "Clerical vs manager" and "Custodial vs Manager" significantly differ. But to confirm that, we would not tell confidently from the means plot alone. So, we do the Post Hoc tests.

Watch this video to see the steps.
 

P.S: I didn't go in every detail of the Post-Hoc tests but I hope that it gives you some basic understanding of comparing the means of more than two groups and identifying which groups/pairs in specific had their means differed.

Thank you and catch you in the next tutorial!!