Draft:Dunn's test

Dunn's test is a non-parametric post hoc procedure used to identify which specific pairs of groups differ significantly following a significant result from the Kruskal–Wallis test. It was introduced by Olive Jean Dunn in 1964.

Because the Kruskal–Wallis test is an omnibus test, a significant result only indicates that at least one of three or more independent groups differs from the others; it does not indicate which particular pair or pairs are responsible. Dunn's test addresses this by performing pairwise comparisons of mean ranks while controlling the family-wise error rate across all comparisons made.

Background

The Kruskal–Wallis test, developed by William Kruskal and W. Allen Wallis, is the non-parametric analogue of the one-way analysis of variance (ANOVA) and is used to compare three or more independent groups when the assumptions of ANOVA — particularly normality and homoscedasticity — are not met. As with the parametric Tukey's HSD test following a significant ANOVA F-test, a significant Kruskal–Wallis result is typically followed by a post-hoc procedure to localize the source of the difference. Dunn's test serves this role for the rank-based, non-parametric case.

Formula

Dunn's test reuses the combined rank sums computed for the Kruskal–Wallis test. For a set of k independent groups drawn from a pooled sample of N observations, let $R_{j}$ denote the sum of ranks in group j and $n_{j}$ its sample size. The mean rank of group j is

${\bar {R}}_{j}={\frac {R_{j}}{n_{j}}}$

For every pair of groups i and j, the test statistic is

$z_{ij}={\frac {{\bar {R}}_{i}-{\bar {R}}_{j}}{\sqrt {{\dfrac {N(N+1)}{12}}\left({\dfrac {1}{n_{i}}}+{\dfrac {1}{n_{j}}}\right)}}}$

Under the null hypothesis that groups i and j come from the same distribution, $z_{ij}$ is approximately standard normal, and a two-tailed p-value is obtained from the standard normal cumulative distribution function. When tied ranks span multiple groups, the denominator is corrected using the same tie-correction factor applied in the Kruskal–Wallis H statistic.

Because ${\binom {k}{2}}$ comparisons are performed simultaneously, the raw p-values are not compared directly against a significance threshold; a multiple-comparison adjustment is applied first.

Multiple comparison correction

The two adjustments most commonly paired with Dunn's test are:

• Bonferroni correction: each raw p-value is multiplied by the number of comparisons m (capped at 1), equivalent to comparing the raw p-value against $\alpha /m$. This guarantees the family-wise error rate does not exceed $\alpha$ but is conservative, since every comparison is penalized equally regardless of its individual evidence.
• Holm's step-down procedure: the m raw p-values are sorted from smallest to largest, and the r-th smallest is compared against $\alpha /(m-r+1)$, with testing stopping at the first non-rejection. Holm's method controls the same family-wise error rate as Bonferroni correction but is uniformly at least as powerful.

Other adjustments, such as the Benjamini–Hochberg procedure, are also used in some implementations.

Assumptions

As a rank-based procedure applied to the same data as the Kruskal–Wallis test, Dunn's test inherits that test's assumptions:

• the groups being compared are independent;
• observations within each group are independent of one another;
• the data are measured on at least an ordinal scale.

Dunn's test is conventionally applied only after the Kruskal–Wallis test has returned a statistically significant result, since pursuing pairwise comparisons following a non-significant omnibus test increases the risk of false discoveries without a prior signal justifying the search.

Interpretation

Each pairwise comparison is evaluated independently against the chosen significance level (commonly $\alpha =0.05$), using the adjusted rather than the raw p-value. It is possible, and not uncommon, for the Kruskal–Wallis test to return a significant result while several individual pairs fail to reach significance under Dunn's test — reflecting the loss of statistical power that multiple-comparison correction imposes on each pair.

Comparison with related procedures

• Nemenyi test: the analogous post-hoc procedure following a significant Friedman test, used for related (repeated-measures) rather than independent groups.
• Tukey's HSD test: the parametric analogue used after a significant one-way ANOVA, comparing group means directly under normality and equal-variance assumptions rather than mean ranks.
• Repeated Mann–Whitney U tests with Bonferroni correction: an alternative in which each pair is ranked independently rather than from a single combined ranking; Dunn's test is generally preferred because it uses a consistent rank structure across all comparisons.

Software implementations

Dunn's test is implemented in a number of statistical software packages, including the R packages dunn.test and PMCMRplus, and the Python package scikit-posthocs, which provides the function posthoc_dunn with a choice of p-value adjustment method.

See also

• Kruskal–Wallis one-way analysis of variance
• Post-hoc analysis
• Multiple comparisons problem
• Holm–Bonferroni method
• Bonferroni correction
• Nemenyi test

Further reading

• Hollander, Myles; Wolfe, Douglas A.; Chicken, Eric (2013). Nonparametric Statistical Methods (3rd ed.). Wiley. doi:10.1002/9781119196037.
• Bruce, Peter; Bruce, Andrew; Gedeck, Peter (2020). Practical Statistics for Data Scientists (2nd ed.). O'Reilly Media.

External links

• Dunn's Post-Hoc Test Explained: Formula, Assumptions & Python Example — a tutorial with fully worked numerical examples and a Python (scikit-posthocs) walkthrough.
• Kruskal–Wallis Test — companion explainer on the omnibus test that Dunn's test follows.

Category:Statistical tests Category:Multiple comparisons Category:Non-parametric statistics