Scheffé's method

In statistics, Scheffé's method, named after American statistician Henry Scheffé, is a method for adjusting significance levels in a linear regression analysis to account for multiple comparisons. It is particularly useful in analysis of variance (a special case of regression analysis), and in constructing simultaneous confidence bands for regressions involving basis functions.

Scheffé's method is a single-step multiple comparison procedure which applies to the set of estimates of all possible contrasts among the factor level means, not just the pairwise differences considered by the Tukey–Kramer method. It works on similar principles as the Working–Hotelling procedure for estimating mean responses in regression, which applies to the set of all possible factor levels.

The method

Let ${\textstyle \mu _{1},\ldots ,\mu _{r}}$ be the means of some variable in ${\textstyle r}$ disjoint populations.

An arbitrary contrast is defined by

C=\sum _{i=1}^{r}c_{i}\mu _{i}

where

\sum _{i=1}^{r}c_{i}=0.

If ${\textstyle \mu _{1},\ldots ,\mu _{r}}$ are all equal to each other, then all contrasts among them are $0$ . Otherwise, some contrasts differ from $0$ .

Technically there are infinitely many contrasts. The simultaneous confidence coefficient is exactly ${\textstyle 1-\alpha }$ , whether the factor level sample sizes are equal or unequal. (Usually only a finite number of comparisons are of interest. In this case, Scheffé's method is typically quite conservative, and the family-wise error rate (experimental error rate) will generally be much smaller than ${\textstyle \alpha }$ .)^[1]^[2]

We estimate ${\textstyle C}$ by

{\hat {C}}=\sum _{i=1}^{r}c_{i}{\bar {Y}}_{i}

for which the estimated variance is

s_{\hat {C}}^{2}={\hat {\sigma }}_{e}^{2}\sum _{i=1}^{r}{\frac {c_{i}^{2}}{n_{i}}},

where

${\textstyle n_{i}}$ is the size of the sample taken from the ${\textstyle i}$ ^th population (the one whose mean is ${\textstyle \mu _{i}}$ ), and
${\hat {\sigma }}_{e}^{2}$ is the estimated variance of the errors.

It can be shown that the probability is ${\textstyle 1-\alpha }$ that all confidence limits of the type

{\hat {C}}\pm \,s_{\hat {C}}{\sqrt {\left(r-1\right)F_{\alpha ;r-1;N-r}}}

are simultaneously correct, where as usual ${\textstyle N}$ is the size of the whole population. Norman R. Draper and Harry Smith, in their 'Applied Regression Analysis' (see references), indicate that ${\textstyle r}$ should be in the equation in place of ${\textstyle r-1}$ . The slip with ${\textstyle r-1}$ is a result of failing to allow for the additional effect of the constant term in many regressions. That the result based on ${\textstyle r-1}$ is wrong is readily seen by considering ${\textstyle r=2}$ , as in a standard simple linear regression. That formula would then reduce to one with the usual ${\textstyle t}$ -distribution, which is appropriate for predicting/estimating for a single value of the independent variable, not for constructing a confidence band for a range of values of the independent value. Also note that the formula is for dealing with the mean values for a range of independent values, not for comparing with individual values such as individual observed data values.^[3]

Denoting Scheffé significance in a table

Frequently, subscript letters are used to indicate which values are significantly different using the Scheffé method. For example, when mean values of variables that have been analyzed using an ANOVA are presented in a table, they are assigned a different letter subscript based on a Scheffé contrast. Values that are not significantly different based on the post-hoc Scheffé contrast will have the same subscript and values that are significantly different will have different subscripts (i.e. 15_a, 17_a, 34_b would mean that the first and second variables both differ from the third variable but not each other because they are both assigned the subscript "a").^{[citation needed]}

Comparison with the Tukey–Kramer method

If only a fixed number of pairwise comparisons are to be made, the Tukey–Kramer method will result in a more precise confidence interval. In the general case when many or all contrasts might be of interest, the Scheffé method is more appropriate and will give narrower confidence intervals in the case of a large number of comparisons.

References

^ Maxwell, Scott E.; Delaney, Harold D. (2004). Designing Experiments and Analyzing Data: A Model Comparison. Lawrence Erlbaum Associates. pp. 217–218. ISBN 0-8058-3718-3.
^ Milliken, George A.; Johnson, Dallas E. (1993). Analysis of Messy Data. CRC Press. pp. 35–36. ISBN 0-412-99081-4.
^ Draper, Norman R; Smith, Harry (1998). Applied Regression Analysis (2nd ed.). John Wiley and Sons, Inc. p. 93. ISBN 9780471170822.

Bohrer, Robert (1967). "On Sharpening Scheffé Bounds". Journal of the Royal Statistical Society. Series B. 29 (1): 110–114. JSTOR 2984571.
Scheffé, H. (1999) [1959]. The Analysis of Variance. New York: Wiley. ISBN 0-471-34505-9.

External links

Scheffé's method

This article incorporates public domain material from the National Institute of Standards and Technology

[MaxwellDelaney-1] Maxwell, Scott E.; Delaney, Harold D. (2004). Designing Experiments and Analyzing Data: A Model Comparison. Lawrence Erlbaum Associates. pp. 217–218. ISBN 0-8058-3718-3.

[MillikenJohnson-2] Milliken, George A.; Johnson, Dallas E. (1993). Analysis of Messy Data. CRC Press. pp. 35–36. ISBN 0-412-99081-4.

[3] Draper, Norman R; Smith, Harry (1998). Applied Regression Analysis (2nd ed.). John Wiley and Sons, Inc. p. 93. ISBN 9780471170822.

[1]

[2]

[3]