A PECULIARITY OF THE WILCOXON-MANN-WHITNEY

RANK-SUM TEST Robert Dawson Saint Mary’s University Halifax, Nova Scotia Canada B3H 3C3

Keywords: intransitivity, rank-sum test, Behrens-Fisher problem

ABSTRACT

Pothoff showed in 1963 that the Wilcoxon-Mann-Whitney test is a test of relative location whenever the two distributions are symmetric. By contrast, Steinhaus and Trybula exhibited,in 1959, a set of three distributions for which the test would yield intransitive cycles. It is shown that the latter behaviour is usual even within Behrens-Fisher families of distributions. Sufficient conditions for the test to be acyclic, and hence a test for relative location, on a Behrens-Fisher family will be given.

1. INTRODUCTION AND DEFINITIONS

If a test of relative location, applied to samples from two populations, rejects the null hypothesis of equal location, it is usual to interpret the test by stating the apparent direction of the difference. This is, of course, not the same as performing a one-sided test; the experimenter has used the more conservative two-sided critical values {s_{α/2}, s_{1−α/2}}. The probability of observing such a result if the true state of affairs is not represented by the null hypothesis, but by the other tail of the alternative hypothesis, is far less than α/2, so it is legitimate to ignore this alternative.

Thus, even a two-sided test of relative location induces some sort of order relation on populations and on samples. If a test (with specified significance level) is applied to two samples {x_{i}} and {y_{i}}, the null hypothesis of equal location is rejected, and the value of the test statistic suggests the location on the real line of the {x_{i}} to be larger than that of the {y_{i}}, we will write {x_{i}} ≻_{t} {y_{i}} If, for iid samples {X_{i} : i = 1, 2, …, m} ∼ X and {Y_{i} : i = 1, 2, …, n} ∼ Y, P ({X_{i}} ≻_{t} {Y_{i}}) → 1 as m, n → ∞, we will write X ≻_{t} Y.

We will call a test transitive if, whenever {x_{i}} ≻_{t} {y_{i}} and {y_{i}} ≻_{t} {z_{i}}, we also have {x_{i}} ≻_{t} {z_{i}}. If whenever X ≻_{t} Y and Y ≻_{t} Z we have X ≻_{t} Z, we will call the test consistently transitive. If, when {x_{i}} ≻_{t} {y_{i}} and {y_{i}} ≻_{t} {z_{i}}, we never have {z_{i}} ≻_{t} {x_{i}}, the test will be called acyclic; and if X ≻_{t} Y ≻_{t} Z implies that P ({Z_{i}} ≻_{t} {X_{i}}) → 0 as the sample sizes m, n → ∞, we will call the test consistently acyclic. Clearly

transitive ⇒ consistently transitive
⇓  ⇓
acyclic ⇒ consistently acyclic .

For example, it may be verified that the two-sample Student’s t test is acyclic, but not transitive. (Suppose, for instance, that we have three samples, with n_{1} = n_{3} = 2, n_{2} = 100; \overline{x_{1}} = −2, \overline{x_{2}} = 0, \overline{x_{3}} = 2; and s_{1} = s_{2} = s_{3} = 1. We would reject μ_{1} = μ_{2} and μ_{2} = μ_{3} at the 5% significance level, but not μ_{1} = μ_{3}.)

We may also consider whether a test is consistently transitive, or consistently acyclic, for a restricted family of distributions. As the probabilities depend on the distributions, a test which is consistently acyclic for one family of distributions may not be so for another. However, it is obvious that if a test is consistently transitive, or consistently acyclic, for a family of distributions, it is so for any subfamily.

Intransitivity and cyclicity have a long history in political science and probability theory. De Condorcet (1785) showed that cyclic patterns could occur in voting preferences, and Arrow showed more recently that such situations could not be avoided within non-dictatorial voting systems. Steinhaus and Trybula (1959) showed that the “stochastically greater” relation P (X > Y) > 1/2) between random variables is cyclic, and found the best possible upper bound on min{P (X > Y), P (Y > Z), P (Z > X)}.

In statistics, Gabriel (1964) used the term “transitive” in 1964 to describe simultaneous testing procedures which, if used to test hypotheses H and I such that the truth of H logically implies that of I, would always reject H if I was rejected. This is similar to our usage here; if a transitive test rejects (θ_{X} ≤ θ_{Y}) ∨ (θ_{Y} ≤ θ_{Z}) with a certain set of samples, it must reject θ_{X} ≤ θ_{Z} also. However, in Gabriel(1969), p.229, this property was renamed, the name coherence being considered more appropriate in the general case. The definitions are not identical; we do not assume, as did Gabriel, that the hypotheses are stated in terms of sets of parameter values. Our definition then includes, for instance, the interpretation in Mann and Whitney(1947) of the rank-sum test as a test of whether one random variable is stochastically larger than another.

Lehmann (1957) defined a family of tests (again, with parametric hypotheses) to be compatible if, with probability 1, the combined outcomes of the tests yield at least one possible set (θ_{1}, …, θ_{n}) of parameter values. Suppose that (θ_{1}, …, θ_{n}) are location parameters for different random variables, and that we test the null hypotheses H_{0ij} : θ_{i} = θ_{j}. If we interpret a rejected null hypothesis as implying whichever of θ_{i} < θ_{j}, θ_{j} < θ_{i} is compatible with the value of the test statistic, and do not consider a failure to reject as restricting the possible values of (θ_{1}, …, θ_{n}), then the testing procedure is compatible if and only if it is acyclic.

A test which is not consistently acyclic for a family of distributions is not a reliable test for any real-valued location parameter on that family. For, as the ordering on the real numbers is transitive, we cannot have θ_{X} > θ_{Y} > θ_{Z} > θ_{X} although we have X ≻_{t} Y ≻_{t} Z ≻_{t} X. Therefore, there must be pairs of distributions X, Y with θ_{X} = θ_{Y} but X ≻_{t} Y. When the test is applied to samples with such distributions, α → 1 as the sample size approaches ∞.

2. THE RANK-SUM TEST CAN BE CYCLIC

The Wilcoxon-Mann-Whitney rank-sum test, introduced by Wilcoxon(1945) and in a different but equivalent form by Mann and Whitney (1947), is not consistently acyclic. As observed by Steinhaus and Trybula (1959), there exist triples of random variables X, Y, Z such that P (X > Y), P (Y > Z), and P (Z > X) are all greater than 1/2. It follows that, as the sample sizes approach ∞, the probability that the rank sum test will order these three variables cyclically approaches 1.

Probably the best-known example of this phenomenon is due to Efron (in Gardner (1970)), who gave an example of a set of four dice which generate such random variables. In fact, one needs only three dice, labelled for instance {1, 1, 5, 5, 5, 5}, {3, 3, 3, 4, 4, 4, }, and {2, 2, 2, 2, 6, 6}.

It has been claimed in the literature (David (1963), p.19) that:

...the “paradox”...cannot occur if y_{1}, y_{2}, y_{3} have continuous distributions, since the relative locations of the medians then determine whether a particular preference probability is greater than, equal to, or less than 1/2.

However, this is not the case; the discrete random variables associated with Efron’s dice may easily be modified to yield continuous distributions with the same behaviour. The true resolution of the “paradox” is that the rank-sum test is not, in general, a test for the location of the median.

For a shift alternative, the rank-sum test is consistently transitive, for, if F_{X}(x) = F_{Y} (x + d), P (X > Y) > 1/2 if and only if d > 0. It may be noted that it is not always transitive, or even acyclic; it is possible that three specific random samples, drawn from populations differing only by a shift, may exhibit the rank-sum intransitivity paradox. However, this becomes improbable as the sample size increases. Potthoff (1963) has shown, moreover, that within the family of symmetric distributions the rank-sum test is a test for the location of the center of symmetry, and is thus consistently transitive.

If we want to test for differences of location within a family of distributions which may differ in both position and scale, so that any two of them are related by a relation F_{X}(x) = F_{Y} (ax + d) for a > 0, we are studying a type of Behrens-Fisher problem. For the rank-sum test to be consistently acyclic on a Behrens-Fisher family of continuous distributions, it is necessary and sufficient that whenever X, Y are distributions in the family with the same median, P (X > Y) = 1/2. For suppose that this is not the case; say P (X > Y) > 1/2. Then, by continuity, there exists ϵ > 0 such that P ((X − ϵ) > (Y + ϵ)) > 1/2. Let Z be an extremely narrow distribution in the same family, with the same median as X and Y; it is clear that P (Z > (X − ϵ)) > 1/2 and P (Y + ϵ) > Z) > 1/2. So, with probability 1, (X − ϵ) ≻_{t} (Y + ϵ) ≻_{t} Z ≻_{t} (X − ϵ) if the sample size is sufficiently large.

It is therefore necessary and sufficient that, for a member of the family with median 0 and pdf f_{X}, we should have

∫_{−∞}^{∞} ∫_{−∞}^{x} f_{X}(x)af_{X}(ay) dy dx = 1/2 ∀a > 0 .

Suppose f_{X} to be analytic, with all moments existing. We have, by a change of variable,

∫_{−∞}^{∞} ∫_{−∞}^{x} f_{X}(x)f_{X}(ay) ady dx = ∫_{−∞}^{∞} ∫_{−∞}^{ax} f_{X}(x)f_{X}(y) dy dx.

Therefore, differentiating with respect to a,

∫_{−∞}^{∞} xf_{X}(x)f_{X}(ax) dx = 0 ∀a > 0  (1)

This integral is convergent, as xf_{X}(x) is bounded.

Differentiating again with respect to a, we obtain

∫_{−∞}^{∞} x^{2}f_{X}(x)f′_{X}(ax) dx = 0 ∀a > 0.

In the limit as a → 0 this yields

∫_{−∞}^{∞} x^{2}f_{X}(x) dx f′_{X}(0) = 0.

As f_{X} is a pdf, the integrand is always nonnegative and the integral is positive; so we conclude that f′_{X}(0) = 0.

Repeating this differentiation 2, 4, 6, … times more, we see that f_{X}^{(2n+1)}(0) = 0, and we conclude that if f_{X} is analytic, and not too heavy-tailed, it is symmetric about its median. Thus there are no Behrens-Fisher families of such distributions for which the rank-sum test is a test of location except for those — the symmetric families — given by Potthoff (1963).

If we do not make such strong assumptions about f_{X}, there are other such families. A simple example, which is easily shown to satisfy (1) is the Behrens-Fisher family containing

f_{X}(x) = \{ \begin{matrix}0 : & x < −1/2 \\ 1 : & −1/2 ≤ x < 0 \\ 1/2 : & 0 ≤ x < 1 \\ 0 : & 1 ≤ x \end{matrix} .

More generally, if we write f_{X}(x) = f^{+}(x) + f^{−}(−x) where

f^{+}(x) = \{ \begin{matrix}0 : & x < 0 \\ f_{X}(x) : & x ≥ 0 \end{matrix}
f^{−}(x) = \{ \begin{matrix}0 : & x < 0 \\ f_{X}(−x) : & x ≥ 0 \end{matrix}

then (1) holds if and only if

∫_{0}^{∞} xf^{+}(x)f^{+}(ax) dx = ∫_{0}^{∞} xf^{−}(x)f^{−}(ax) dx ∀a > 0 .

If we let g^{±}(x) = e^{x}f^{±}(e^{x}), this reduces to

∫_{−∞}^{∞} g^{+}(x)g^{+}(x + c) dx = ∫_{−∞}^{∞} g^{−}(x)g^{−}(x + c) dx ∀c  (2)

— that is, the autocorrelations of g^{+} and g^{−} must be the same. The bestknown solutions to this functional equation are of the form g^{−}(x) = g^{+}(c±x). After transforming back, these correspond to

f^{−}(x) = kf^{+}(kx)  (3)

(when g^{−} is a translation of g^{+}) or

f^{−}(x) = (k/x^{2})f^{+}(k/x)  (4)

(when it is a reflection).

There are, in general, many more functions which have a given autocorrelation - it is necessary and sufficient for the absolute values of the Fourier transforms to be the same everywhere. Examples which are everywhere nonnegative, and hence correspond to probability densities, can be constructed using convolution.

The rank-sum test is consistently acyclic for any Behrens-Fisher family of continuous distributions satisfying (2). However, it must be said that this does not appear to include many useful families except for the symmetric ones. To be continuous at x = 0, an asymmetric distribution satisfying (3) must have f_{X}(0) = 0, and be bimodal; while a continuous distribution satisfying (4) must either have f_{X}(0) = 0 or lim_{x→∞} x^{2}f_{X}(x) = f_{X}(0) > 0.

3. PRACTICAL IMPLICATIONS - OR NOT?

Potthoff’s results imply that the rank-sum test is consistently acyclic within many widely-used families of distributions, such as the normal and uniform distributions. Moreover, it is consistently acyclic within any family, such as the lognormal distributions, which is a monotone transformation of a symmetric family (Behrens-Fisher or otherwise). Thus,in many cases, the pathology does not arise.

It should be noted, moreover, that cyclicity will rarely be observed in rank-sum tests of actual data. Steinhaus and Trybula (1959) have shown that

min\{P (X > Y), P (Y > Z), P (Z > X)\} ≤ τ^{−1} = (\sqrt{5} − 1)/2 ≈ 0.618

for any distributions X, Y, Z whatsoever. This implies that for any three samples {X_{i}}, {Y_{i}}, {Z_{i}} of size n, we have

min{W_{XY} , W_{Y Z}, W_{ZX}} ≤ τ^{−1}n^{2} .

For a two-tailed test at the 5% significance level, this is not statistically significant for n < 48. Experiment bears this out; if {X_{i}} contains 19 values close to 1 and 31 close to 4, {Y_{i}} contains 50 values close to 3, and {Z_{i}} contains 31 values close to 2 and 19 close to 5, all three rank-sum tests will reject the null hypothesis in favor of a two-sided alternative at the 5% significance level, in a cyclic fashion; but attempts to reduce the sample sizes fail.

For random samples from commonly encountered distributions, the phenomenon is even more elusive. To illustrate this, we consider the case of the shifted exponential family of distributions, with f_{X}(x) = e^{−(x/a+b)}/a for x > a and 0 for x < a. Suppose we have three random variables X, Y,and Z within this family, such that min{P (X > Y), P (Y > Z), P (Z > X)} > 1/2. At least one of X̃ ≤ Ỹ,Ỹ ≤ Z̃, and Z̃ ≤ X̃ must hold; let us suppose the first without loss of generality. Then we have P (X < Y) ≤ P ((X − X̃) < (Y − Ỹ)) ≤ max(P (U < V)) where the maximum is over all pairs of shifted exponential distributions with the same median.

To find this maximum, we take (without loss of generality) one distribution to have

f_{U}(x) = \{ \begin{matrix}e^{−(ln(2)+x)} & if x > − ln(2) \\ 0, & otherwise, \end{matrix}

and the other to have

f_{V} (x) = \{ \begin{matrix}e^{−(ln(2)+x/a)}/a & if x > −a ln(2) \\ 0, & otherwise, \end{matrix}

with a > 1.

Then

P (U < V) = ∫_{− ln(2)}^{∞} e^{−(ln(2)+x)} ∫_{−a ln(2)}^{x} e^{−(ln(2)+y/a)}/a dy dx
= 1 − (1 + \frac{1}{a})2^{(1−1/a)}.

Differentiating this last expression and setting it equal to 0, we find that it achieves its maximum for a = ln(2)/(1 − ln(2)) = 2.25889 …, and this maximum has the value 1 − e ln(2)/4 = 0.52895765 . . .. This is then an upper bound on min{P (X > Y), P (Y > Z), P (Z > X)}.

Now, for large samples, the upper two-sided 5% critical value for W_{XY} is approximately n^{2}/2 + z_{0.975}\sqrt{(2n^{3} + n^{2})/12}. Using the upper bound found above, we can calculate that

min{W̃_{XY} , W̃_{Y Z}, W̃_{ZX}} ≈ min{E(W_{XY}), E(W_{Y Z}), E(W_{ZX})}
= n^{2} min{P (X > Y), P (Y > Z), P (Z > X)}

will be less than this critical value for all n less than about 800. Therefore, for sample sizes less than this, at least one of the three tests (performed at the 5% significance level) will have power less than 50%. This implies that while, in theory, there exist triples of shifted exponential distributions which are consistently cyclically ordered by the rank-sum test, the effect is weak enough that it will require rather large sample sizes for this behaviour to manifest itself frequently.

4. SUMMARY

We have seen that the rank-sum test can potentially exhibit a pathology in which the implied ordering on three samples is cyclic, and hence incompatible with the ordering of any real-valued measure of location. In particular, almost all Behrens-Fisher families of distributions contain triples such that this pathology occurs among them with probability approaching 1 for large sample size. However, the required sample size is usually large enough that the pathology will rarely be observed in practice.

5. ACKNOWLEDGEMENT This work was supported by a grant from NSERC Canada.

BIBLIOGRAPHY

[1] M.J.A.N.C de Condorcet(1785). Essai sur l’application de l’analyse à la probabilité des décisions rendues à la pluralité des voix, Paris

[2] H.A. David (1963). The Method of Paired Comparisons,London, Charles Griffin & Company.

[3] K. R. Gabriel (1964). “A procedure for testing the homogeneity of all sets of means in analysis of variance”, Biometrics 20 494-477

[4] K.R. Gabriel (1969). “Simultaneous test procedures - some theory of multiple comparisons” Ann. Math. Stat. 40 224-250

[5] M. Gardner (1970). “The paradox of the nontransitive dice and the elusive principle of indifference”, Scientific American 223 110-114

[6] E.L. Lehmann (1957). “A theory of some multiple decision problems, I” Ann. Math. Stat. 28 1-25

[7] E.L. Lehmann, (1975). Nonparametrics , San Francisco, Holden-Day,

[8] H. Mann and D.R. Whitney (1947). “On a test of whether one of two random variables is stochastically larger than the other”, Ann. Math. Statist. 18 50-60

[9] R.F. Potthoff (1963). “Use of the Wilcoxon statistic for a generalized Behrens-Fisher problem”, Ann. Math. Stat. 34 1569-1599

[10] H. Steinhaus and S. Trybula (1959). “On a paradox in applied probabilities”, Bul. Acad. Polon. Sci. 7, 67-69

[11] F. Wilcoxon (1945). “Individual comparisons by ranking methods”, Biometrics 1 80-83