Jump to content

User:Reuqr/A necessary and sufficient condition for Simpson's paradox to occur

From Wikipedia, the free encyclopedia

We are considering the simplest case in which Simpson's paradox can occur, namely a 2 × 2 × 2 count table. A specific example might be a test of whether a new drug for a medical condition is more effective than a placebo. One starts with a set of individuals with the medical condition, and divides it into two groups. To all individuals in the first group one gives the drug (say there are individuals in this group), and to each individual in the other, the placebo (say there are individuals in that group). Then in each group one counts in how many individuals the medical condition improved (say and , respectively). If the rate of improvement for the drug group, , is greater than the rate of improvement in the placebo group, , then the experiment suggests that the drug is effective; otherwise we conclude that the drug is no more effective than the placebo. (In a real study, the case of the favorable outcome, , is not in and of itself enough to conclude that the drug is effective; one would at least also need to make sure that the probability that this happened by accident (i.e. the p-value) is small enough.)

But it could be that there is a confounding variable, which is something that can in principle influence the probability of improvement as much as what treatment one gets. For example, it may matter whether the medical condition is, say, early-stage or late-stage. So now one decides to stratify the sample, i.e. to consider separately the individuals with the early-stage and the late-stage condition. Suppose that in the drug group, there were individuals with the early-stage condition and individuals with the late-stage condition (); the condition improved in and individuals, respectively (). The corresponding numbers for the placebo group are , , , and . (Note that the data now consist of a total of 8 numbers, which can be thought of as organized into a 2 × 2 × 2 table.)

placebo drug
early stage Group 1
Group 2
late stage Group 3
Group 4
both stages



And now it can happen that whatever conclusion one reached about the effectiveness of the drug before the groups were stratified, the conclusion when the strata are considered separately could be the opposite: if that happens, one has Simpson's paradox. For example, it could be that if each stratum is considered separately, it appears that the drug is more effective than the placebo,

  and   (1)

even though the non-stratified data suggested that the drug is no better than the placebo:

(2)

We are now going to derive a necessary and sufficient condition for Simpson's paradox to occur. The trick is to rewrite each side of Eq. (2) as a weighted average of (equivalently, linear interpolation betwen) the quantities appearing in Eq. (1). In both D and L cases, this is done as follows:

now in the first term we multiply and divide by E, and in the second, by L:

(3)

note that . (So on may say that and are normalized weighted means). Thus, we rewrite Eq. (2) as

Editing

[edit]

(4)

while Eq. (1) becomes

  and  

(5)

How can Eq. (4) hold given that Eq. (5) does?

To see how this can be possible, one must first realize the following: if and , then the weighted average may be any number between and .

From this it should be obvious, to begin with, under what circumstances Eq. (4) cannot hold (no matter what weights—'s and 's—one chooses): it cannot hold if both of are greater than both of

Editing

[edit]

So, for Simpson's paradox to be possible, there must be some overlap between the "P" interval (i.e. the interval between and ) and the "D" interval (i.e. the interval between and ); see Fig. 1.

The "P" and "D" intervals shown as partly overlapping.
Figure 1. An example of how the "P" and "D" intervals could overlap and still be consistent with Eq. (5). Note that one could also have the 's be the lower ends of the intervals, and the 's on the higher.

If there is such an overlap, then we will have Simpson's paradox whenever most of the weight on the "P" side is given to the higher end of the interval, and most of the weight on the "D" side, to the lower end. Below we derive the precise condition for this to happen.

Note that Eq. (5) does allow this to happen: it just says what happens when "'s are compared with 's, and 's with 's" (namely, that the ␣s will be less than the ␣s). But it says nothing about what happens when "'s are compared with 's": it can happen that one of the ␣'s be greater than one of the ␣'s. (See Fig. 1. A numerical example:   and   , but )



To proceed, it will be convenient to change notation slightly so as to make manifest the ordering among the 's and 's. To start with, we will make it so that the smallest quantity is always denoted by . This is done as follows: given Eq. (5), the smallest is either or . If it is , then we replace all the 's by 's and all the 's by 's, and also all the 's by 's, and all the 's by 's. In short, and If is the smallest, then we do it the other way around, i.e. in the replacements scheme we switch places of and

Equation (5) becomes

  and    (with ),

(6)

while Eq. (4) becomes

(7)

To visualize this, think of as a line segment between and on a number line where the values increase to the right. First, have the "P" interval, between and namely, the interval

There is also the "D" interval, between and . Equation (6) does not settle which of these is to the right and which to the left. All it says is that lies entirely to the right of the "P" interval. But may, as far as Eq. (6) is concerned, lie anywhere to the right of the start of the "P" interval (i.e. to the right of ), including to the right of . However, in order for Simpson's paradox to be possible, the "P" and "D" intervals must overlap, i.e. should lie inside the "P" interval (i.e. the ordering must be ).

Editing

[edit]

similarly, we have the "D" interval. Equation (5) says that the beginning of the "P" interval (i.e. its left end) must be to the left of the beginning of the "D" interval, and that the end of the "P" interval (i.e. its right end) must be to the left of the end of the "D" interval. But there is still the possibility that the intervals overlap—that "D" interval begins before the "P" interval ends.


To proceed, we first need to figure out the ordering among the 's and 's. Given Eq. (5), the smallest is either or .

Let us relabel, in Eqs. (4) and (5), the 's and 's by 's and 's (and the 's and 's by 's and 's), but not necessarily respectively; we do it so that is the smallest. In other words, if is the smallest, then we replace all the 's by 's and all the 's by 's (also, all the 's by 's and all the 's by 's); if is the smallest, then we do it the other way around. Equation (5) becomes

  and   (with ),

(6)

and we write Eq. (4) as

(7)

Why we chose to use write in terms of on the left (i.e. ), but the other way around on the right, will become apparent in a moment. Note that Eq. (6) says that we have one key question is where in this ordering fits . We first show that one cannot have Simpson's paradox unless . First note that a linear interpolation , with , must lie at or between and , i.e.

(8)

Now if then, since , we have that

(9)

but, applying Eq. (8) to and , we get , i.e. the opposite of Eq. (7).

So let us therefore consider the case where . In that case we have .