Talk:Rule of three (statistics)

Requested move

The following discussion is an archived discussion of a requested move. Please do not modify it. Subsequent comments should be made in a new section on the talk page. Editors desiring to contest the closing decision should consider a move review. No further edits should be made to this section.

The result of the move request was: moved per request. Favonian (talk) 22:07, 4 January 2013 (UTC)

---

Rule of three (medicine) → Rule of three (statistics) – to be more accurately descriptive of content and less restrictive on the scope of the article 75.172.49.186 (talk) 16:49, 28 December 2012 (UTC)

• Oppose – until the article has references showing the use of this concept outside of medicine, I don't see a reason to retitle it. Dicklyon (talk) 19:18, 28 December 2012 (UTC)

An excellent reason to oppose. You might like to reconsider though, given the content of the "Professor Mean" reference, and any number of others that could be shown. See below. Noetica^Tea? 02:40, 1 January 2013 (UTC)

• Support. I have now researched this closely. (See a section below on this talkpage; and see recent changes to the article.) I am satisfied that the rule is applied in statistical inference generally, even though it is especially well suited to inferences in medical and pharmaceutical research. Some recent sources found in Google books:

Published in 2006
Published in 2011

See also my added note in the article. There is a distinct meaning for the term in statistics, but it does not yet have its own WP article. Until it does, this article deserves the title Rule of three (statistics). Noetica^Tea? 02:40, 1 January 2013 (UTC)

• Support The relationship is easily derived in a way that has nothing to do with medicine - just probability. Confidence intervals and event-outcome trials are found in many contexts outside of medicine. —BarrelProof (talk) 04:23, 2 January 2013 (UTC)

The above discussion is preserved as an archive of a requested move. Please do not modify it. Subsequent comments should be made in a new section on this talk page or in a move review. No further edits should be made to this section.

Proposed rewrite of the lead

There are very many errors, bad links, and confusions generally in the lead:

In the statistical analysis of clinical trials, the rule of three states that if no major adverse events occurred in a group of n people, then the interval from 0 to 3/n can be used as a 95% Confidence interval that for the probability that a corresponding major event will arise for a single new individual. This is an approximate result, but is a very good approximation when n is greater than 30.

For example, in a trial of a drug for pain relief in 1500 people, none have a major adverse event. The rule of three says that 0 to 1 in 500 people is a 95% confidence interval for the rate of adverse events.

This rule is useful in the interpretation of drug trials, particularly in phase 2 and phase 3, which frequently do not have the statistical power or duration to find the relationship between the intervention and adverse events. They are designed to test the efficacy of a drug, and often the discovery of adverse events is not in the interests of the sponsors.

It should also be noted that this rule applies equally well to any trial done n times. It need not refer to medical or clinical settings. For example, if testing parachutes from the same batch, you test 300 and they all open successfully, the chance of another parachute from the same batch failing to open is likely to be less than 3/300, i.e. less than 1 in 100.

I propose this improvement:

In the statistical analysis of clinical trials, the rule of three states that if a certain event did not occur in n subjects, 0 to 3/n is a 95% confidence interval for the probability that no such eventsuch an event will occur in some randomly chosen new subject. When n is greater than 30, this is a good approximation to results from more sensitive tests.

For example, in a trial of a pharmaceutical drug for pain relief in 1500 people, there is no adverse event for any subject. From the rule of three, it can be concluded with 95% confidence that there is less than 1 chance in 500 that any given person will experience an adverse event (or equivalently, that fewer than 1 person in 500 will experience an adverse event).

The rule is useful in the interpretation of drug trials, particularly in phase II and phase III where statistical power and duration are sometimes less than ideal. The rule of three applies beyond medical research, to any trial done n times. If 300 parachutes from the same batch are randomly tested and all open successfully, then it is concluded with 95% confidence that the probability of failure in any other given parachute from the same batch is less than 3/300 (less than 1 in 100).

I'm 99% confident that this is indeed an improvement, but also that it needs more work. These are highly technical concepts, and great precision is needed. Advice would be welcome, from someone closer to their detailed application.

Please do not alter my proposed text above, but show suggestions in a new draft.

Noetica^Tea? 23:31, 29 December 2012 (UTC)

I think it's still not quite right. "0 to 3/n is a 95% confidence interval for the probability that no such event will occur in some randomly chosen new subject" sounds like it's saying the probably of NOT getting the adverse event is low. It should say the there's 95% confidence in the probability of the event being below 3/n. I'm not sure it makes sense to say "in some randomly chosen new subject"; maybe, depending on how probabilities are conceptualized here (or in medicine). I'm not in either stats or medicine, so I don't know how they prefer to express these things, but I think it's not there yet. Your second and third paragraphs look clear and correct to me. Dicklyon (talk) 00:28, 30 December 2012 (UTC)

Ah yes; I've struck and fixed that slip. As for "in some randomly chosen new subject", that may look awkward; but it or something equally wordy is needed. For example, formulations like these fail: "in some new subject"; "in a new subject". They do not restrict the probability in question to just one subject, but that is essential. More accurate still: "in the case of any single randomly chosen new subject". There is something to be said for the existing wording, just here: "for a single new individual". But I was worried about "a" in that, and "individual". In sum, I now think this may be better for the opening sentence:

In the statistical analysis of clinical trials, the rule of three states that if a certain event did not occur in n subjects, 0 to 3/n is a 95% confidence interval for assignment of a probability, for any single new subject, that such an event will occur.

As I see it, confidence interval as used here involves a kind of second-order probability: probability concerning accurate assignment of probabilities. And all of that connects with philosophical and expository difficulties with objective versus subjective probability. Let's see what others will say.

Noetica^Tea? 01:10, 30 December 2012 (UTC)

Better; I'm still not sure it's the best way to express it. Checking the link to confidence interval, I can agree that the 95% is correctly portrayed, though it's almost looking like it's saying there's a 95% change of getting an event at some rate in that interval; I guess that's true, as the interval includes zero and has a low upper bound. But focusing on an individual still seems awkward; maybe "the probability that such an event will occur in each randomly chosen new subject"? Dicklyon (talk) 01:41, 30 December 2012 (UTC)

I was going by the content of the lead as it now stands, for that focus on the individual subject. Having looked at some sources, I now think that content is not sound. It is better to stick with confidence intervals for estimation of population parameters, so that there is no trouble with second-order probabilities. In other words, this from my draft is good:

fewer than 1 person in 500 will experience an adverse event

And this, though it is equivalent, is problematic because it extends beyond population parameters:

there is less than 1 chance in 500 that any given person will experience an adverse event

By the way, your restatement falls into the same trap as other statements:

the probability that such an event will occur in each randomly chosen new subject

That can still be read as concerning a probability aggregated over all randomly chosen new subjects! The order is crucial. This is different, and better:

the probability for each randomly chosen new subject that such an event will occur

I will wait to see what other comments turn up. If none do, I might reword the whole lead in terms of population parameters – with perhaps one cautious mention of an equivalence in probability for "a single new subject".

Noetica^Tea? 02:18, 30 December 2012 (UTC)

While the above is an improvement, I think we need something dead simple for the lead that can be understood by non-statiticians. This description I think provides a good model. For example, if 100 hundred individuals are tested and no adverse event are found, there is less than 3/100 (3%) probability than an adverse event will be found if a larger group of individuals are tested. A more precise and detailed description can then be included later in the article. Boghog (talk) 10:40, 31 December 2012 (UTC)

That's an interesting link, Boghog. But there are confusing and misleading ways of stating things there. And I'm afraid your sentence is misleading also, because you don't make it clear that the probability in question concerns each and every new case in isolation, among the new ones yet to be examined. An excerpt from your linked page:

It says that if you’ve tested N cases and haven’t found what you’re looking for, a reasonable estimate is that the probability is less than 3/N. So in our proofreading example, if you haven’t found any typos in 20 pages, you could estimate that the probability of a page having a typo is less than 15%. In the perfect pitch example, you could conclude that fewer than 3% of children have perfect pitch.

What I have underlined we can all reliably understand immediately. Contrast this, in what precedes: "if you haven’t found any typos in 20 pages, you could estimate that the probability of a page having a typo is less than 15%." No! It should be "the probability for any given page that it has a typo", to exclude this misreading: "the probability that at least some page or other has a typo". See the difference? The intended meaning is simple enough; but showing that meaning lucidly is the hard part.

Where exposition is lucid and not misleading ("that fewer than 3% of children have perfect pitch"), there is talk of proportions of the relevant population, rather than of probabilities. As I concluded before. From print sources I have seen, I still think that is better.

Three new points:

• Certainly the scope of the rule goes beyond medicine, and the article should be moved, except that:
• Relatedly, and awkwardly, there are competing mathematical uses of "rule of three", including one in statistics that also turns up in analysis of medical data and the like. It involves acceptance of three standard deviations as giving suitable confidence intervals for practical inferences. See this fascinating source, for example. But look at the entries at our DAB page Rule of Three, where that three-SD meaning is not to be found.
• I tried to move that DAB page to Rule of three, but could not because the destination already exists as a redirect. WP:MOSCAPS and the listed articles plainly support lower case.

Let's see what more will be said.

Noetica^Tea? 11:35, 31 December 2012 (UTC)

Your right. I should have been more careful when I wrote that. Something along the lines "if 100 hundred individuals are treated and no adverse events are observed, there is less than a 3 in 100 chance (3%) that an adverse event will be found in any treated individual" is probably better. I agree that the scope of the concept goes beyond medicine. Does this concept have a less ambiguous name? Boghog (talk) 14:08, 31 December 2012 (UTC)

Noetica, I think you've got it figured out. The examples are simple enough, and statement of the rule precise enough, I think. Go for it. Dicklyon (talk) 17:38, 31 December 2012 (UTC)

My thanks to Dicklyon and Boghog. I am about to replace the present lead with this very careful version:

In statistical analysis, the rule of three states that if a certain event did not occur in a sample with n subjects, 0 to 3/n is a 95% confidence interval for the rate of occurrences in the population. When n is greater than 30, this is a good approximation to results from more sensitive tests.

For example, a pain-relief drug is tested on 1500 human subjects, and no adverse event is recorded. From the rule of three, it can be concluded with 95% confidence that fewer than 1 person in 500 (or 3/1500) will experience an adverse event.

The rule is useful in the interpretation of clinical trials generally, particularly in phase II and phase III where often there are limitations in duration or statistical power. The rule of three applies well beyond medical research, to any trial done n times. If 300 parachutes are randomly tested and all open successfully, then it is concluded with 95% confidence that fewer than 1 in 100 parachutes of the exactly the same specifications (3/300) will fail.

All links have been considered and checked; and all expression has been made as reader-friendly as I could manage without compromising precision. In accord with the best sources, probability is now not mentioned at all.

Noetica^Tea? 01:20, 1 January 2013 (UTC)

The graph is uninformative

The figure suggests that the error in the approximation is small. That is done in words in the text.

After x=10^1 on the x-axis the lines are unable to be distinguished. This communicates no sense of the error - it is wasted space. This makes the right 75% of the figure non-informative.

A better figure would have absolute relative error, or its logarithm base-10, for the 95% confidence bound displayed on the y-axis. — Preceding unsigned comment added by 144.191.148.7 (talk) 12:58, 13 July 2015 (UTC)