Talk:Statistical proof

Attempt to fix and add references

I am not an expert statistician, but I and familiar with introductory statistics and a few advanced methods. My background is in biology (ecology, evolution, and genetics). When I arrived, most of the material was unsubstantiated and lacked WP:V standards and guidelines. Hence, I have laid down a path using the best of my knowledge - but it would be nice to get some expert assistance with this article. I've inserted a few references that I'm familiar with and have put this page on my watchlist. I'll try to enlist people I encounter who are more knowledgeable in this area to assist.Thompsma (talk) 06:17, 10 November 2011 (UTC)

I am a statistician and I find this article very odd. I'm tempted to say there is no such thing as "statistical proof". Statistics is not about proving things, as far as I understand it, but about attempting to identify and make valid inferences from data - but always couched in probabilistic terms, not in black and white. A statistical test is not a proof or a disproof of a null hypothesis (or its alternative hypothesis). I'm also watching this page and I'll be back. Tayste (edits) 01:04, 11 November 2011 (UTC)

PS: please resist adding too much to the article until other editors have had a chance to absorb what is there, assess the references and reach a consensus about this article. Tayste (edits) 01:33, 11 November 2011 (UTC)

I think there is sufficient literature WP:V to suggest that there is such a thing as statistical proof.[1] It is better to address the topic rationally rather than ignore it. The term statistical proof appears with enough regularity that it would be unusual not to offer an explanation. I may not be a statistician, but I am well versed in the philosophy of sciences and have come across statistical proof many times in my readings. My background is in genetics and employ statistical methods for analyzing mutation rates, phylogenies, and other biological applications.Thompsma (talk) 01:52, 11 November 2011 (UTC)

I came here because I was working on evolution as fact and theory and this got me thinking about proof. The old come back "Prove it!!" is a frustrating retort to deal with. My understanding of proof in the history of science and philosophy is that it is nothing more than the verifiable and rational demonstration of fact and theory that is used to convince others that the results are converging on truth. In this sense, there are two different kinds of proof. Mathematic proof operates using symbolic formula, which exists in statistics as well, and in the experimental sense of testing a theory. Statistical proof links the math to the experiment.Thompsma (talk) 01:59, 11 November 2011 (UTC)

It is also an important concept in legal settings (e.g., [2],[3]).Thompsma (talk) 02:10, 11 November 2011 (UTC)

Thee is no point in having another article that philosophises about aspects of statistics inference, since that is done elsewhere. If "law" said to be an important application, then there is already Evidence under Bayes theorem, and other entries in category:Forensic statistics. I think the recent changes have been appalling, in that they turned an article that at least pointed out there are many different fields of application that use the term in different ways, into a meandering essay from one persons point of view. I see that we have lost various citations to use of the term "statistical proof" from earlier versions, notably those formerly under the heading of "pure mathematics" and I guess they may be others. I think recent edits have concentrated on one version of what one group of people may mean by "statistical proof" in one context, rather than on showing how the term is used in different places. We certainly don't need another article covering the same ground as Philosophy of statistics, Philosophy of science, Scientific method, Inductive reasoning, Inference, Statistical inference, Bayesian inference, etc.. Melcombe (talk) 10:14, 11 November 2011 (UTC)

It is not one persons view - it has been cited using WP:V literature.

It certainly seems to one person's view that all other possible established meanings need not be mentioned.

The previous citations that have been lost were hardly up to WP:V standards - some had broken links and they were not cited, it was just notes with links that went all over the place. You are going to argue that when I first came here that this page[4] was better? It was filled with jumbled nonsense and original research (WP:NOR). Proof has historical significance in the history of science. Chamberlain's method of multiple hypotheses for example:

• "Each hypothesis suggests its own criteria, its own means of proof, its own method of developing the truth, and if a group of hypotheses encompass the subject on all sides, the total outcome of means and of methods is full and rich."[5]

Philosophers have been commenting on this for centuries and to claim that this has been revised to explain what "one group of people may mean by "statistical proof"" is absurd!! I have explored multiple sources from statistical literature, to legal examples, and through the philosophical literature. I am well versed in Karl Popper philosophy and in the history of science. Here is an apt quote that speaks to this matter:

• "Descartes' analytic method and Hume's experimental method are both "the scientific method," but as methods they have little in common. There are many philosophic methods, but there is no philosophy of method. Instead there are controversies about method which employ a semantics of method to discover that words like "analysis" and "synthesis," "discovery" and "proof," "induction" and "deduction" have many meanings, and which move from that discovery to prove either that no problem of method is involved because the meanings do not refer to the same situation or issue, or that, if there is an opposition, one of the opposed sets of meanings does not apply to discovery and proof in science or is intrinsically meaningless and absurd. The many meanings of these and like terms have been developed in complex discussions of the nature and interrelations of methods: they may be studied in a philosophy of methodological processes, or they may be set in oppositions and juxtapositions in a semantics of terms and their meanings."[6]

If we don't approach these issues what use is Wikipedia? This book [7] is titled "Statistical Proof". Chapter 5 of this book[8] is titled "Statistical Proof". Statistical proof exists in the literature and we need an objective way using WP:V to explain what is meant by this and this is what I have attempted to do.Thompsma (talk) 18:52, 11 November 2011 (UTC)

There is a historical component to proof and this is where things get difficult. As the statistics article states, some consider statistics to be a mathematical science, while others see it as a branch of mathematics. There is the discipline of mathematical statistics and there is mathematical proof. The question becomes, is there a difference between a mathematical and a statistical proof? Does proof in statistics just boil down to the math of it? In the history of proof in the philosophy of science, how does this branch network into the history of statistical versus mathematical proof? In as much as there is a distinction between math and statistics, there is a similar distinction between mathematical proof and statistical proof. We are entitled in Wikipedia to give an honest disclosure of this topic and there is sufficient reason to grant its place among the list of other articles on philosophical sciences. I have read that Popper felt that there was no proof in science: "Or, as the philosopher Karl Popper says today, there is no such thing as proof in science -- because some later alternative explanation may be as good or better -- so that science advances only by disproofs. There is no point in making hypotheses that are not falsifiable, because such hypotheses do not say anything..."[9] However, Popper talks about proof: "If you insist on strict proof (or strict disproof¹) in the empirical sciences, you will never benefit from experience, and never learn from it how wrong you are."[10] In the footnote to strict disproof Popper states: "no conclusive disproof of a theory can ever be produced". Popper of course regularly referred to mathematical proofs in his writing.[11] I'm very tempted to say to the challenge that there is no "statistical proof" - prove it!Thompsma (talk) 20:47, 11 November 2011 (UTC)

I'm trying to pick up the thread of discussion. The critical issue is first whether "statistical proof" is a "real" term, and secondarily whether a discussion of it belongs here or on auxiliary pages about philosophy of science and such, correct? Thompsma explains that chapter 5 of this book[12] is titled "Statistical Proof". I don't have the book at hand. This appears to be a book about expert testimony for the court. May I ask how it defines "statistical proof"? Friedman is a statistician of note who has worked on defensible statistical argument for the law so perhaps in that context, at least, it is meaningful. Tentatively, I'd suggest that the article should give a definition consistent with its use in this field and possibly other fields (philosophy?) and refer to other articles for further discussion of the finer aspects. As a trained statistician, the phrase "statistical proof" makes me nervous in a way that it would not if the phrase were "statistical disproof", but I am not an expert on the application of statistics in the law. I expect that what a lawyer calls proof is what I would call "rejection of all hypotheses consistent with innocence, at a legally acceptable significance level" or something similar with a Bayesian lilt. The current article, however, talks about the role of statistical proof in the sciences. This is not a common phraseology in my piece of the world. A quick check of PubMed shows that it does exist in some papers (57 at the moment). I suspect that they were using it as a shorthand for "find a statistically significant result that gives evidence for a conclusion" rather than proof in any grander sense. In particular I don't think they are using it as a technical term. Marc.coram (talk) 05:48, 12 November 2011 (UTC)

Thanks Marc.coram - I share your concerns. I hope you will take the time to read through the article and note that I address your concerns and carefully worded the sentences so that proof does not mean verification or certainty. I'm a trained scientist and the idea of proving something as opposed to the philosophy of falsification is a difficult area to tread into. However, proof does not necessarily mean that you are setting out to prove something or that researchers are using statistics as a means to obtain certainty. I fully understand where you are going with "statistical disproof". Proof in the history of science can also refer to a convincing demonstration to a public or peers, not an impermeable fact. In this sense, using statistics is a form of proof, not of the theory, but for the review process demonstrating that the right kind of methods (statistical and otherwise) were used to rigorously falsify the null-hypothesis. Some of the papers I have come across have not used it as a technical term - short hand as you have indicated. However, some paper I have come across are using proof in the sense of method demonstration not proof of theory. I don't have access to the linked book by Friedman either - I put in a request from our library to have it delivered.Thompsma (talk) 07:11, 12 November 2011 (UTC)

I refer you back to the earlier version of the article at [[13]] ... this had at least three uses of the term "statistical proof", backed up by citations, that are not included in the present version (some others may also be valid but are more doubtfull and have no citations). The fact that it may sometimes not be used as a technical term is largely irrelevant ... to ignore these usages is a mistake. Let's go back to the earlier version and expand it in a balanced way from there, leaving all the more technical discussion to other existing articles. (On the point of broken links in citations in the earlier version ... at least some of those are easiliy fixed, for example http://www.hi-mag.com/health-insurance/product-area/cash-plans-dental/article363934.ece .) Melcombe (talk) 12:21, 15 November 2011 (UTC)

That earlier version was a bunch of gobbly gook. If you fix the citations - then I will take a look at them. Actually, I did look at several of them and stick by the changes that were made. It is not that they were ignored, it was that the text in that earlier version was nonsensical - grammatically and scientifically. I may not be a professional statistician, but I am scientifically literate enough to know when I am reading garbage. The current version is cited with appropriate literature and the text supports and conveys the meaning of the information within the cited sources.Thompsma (talk) 19:36, 15 November 2011 (UTC)

Here is an example of the scientific nonsense in that earlier version:

• "In a typical statistical test, the ratio of probabilities of an event occurring under each of two hypotheses (the likelihood ratio) is evaluated, and if the likelihood ratio is "sufficiently large", it is asserted that there is “statistical” proof of one hypothesis."

First off - a likelihood ratio is not a "typical statistical test" (whatever that my be??). Second, no statistician would ever assert that there is statistical proof of one hypotheses in the manner given. That goes against the scientific principle of falsifiability. Whoever wrote that probably transcribed and translated the text from a book, but clearly did not understand the scientific philosophy behind it.Thompsma (talk) 08:23, 16 November 2011 (UTC)

I finished reading the article [14] that Melcombe directed me to above. Here is the only reference to "statistical proof" in the article:

• "Nevertheless, there is no need for those who feel that they have “heard it all before” to start tearing out the few shreds of hair that remain. This year is genuinely different. Optimism is no longer merely the product of gut feelings but is supported by statistical proof both from the providers themselves and, much more importantly, from independent authorities like industry analyst Laing & Buisson."

What can you say about the article other than some vague reference was made to statistical proof? I can give you 1000 papers of that sort - but it is meaningless. The statistical proof that is being referred to in that article, btw, is what this article explains. The linked article gives no guidance on the topic and statistical proof is far from its thesis topic. The reason why statistical proof has special meaning in a legal context (and given its own section herein) is because it intertwines with the definition of legal burden of proof.Thompsma (talk) 08:45, 16 November 2011 (UTC)

Oh dear, so Thompsma has decided that Wikipedia readers are not entitled to know that the term "statistical proof" is used in particular ways because they happen not to conicide with his view of what the term should mean. The earlier version of the article had sufficient citations to show that (at least some of) the different usages do appear in published sources: certainly enough to meet WP:V for what was in the article. So what if "statistical proof" is being used informally with no precise definition, by people who have no need of a precise definition? . Melcombe (talk) 14:10, 21 November 2011 (UTC)

Sorry Melcombe - your post makes little sense in light of the response I have given. I am not presenting my view - you are mistaken - please review the article as it has been modified because it is cited using papers that specifically discuss the philosophy of statistical proof. These are not my ideas, but ideas of statisticians and other scientists that have been cited using WP:V. Please re-read my concerns regarding the previous "citations" and how they only mention statistical proof in passing but offer no guidance on what they meant by the term. It is not about sufficient citations, it is about use of proper citations and expressing what those citations mean. I am well versed in Wikipedia guidelines and have been a solid contributor to science articles in here for a number of years. Please read through WP:V and you will see that it is not about "sufficient" or "enough" citations, but it is about the material in the citations themselves that matters.Thompsma (talk) 17:44, 21 November 2011 (UTC)