Talk:Peer review/Archive 2

Archive 1

Archive 2

Peer review as an encouragement to mediocrity

Obviously, since a far greater percentage of the human population, or, it can be argued, a non-biased subset of the human population are average, mediocre, or below average than are exceptional or above average, peer review will automatically stifle true genius and encourage mediocrity. Of course, some will argue that in a collection of scientists with Ph.D.'s, the average intellect is of much better quality than that of the average human. I don't know this to be the case, in fact I suspect there are as many average or below-average IQ's in such a group as there are in the general population, but I do know that, even in that group of supposedly better-than-average intellects, the true geniuses will be few and far between. How many Newtons were there? How many Einsteins? Therefore, peer review suffers from the same thing democracy so often does, as lamented by Thomas Jefferson, that the idiots, comparatively speaking, far outnumber the geniuses. Democracy has no place in a search for truth, as the truth will remain what it is no matter how many imbeciles disbelieve it. If an idea or study is to be rejected, it should be rejected individually by the members of the general population, and should be kept from none of them by this Nazi-istic, book-burning ideology of: "Don't let people decide for themselves. We don't agree with this idea and it is therefore too dangerous to let the average person be exposed to it." They forget that some people can think quite well without their help, sometimes better, and that, barring that, they always have the right to think and decide without their help, if they wish.Ronar 05:50, 13 April 2007 (UTC)

Peer review of non-printed journals

I observe that the article has a bent toward the assumption that peer-reviewed journals are printed; i.e., printed on paper, and distributed as a tangible physical object. This is obviously the way it has been done for several centuries since the advent of movable-type printing in circa 1550. For example, in the second paragraph of the article lead: "... that are accepted for publication to a publisher who will be responsible for organizing redactory services, printing and distribution of the publication."

The printed/tangible form is no longer, ipso facto, required. In the social sciences I have begun to see serious peer-reviewed journals that are published in electronic form only. Two examples from economics are Econ Journal Watch ([[1]]), which claims six Nobel laureates as members of the EJW Advisory Council (Buchanan, Coase, Mundell, North, Schelling, Smith), and EH.net ([[2]]), published by the well-known History of Economics Society [[3]]. Both are peer-reviewed by the usual cast of mainline professors from a variety of universisties.

I don't want to start a firestorm of debate, but I do think we should begin to evolve this article to a more agnostic position, one where peer review is independent of the method of publication, print or electronic. Others please weigh in. N2e 03:19, 2 May 2007 (UTC)

Of course and I thought everyone realized this. Every journal has to be taken individually; the quality of the peer review will depend on the journal and has no particular relation to print or electronic. As examples, the peer review for BMC Biology is very strong, and there is no printed version; similarly for PLOS Biology which for a year stopped producing print. Both of these examples happen to be open access, but that's just because most open access journals do not attempt to finance print, which does require subscription fees. As an extreme example, the American Physical Society announced some time ago its readiness to stop print on Physical Review if there was no further demand for it. DGG 06:26, 2 May 2007 (UTC)

peer review failures

I think the examples in this section are unsupported and do not belong. Some of them are errors, not failures; some are disputed cases. Classifying them this way is OR. What is necessary to support them is references to sources that explicitly discuss them as failures of peer review. This is a matter of opinion, not of 2+2=4, and we cannot use our own unsupported views on these. Additionally, these are unrepresentative examples--there are some more drastic ones not discussed. The solution as far as this article is concerned is to branch these to a separate article so we can at least keep this article free of detailed controversy. I intend to do that split, subject to discussion here. How to handle the new article? Again, by making it a summary--the major cases deserve articles of their own. Some probably have them, and perhaps we can look for them and use a category. But thats for discussion on the new talk p. OK? DGG 06:13, 19 June 2007 (UTC)

thank you. could we have a page of "alleged" peer review failures linked to this articles, or something like that? looking forward to your addition. Chrissy385 22:12, 19 July 2007 (UTC)

A difficult proposition, but I will copy the headings at least from here. I see the page as a summary page, with individual notable failures or alleged failures on individual pages--the notable ones have quite enough literature to support them. It would help to have some uniformity in handling it on the summary page. How do you see it.? DGG (talk) 04:40, 20 July 2007 (UTC)

I've shifted most of the stuff that was here over to Peer review failure. Only shifting the problem, I know, but at least it's more focussed that way. 150.203.35.113 (talk) 05:21, 21 October 2008 (UTC)

chuine + keenan

with respect to the last revert. There is a claim here that the publication of Chuine paper is a demonstration of a peer review failure on the say so of one frankly trivial counter point (Keenan) and the backing of a non peer reviewed hatchet job (Carter et al) on the stern report (see here for more rebuttals: http://www.world-economics-journal.com/ ). If every paper that was ever challenged was a peer review failure, then half the literature would be here. This claim appears to have done the rounds of a various segment of the blogosphere and yet has never been picked up outside of those echo chambers. Failures of peer review would seem to me to require widespread recognition that such a failure had indeed occurred. As a counter example, I would offer Soon and Baliunas (2003) which lead to the resignation of six editors at the journal concerned (Climate research) due to the poor peer review and unjustified statements therein. Curiously enough the editor responsible for that is an author on the Carter et al paper (De Frietas). Chuine et al is neither an important paper, nor an important enough 'failure' (even assuming the Keenan analysis is justified, which is not at all clear) for it to be mentioned here. As it stands it is verging on libelous. Thus this section should be removed as i) inappropriate - the 'failure' not having been demonstrated beyond a single (and trivial) critique, ii) irrelevance, this was not an important paper (a brief communication of some anecdotal interest but of limited implication) and iii) partiality, much worse (and clearly documented) failures are not listed (I mention one above, but there are many more, Friis-Christensen and Lassen (1991, Science), or Douglass et al, 2004 GRL for instance). Leaving this in is a travesty of good scholarship.

You are quick to accuse the text as being libelous, when the text gives no indication of signaling out Chuine for criticism. The argument you give justifies it being replaced rather than removed. You are free to replace it with more important failures, but simply removing it is unproductive to creating complete discussion of the topic.--Jorfer 21:49, 20 June 2007 (UTC)

accusations that a paper is a clear example of 'peer review failure' is something that is seen within the academic community as extremely serious, and could do serious harm to an authors reputation. Thus for a paper to be listed there, there must be good cause and serious backup. Neither applies in this case. I offered three examples of what I thought were more grevious cases - especially the Soon and Baliunas paper, and I perfectly happy to have that replace the current example. My experience is with the climate related literature and so I do not want to comment on the other examples. I note that the S+B paper is already described as a peer review failure on Baliunas's page and so it makes sense to cross link it here: [4]

Since there was no apparent objection to this suggestion, I have gone ahead and replaced the disputed reference, to one that I think everyone can agree on. If there are objections, or better referencing required , please let me know.

This is not the place to discuss the global warming controversy. Any examples based on it are--very obviously--divisive, and will be removed to maintain the integrity & NPOV of this article. (regardless of which side they are on). There are enough articles already to discuss this topic, or start a new one: Allegations of publication fraud in the global warming controversy, or some such thing. and put a link to it in this one. DGG 05:49, 26 June 2007 (UTC)

practical thing is to split

as discussed earlier, and since there has been no objection over the last 3 weeks, I have split the peer review failures and peer review fraud sections outt into a new article Peer review failure, I have also copied over the relevant part of the talk page discussions.

—Preceding unsigned comment added by DGG (talk • contribs) 06:40, 12 July 2007 (UTC)

Is Peer Review an Effective Tool for Quality Control?

SNIPPET:

http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371/journal.pmed.0040286

"Peer review has not been proven to be an effective tool for quality control, so we cannot rely on journals' peer review systems to guard against biases created by ghost managing [47–49]. Indeed, MECCs [medical education and communication companys] are effective at creating publishable articles and getting them published in peer-reviewed journals."

references:

47. Campanario JM (1998) Peer review for journals as it stands today — Part I. Sci Commun 19: 181–211.

48. Campanario JM (1998) Peer review for journals as it stands today — Part II. Sci Commun 19: 277–306.

49. Jefferson T, Rudin M, Brodney Folse S, Davidoff F (2006) Editorial peer review for improving the quality of reports of biomedical studies Cochrane Database Syst Rev 2. MR000016.

--- Alan2012 14:19, 30 September 2007 (UTC)

Unclear language

"...the United States Office of Research Integrity has sanctioned reviewers..." - I don't think "sanctioned" is a good word to use here, as it could be read as meaning that the USORI had *approved* reviewers. I'm not actually sure whether American English uses "sanction" in this positive context, but British English certainly does. For example, see this report from the "Independent" newspaper, which includes "when the Glazers have sanctioned further spending at Manchester United" - meaning *approved* it, not penalised it. 86.136.251.18 02:22, 10 October 2007 (UTC)

conferences

I have removed some unsourced speculation. if there is documentation, perhaps it might g in a separate section. DGG (talk) 01:57, 22 November 2007 (UTC)

post publication peer review

The following link was removed from the article on Feb 5th. www.JournalReview.org JournalReview.org is a non-profit, free, academic, educational resource which I - as well as about 7000 other physicians and health care professionals internationally participate in. The site allows participants to discuss published medical literature with colleagues, offering real-time post publication peer review. Journalreview.org allows discussion of all literature indexed by the national library of medicine. Anyone is able to read the data and discussion posted on JouranlReview.org. Participants can annotate citations by adding metadata beyond what the abstract offers. This allows authors to suplement their work, respond to questions, and alows researchers to gain ideas on how future efforts may be directed to help expand a field. The site charges no fee to use, and shows no advertisements. The resources has been discussed in several peer reviewed publications. Lets use this space to discuss the merits of including a direct link to the resource in this wiki article. It has been an external link offered in this article since 2005! EBMdoc (talk) 15:35, 6 February 2008 (UTC)

Sounds like a sales pitch. Your contributions to wikipedia under EBMdoc and Special:Contributions/Researcherbasic- -Special:Contributions/Basicsci99- -Special:Contributions/Labrat2- -Special:Contributions/Resdir- -Special:Contributions/24.185.216.101- -Special:Contributions/129.59.114.13- -Special:Contributions/68.236.181.27- -Special:Contributions/70.136.116.10- -Special:Contributions/69.117.14.111- -Special:Contributions/Iat- -Special:Contributions/Awakennow- -Special:Contributions/Neuro18- -Special:Contributions/Eyewell- -Special:Contributions/Pedsdir- -Special:Contributions/Uroly- -Special:Contributions/138.5.243.14- -Special:Contributions/PMR8- -Special:Contributions/71.125.237.249- -Special:Contributions/MovingScience- -Special:Contributions/209.75.253.2, consist entirely of adding external links to journalreview.org and is considered WP:Spam. Looking through your contributions as a whole, the all seem to be journalreview.org related only. Please do not continue adding links to your own websites to Wikipedia. It has become apparent that your account and IP's are only being used for spamming inappropriate external links and for self-promotion. Wikipedia is NOT a "repository of links" or a "vehicle for advertising" and persistent spammers will have their websites blacklisted. Please see the welcome page and Wikipedia:Civility. Avoid breaching relevant policies and guidelines. You're here to improve Wikipedia -- not just to promote journalreview.org right? --Hu12 (talk) 13:44, 7 February 2008 (UTC)

in my opinion that link should not have been here, not being closely enough related to the topic of the article, and I apologize that it has been kept so long. Nor are some at least of the above links appropriate. It may be a useful site, and might even warrant an article, if you have really good references to support its notability, but that link was not appropriate here. Try writing an article on it on your talk page, and let me know & I'll take a look. DGG (talk) 16:20, 7 February 2008 (UTC)

Aduit my flow chart please

I have created a simple flow chart for the peer review process, and would like to receive feedback on it before I upload it to Wikipedia. Please view and comment here User:Msanford/Image_audit, thank you! --Msanford (talk) 20:37, 4 March 2008 (UTC)

Split proposal: Open peer review

I believe that this article is maturing enough to split out sections that are broad enough to merit their own article (namely, open peer review, among a few others). I definiely think that Open peer review deserves its own article, with the usual intro text transcluded here. If approved by the community, I'll naturally happily perform the split. Msanford (talk) 17:37, 10 March 2008 (UTC)

I agree, open peer review merits its own article, but there should be at least a paragraph about it here. Michael Courtney (talk) 12:22, 1 April 2008 (UTC)

Of course we'd leave a trace here. Thanks for commenting! Msanford  T  21:18, 1 April 2008 (UTC)

Seeing the limited activity here in the recent months, I've decided to be bold and forked the new article myself. --Lysy^talk 13:14, 3 August 2008 (UTC)

Journal "fishing"

DGG just removed, quite rightly, a section on "journal fishing" that was supported only by a blog post. I have seen this behaviour described in the literature as "journal shopping", and have just looked around for some possible sources, in case someone is interested in rewriting based upon reliable sources. Here are some possibilities:

• Vardeman & Morris. Statistics and Ethics: Some Advice for Young Statisticians. The American Statistician, Vol. 57, 2003.
• O'Shea. Confidentiality. Optical Engineering, Vol. 45, 2006.
• Miller et al. On exemplary scientific conduct regarding submission of manuscripts to biomedical informatics journals. Computer Methods and Programs in Biomedicine, Vol. 81, 2006.

Jakew (talk) 17:00, 2 May 2008 (UTC)

Laissez-faire?

Physicists, for example, tend to think that decisions about the worthiness of an article are best left to the marketplace.

What is that supposed to mean? If it means something like "physicists tend to think worthiness is best judged by what physicists, on the whole, find worthy," then why doesn't it say that? If it means something else, then what on Earth? 61.69.2.133 (talk) 14:36, 3 June 2008 (UTC)

Audit

Most journals (and grant agencies like NSF) have a policy that authors must archive their data and methods in the event another researcher wishes to replicate or audit the research after publication.

Er, really? In all my years of publishing in international physical science journals, I have never once seen any such statement of policy (and I do read journals' publication policies). 61.69.2.133 (talk) 14:40, 3 June 2008 (UTC)

photo

The photo at the top of the article is obviously not a reviewer, but a CSR employee checking a huge load of grants, probably for administrative flaws. No reviewer would ever be burdened with (or accept such) a huge load of applications.... --Crusio (talk) 11:35, 10 August 2008 (UTC)

referee vs reviewer

I find the term "reviewer" preferable over "referee". The latter implies some decision-making power. In reality, it's the editor that makes a decision, the reviewer provides a recommendation. (It is true that some editor's make theirs an easy job by just taking a straw poll of the reviewers' "votes", but that doesnt change the principle). The same goes for grant review. Reviewers come with recommendations,but then it is a different body (at NIH this would be the coucil of one of the NIH Institutes) that makes the final decision. They can (and sometimes will) reverse review rankings based on other considerations (such as research priorities). In addition, "reviewer" is more in line with the article's title: Peer review. --Crusio (talk) 11:39, 10 August 2008 (UTC)

Criticism of peer review mostly bogus

The scientific and academic communities at-large overwhelmingly support peer review, often simply because there is no better system than to have expert work in a given field checked by other experts whose published work also has to go through the same process. The bulk of the criticism for peer review stems from cabals like the Discovery Institute, who cannot get their rhetoric published in peer-reviewed journals owing to its complete lack of scientific credibility, or from pseudo-academic right-wing think tanks such as the Heritage Foundation or the Cato Institute, who engage in nothing resembling actual research subject to scholarly peer review, and whose genesis derives from massive funding by conservative individuals and conservative corporate sponsors attempting to undermine legitimate academic conclusions such groups deem inconvenient to their unfettered free market and religious agendas. For a position paper illustrating the conservative mind-set opposed to peer review and in support of crusading ultra-right think tanks, see the Powell Memo.

The peer review system is certainly only as good as the knowledge and integrity of the scholars engaged in such review, but as a system it is not broken, nor in need of overhaul. It is the right-wing think tanks, publications such as the National Review, and those who agree with their views who cannot abide peer review, because subject to academic and scientific scrutiny, the substance of their arguments quickly evaporate.PJtP (talk) 14:37, 10 August 2008 (UTC)

I agree that peer review is the best system we actually have. However, it is not only the right-wing and intelligent design people that criticize the system. Many scientists do, too, although I have to admit that this generally (but not always) are the people producing mediocre stuff that gets rejected all the time... --Crusio (talk) 15:14, 10 August 2008 (UTC)

the standard peer review is the best system we have only in the sense that no other system has been widely adopted, not that no other system has been devised, and not that there is any actual evidence showing the effectiveness of the present system over others. has never been actually demonstrated. Of course, the same lack of firm evidence is true of all other aspects of the educational and research system, and for that matter of all complex human social arrangements. No scientist would be satisfied in their own scientific work with the sort of evidence for effectiveness they accept for the system under which their work is carried out and published.It sounds plausible, but beyond that almost everything in the way of evidence is either inconclusive or anecdotal. Let's not those who understand it claim too much for it. ( this is of course no more than personal opinion--just like almost everything else said on way or another).DGG (talk) 08:21, 11 August 2008 (UTC)

Did JACS invent modern peer review?

An editorial published this year by the Journal of the American Chemical Society says:

"In its 129 years of publication, the Journal of the American Chemical Society has been at the forefront of many innovations in publishing, dating back to Editor Arthur Lamb introducing the formal peer review process to the journal and the publishing community in 1918. Lamb chose scientists other than members of the editorial board to review manuscripts and to do so anonymously. That innovation served as the benchmark for responsible peer review and to this day remains the lynchpin of scholarly publishing."

Has anyone else heard of that? That bit of information could be useful for the article (especially if it can be verified with another source.) --Itub (talk) 18:16, 12 November 2008 (UTC)

It's an interesting aspect. According to Burnham, J. Am. Med. Assoc. 263 (1990) 1323 [5]

The practice of editorial peer reviewing did not become general until sometime after World War II

yet Benos et al., Advan. Physiol. Edu. 31 (2007) 145 [6] says

The British Medical Journal, however, sent every noneditorial submission to a recognized expert by at least 1893

citing the Burnham paper (which I only have access to the abstract).

Does this clear it up? I don't think so! 150.203.35.113 (talk) 03:30, 19 November 2008 (UTC)

It would be interesting to find out if the BMJ reviewers were also anonymous. --Itub (talk) 05:57, 19 November 2008 (UTC)

{mergeX} 'Open peer review' and 'Anonymous peer review'

There are two articles in wikispace that ought to be merged with this one: Open peer review and Anonymous peer review. These are two faces of the same coin, and not independent topics that can be discussed independently of one another. Moreover, neither can be discussed without also discussing the criticism of the peer review process, which this article has a section on, and which rightly belongs here. That's not to say that the 'pedia can't have a Criticism of the peer-review process article, but its roots need to be here in this article. -- Fullstop (talk) 17:47, 11 December 2008 (UTC)

Sure. I don't want 3 articles if 2 will do - or 2 if 1 will do. I just don't want to conceal from the public that "peer review" is often carried out anonymously. I had to read 3 or 4 paragraphs of this article before I found that out (I kind of knew it, but it would be better for all readers if the anonymity aspect were brought out earlier on.) By the way, what percentage of scientific papers are reviewed anonymously? And how independent are the reviewers? Does it vary with the topic of the article or the sub-community of scientists who deal with that topic?

And how does peer review relate to reproducible results? Is running someone else's experiment part of the review process, or what? --Uncle Ed (talk) 18:09, 11 December 2008 (UTC)

Simply put the fact that peer-review is usually anonymous in the lede. Anonymity is the norm.

To answer your questions...

a) Most publications have anonymous peer-review processes in that authors of disqualified papers are not told who reviewed the work. From the point of view of the reviewee, the rejection (and/or recommendations for improvement) comes from the editorial board, i.e. from those who are ultimately responsible for the integrity of the publication.

b) What do you mean by "independent"? Editorial boards send the paper to whoever they think is in the best position to make a "print" or "no print" decision. If that person is unable (or unwilling) to make that decision, it goes to someone else.

c) Referees are invariably experts in the field that the paper is on.

d) Peer review is not related to reproducibility. The experiment won't be reproduced until the community knows that someone produced it to begin with.

-- Fullstop (talk) 01:11, 13 December 2008 (UTC)

Thank you for answering my questions. I will check back soon to see how much of this information is in the article. Your first sentence at "a)" should probably be in the intro. I'd also like to see a section on the anonymity aspect (if it's short) or a summary section on anonymity linked to anonymous peer review using the {{main}} tag (if it's long).

As for independence, I don't mean anything - because I don't know anything! I am inquiring as to whether there are any standards about independence, and why review is anonymous. Often I approach an article more as a reader than a writer, so I have nothing to say here.

Your sentence at "d)" should be in the article. Are scientific journals the primary mode (or method) by which scientists communicate the results of their experiments, analysis, and ideas? Has anonymous peer review been regarded as helping to facilitate this process? (for example, by weeding out obvious junk which other scientists would not want to waste their time double-checking)

I'm also wondering how much checking other scientists do before assuming that a peer-reviewed paper is reliable. Is there a gold standard in science? Is "publishing a peer-reviewed paper" the gold standard? If not, what is the custom or procedure for detecting and communicating [irreproducible results]]? When I was taking courses at Harvard, I flipped over the Wormrunner's Digest and found a humour magazine called "Journal of Irreproducible Results". If peer review and reproducibility are unrelated, than how do scientists check up on each other? --Uncle Ed (talk) 12:35, 13 December 2008 (UTC)

Irrespective of how high the standards of a journal are, the basic purpose of peer-review (i.e. something all peer-review processes implement) is to prevent the integrity of the journal from being compromised, and to prevent others from wasting their time reading gunk.

A Wikipedian familiar with WP:Notability might see it this way: a paper that proposes something absurd will not be acknowledged (even if just to debunk it) from the readership. On the one hand, absurdity (to be distinguished from error) can't be countered with logic, and on the other hand, acknowledging the paper gives it notice (notability). But a theory that is not contested stands, and that can't be allowed to happen either, and that's what peer-review prevents. In effect, peer-review can be considered to be the means to inhibit cranks from misusing a journal to gain notability.

Beyond this basic function (that a paper must be printworthy; it must have at least some merit) that is common to all, the level of acceptance varies from journal to journal. While some journals have such high standards that virtually only landmark papers get published in it, peer review is not generally that stringent. In general, peer-review merely determines if a paper is worthy of publication (see also peer-review failure), that it adheres to scientific method, and that it follows (in-house) standards for publication. In general, no more (or not much more) is done. In-depth evaluation (to include tests for reproducibility) occur after the paper has been published. Only then can the community check if the experiments described in it can be reproduced. These evaluations will then also be published (typically in the same journal), and are not anonymous.

Yes, scientific journals are the primary method by which scientists communicate the results of their experiments, analysis, and ideas. Books -- to include PhD theses and conference proceedings -- are no longer acceptable as a channel through which novel theories are communicated. Books are still subject to book review though, which of course happens in the journals. A publication (book, paper, whatever) that has not elicited notice from its peers, is -- by definition -- unnoted, and hence fringe.

I still don't understand what you mean by "independence". Independence from whom/what? Do you mean from different institutions? (Answer: not necessarily) Or in different fields? (No) Or not having a conflict of interest? (ideally they shouldn't; honest reviewers with COI will notify the editors of this fact). When I asked what you meant by "independent", I meant to ask in what sense you meant "independent."

I suppose peer-review could be considered to be a bit like new-page patrol, with the difference that (barring complete rejection, aka speedies) tagging articles with {{issues}} doesn't imply that the article will also be sent back to userspace. That comparison is lacking in many ways, but the basic process is the same. -- Fullstop (talk) 15:24, 13 December 2008 (UTC)

The basic difference from Wikipedia process is that the peer reviewers are experts in the specific subject of the paper, selected by a more senior general expert in the field (the editor in chief of the journal), and their reviews are subject to his further review. They are thus intellectual and academic peers of the authors of the paper, and are assumed to know as much as they about the specific subject, and thus be able to spot suspicious material and violations of standards. This is radically different from WP, where the material is reviewed by whoever in the community thinks himself to be qualified to give n opinion, and, if challenged, are reviewed by equal non-experts. Thus, academic peer review requires a recognized structure of authority. The academic quality of the editors in chief in turn, is maintained by two mechanisms: for journals published by scientific societies, the society in some manner appoints the editor in chief, and the reputation of t he journal is a function of the reputation of the society. Thus , the journals published by the American Chemical Society have the authority of the society as a whole, representative of professional chemists. For journals published by commercial publishers, and as a second check on societies, the quality of the editors in chief determine the submission of papers--people of a high academic level will publish only in journals of a similar level, and a journal that cannot attract good papers will not attract knowledgeable readers and will fail. True, in a sense, the academic peer review system is internally self-validating--but that is the case generally: PhDs are awarded to people who have the certification of such merit by other PhDs, and such has been the case since the 12th century. There is a certain sense in which Wikipedia does acknowledge expertise, however gained, in the sense that comments of editors here on articles will be accepted if considered to be useful , logical , and well-supported, while editors who appear ignorant or biased will be disregarded. DGG (talk) 00:56, 14 December 2008 (UTC)

Wow, thank you both - full stop and DGG. I need to dig into the articles on peer review, peer review failure, open peer review and so on. Either you have very helpfully digested the most interesting points of these articles for me here, or much of what you said needs to be added to them! :-)

As for independence, if it means having no conflict of interest then the aspect that concerns me is how fresh new ideas get into scientific currency. There's also the imputed occurrence of situations where some kind of pressure (economic or political) is brought to bear on reviewers or editors. This could theoretically happen if a dispute arose, and a large enough majority of scientists in a particular field were somehow beholden to a particular side in a scientific controversy - or just happened to believe strongly in the same viewpoint. I don't know much current science, but I have studied a number of historical cases (all before 1940) in which pioneers faced outright opposition amounting to persecution. I think Semmelweis got run out of town or something. But maybe the publishing and communication system we have now is different, and this sort of thing doesn't happen any more. --Uncle Ed (talk) 23:51, 14 December 2008 (UTC)

Ideally a conflict of interest doesn't occur, because a reviewer who (when reading the paper) determines that there is a COI is (morally) obliged to return the paper to the editor, who then would pass it on to another reviewer. But that's just the ideal situation, and reviewers are human too, and so just as subject to vanities and prejudices as anyone else.

The mere fact that an idea is novel is not a reason to reject a paper. I guess one could say that the benchmark is the question "would anyone respond to this"? There are a number of factors beyond mere content that go into that equation, e.g. whether its a hot topic, whether the author is a known figure, and so on. Insofar as all papers can provoke opposition (or support), pioneer papers are no different from any other paper. And both opposition and support occur by way of response, i.e. after publication.

I suppose that scientists could still be professionally marginalized, but I doubt that they would be committed to mental institutions. ;) -- Fullstop (talk) 11:43, 16 December 2008 (UTC)

Uncle Ed:

As for independence, if it means having no conflict of interest then the aspect that concerns me is how fresh new ideas get into scientific currency.

It is a common misconception that scientists are regularly beholden to particular points of view and don't want to see their sandcastles knocked down. It's not true (and is essentially the difference between the way science is conducted and religions are observed). Most scientists love new ideas and the overturning of the old. It's exciting. It's why many of them became scientists in the first place. Science is all about advancing human knowledge in new and interesting ways. 150.203.35.113 (talk) 03:00, 9 January 2009 (UTC)

Wiki layouts

Can we have the etymology of the phrase, and then a history section regarding the concept of peer review and then go into the subject? Faro0485 (talk) 00:11, 27 April 2009 (UTC)

History of peer review.

[7] refers to peer review as a method earlier than the one mentioned in the article. Faro0485 (talk) 00:15, 27 April 2009 (UTC)

that's Medical peer review, which is related but not the same thing. (This is peer review of publications, not of medical results.) Please be sure to mention it there. Very glad to have seen this, though--it's new to me and very interesting. DGG (talk) 03:02, 28 April 2009 (UTC)

Knowing and Knowledge

Just wanted to re address the subject of how editors determine who are the "others who are experts in the same field"

This begs the question, how do we define a field of knowledge ?

This article has a number of wobbly terms: "academic quality", "field" (meaning field of knowledge ?)

If one's basis for knowledge is knowing who has knowledge, that seems to only displace the problem of knowing, rather than solve it. --InnocentsAbroad2 (talk) 23:12, 17 May 2009 (UTC)

This begs the question, how do we define a field of knowledge ?

• You're being far too literal. The role of the editor is to assign a manuscript to someone who is likely to be able to assess the work with some degree of confidence. No-one cares if a chemistry paper gets sent to an appropriate physicist.
• It's not begging the question.

If one's basis for knowledge is knowing who has knowledge, that seems to only displace the problem of knowing, rather than solve it.

• Yeah, and? So what? As long as the knowledge can be found, why would you care? That's the point of an editor sending manuscripts to expert referees, rather than assessing it alone.

210.9.137.73 (talk) 05:29, 21 June 2009 (UTC)

Well, yeah. An editor can judge what a topic is about and who knows about that area without themselves being expert in the field. How that expertise is defined and assessed is a good point though - subjectively is the real answer. Fences&Windows 00:37, 22 June 2009 (UTC)

Multiple " Peers"

Recently I have heard several Climatologists defend their position by claiming non-Climatologists are not "Peers", and should be ignored. That seems wrong to me in that several sciences, such as Physics, Chemistry, etc., are involved in any full discussion of Climate issues. So, I think we should have a section on the definition of "Peers" as related to multiple discipline involvement in the issue being put forth for review. Anyone want to tackle this?Bogie2 (talk) —Preceding undated comment added 18:07, 6 December 2009 (UTC).

"Peers" is usually understood to mean "someone working in the same field". I am not a climatologist, but a non-climatologist would not be suitable to review work in that field. At most, a physicist could be asked to review some physical aspects of their work, but nothing else. --Crusio (talk) 19:43, 6 December 2009 (UTC)

Wikipedia

Wouldn't it be great if Wikipedia was partly peer-reviewed? I mean, groups of expert could evaluate some of the main articles and confirm that all of there content can be validate and is neutral. Therefore, an user could go to the last peer-reviewed version of an article, when necessary. That would be the best of both world! 24.203.4.38 (talk) 22:43, 26 February 2010 (UTC)

Weasel words

This article has a lot of unsourced weasel words at present; "peer review has been criticized (by who?) as ineffective, slow, and misunderstood", "One of the most common complaints (who is making these complaints?) about the peer review process is that it is slow," and so on. Some of these have already been pointed out in the article; "Others[who?] have pointed out that there is a very large number of scientific journals in which one can publish", but there still appears to be something of a concerted effort to push a viewpoint on the reader. —Preceding unsigned comment added by 118.208.192.144 (talk) 03:46, 28 March 2010 (UTC)

Lack of references

Hi, this page no longer meets all of the featured article criteria. Specifically it has no references, or at least any that are properly cited. Please help add some references, use them to improve the article and cite them. Thanks - Taxman 01:19, Nov 5, 2004 (UTC)

I'm with Taxman. There are only 7 references in the first five sections, yet somebody went through the "criticisms" section with a fine-tooth comb. Fair's fair. Katharine908 (talk) 16:43, 6 April 2010 (UTC)

Famous papers which were not peer-reviewed

From the request for references, added Galileo's Two New Sciences as an example of major articles published before peer review. This is the case from my understanding of reading about Galileo. Is one of his other papers more important? Is there evidence that this was actually peer reviewed? DLH 23:44, 6 June 2006 (UTC)

Peer review is "censorship"?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Archiving this discussion as closed, with the consensus result: "peer review should not be included as a form of censorship" --Ludwigs2 19:51, 28 January 2010 (UTC)

{{rfctag}} User:Pottsf has added the description of Peer review as "censorship" to the article, added Censorship as a "See also", and added the article to Category:Censorship.[8] I've removed the edit since a) it doesn't appear obvious to me that the source used states Peer review is censorship, and b) Peer review is not considered "censorship" in any normal understanding of the word. Comments? Jayjg ^(talk) 02:20, 3 January 2010 (UTC)

• The article already uses the word 'suppression' which conveys the meaning without too many negative connotations. The word 'censorship' implies (in this context) a small group of individuals are consciously controlling access to information to promote a specific viewpoint, which isn't the claim being made. On the contrary, the following paragraph which gives counter arguments to the claim should be researched to find missing references so that the article is balanced.--RDBury (talk) 17:43, 3 January 2010 (UTC)
• No, peer review is not censorship. "Proponents of HIV Denial or Intelligent Design like to compare scientific peer review to censorship. But the truth is that the scientific community has provided ample opportunity for these ideas to be publicly aired, arguably more than they deserve, and ultimately rejected. That is not censorship. Misrepresenting these discredited ideas as victims of censorship risks minimizing the true threats of scientific censorship, as when a government deletes politically sensitive remarks by scientific agency heads and surgeon generals, alters reports by government scientists, or prohibits the publication of sensitive data."(Casadevall, Arturo (2009). "Is Peer Review Censorship?". Infection & Immunity. 77 (4). American Society of Microbiology: 1273–4. {{cite journal}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)). Fences&Windows 00:22, 4 January 2010 (UTC)
• Agree that peer review and censorship are not the same thing. Claiming the latter will only confuse the readers of the article. Labongo (talk) 07:24, 4 January 2010 (UTC)
• not censorship: peer review is actually the opposite, technically speaking - reviewers are obligated to read the material and accept any that meets disciplinary standards for scholarly work, regardless of whether they agree with the results achieved. it only looks like censorship because material which fails to achieve basic scholarly standards is often rejected out-of-hand. --Ludwigs2 00:10, 5 January 2010 (UTC)
• I think all the above arguments are persuasive - we should not add censorship to the see also section of this article. - Tbsdy lives (formerly Ta bu shi da yu) ^talk 00:08, 7 January 2010 (UTC)
• Not censorship. I'm very surprised this needed an RfC. --FormerIP (talk) 01:27, 7 January 2010 (UTC)
• Of course not censorship. Censorship is about suppressing true information, peer review is the opposite. --Crusio (talk) 09:49, 7 January 2010 (UTC)
• I concur. Peer review is a process of verification. If you expand the definition of censorship to include peer review, you might as well go all the way and include proofreading. Novangelis (talk) 14:32, 7 January 2010 (UTC)
• The issue is scientific standards. When referees fall within the description given by Horton, and when reviews are are so badly recorded that they are synonymous with animal grunts, they become censorship. Incidentally, the book on quasi drivers is about censorship and should not be censored.Bourdillona (talk) 17:48, 9 January 2010 (UTC)

• Sorry, but your self-published site is not a reliable source. It is about a technical subject with at the end an angry email from the author protesting the rejection of one of his articles. Such rejections, rightly or wrongly (and there is no way for the general reader to establish who is right), are nothing special and this is not the place to vent your frustration with that particular journal. And anyway, there are thousands of journals. So what they don't want to publish your manuscript? Improve it and send it somewhere else like everybody else does. --Crusio (talk) 18:20, 9 January 2010 (UTC)

• this is a misrepresentation. Further developed papers have been published in journals and invited at conference and in books. The cited book is about a catalogue of unscientific practice of types described by Horton. The book is catalogued accordingly in university libraries. The emails are not angry but rational, and the general reader can easily see that they are correct in relevant parts, as explained. The fact that the book is self published is precisely the point. No frustration is vented, the shortcomings are simply described. Why should a rational scientist send his paper somewhere else as you do? The delay is unnecessary and counterproductive. You have not addressed the issue of scientific standards in reviewing. Wikipedia fails if it allows censorhip. Journals are not necessary and peer review operates now in more sophisticated ways than formerly.
• No, not in the normal usage of the word censorship. Some contributors or sources may be confusing censorship in the sense of "supervision" (normally over morals and conduct) with "review".—Ash (talk) 10:30, 10 January 2010 (UTC)
• in normal usage Jimmy Wales has the stated aim of 'objectivism'. Supervision by capricious sleight of hand would be censorship. After agreement by open debate, supervision is scientific. Reviews that are not up to standards of science (all remarks justified whether by fact, argument or citation) are not peer reviews. defn: peer: "one who takes equal rank with another in point of natural gifts or other qualifications..." (Oxford Engl. Dict.) I had not intended to respond again, but a new issue of definition has arisen. —Preceding unsigned comment added by Bourdillona (talk • contribs) 02:42, 11 January 2010 (UTC)
• This is not the ordinary meaning of the word. DGG ( talk ) 00:38, 13 January 2010 (UTC)
• Peer review is nothing like censorship. Normal scientific process is that scientists voluntarily submit their publications for review to get criticism and validation of their work from their peers. It would only be people that do work that is not recognized by the scientific community that would call this censorship, since their work would not get the validation they themselves think they should get. Censorship does not belong in this article. Valar (talk) 11:18, 14 January 2010 (UTC)
• The edit in question attributes a claim that peer review is "a form of censorship" to this source, presumably this chapter. In the chapter on peer review, "censor" & "censorship" do not appear, though "suppress[ion of] dissent" certainly does. One may argue that censorship is a special case of the suppression of dissent, but the case is not made in this cited source that peer review falls into the smaller category of censorship specifically. However, there may be other sources from which to draw specific claims of "censorship" via peer review (this journal editorial cites two papers that "compared peer review to censorship", for example)...If a properly-sourced description of censorship charges can be produced, censorship can be linked in the text, making a see also link unnecessary; if not, such a see also link would confusing and should be avoided anyway. Categorization in censorship does not appear to be currently appropriate, as it has not been established that peer review is considered censorship in anything beyond a few individual opinions nor that the linkage is particularly relevant. — Scientizzle 18:29, 28 January 2010 (UTC)

The discussion above is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

There is more divergence than consensus and the wickipage should reflect it.

Should it be surprising? Some people find credible a system in which anonymous referees examine the papers of their competitors without any need for reason or requirement to respond to rebuttals.

In particular and semantically, a review which falls below the standards of the paper written is not “peer reviewed”. I was taught that a referee should objectively describe the strengths and weaknesses of a paper and let an editor decide if he chooses to publish. The reviewer should subsequently be prepared to defend or amend his view, but if it is objective he won’t have to.

It doesn’t matter to the serious minded, but it does to wikipedia whose system of subjective, uncredentialled, and more or less anonymous editorship regards journals as authoritative, though not their authors. WP used to be more objectivist.

These observations are made by an author of 200 papers published before first rejection, after he had refereed almost as many There turns out to be an inverse relationship between quality and acceptance.

—Preceding unsigned comment added by Bourdillona (talk • contribs) 11:43, 24 April 2010 (UTC)

Interesting comments on peer review

Should probably by incorporated into the article. From recent CFP - Symposium on Peer Reviewing:

"Only 8% members of the Scientific Research Society agreed that 'peer review works well as it is.' (Chubin and Hackett, 1990; p.192)

"A recent U.S. Supreme Court decision and an analysis of the peer review system substantiate complaints about this fundamental aspect of scientific research." (Horrobin, 2001)

Horrobin concludes that peer review "is a non-validated charade whose processes generate results little better than does chance." (Horrobin, 2001) This has been statistically proven and reported by an increasing number of journal editors.

But, "Peer Review is one of the sacred pillars of the scientific edifice" (Goodstein, 2000), it is a necessary condition in quality assurance for Scientific/Engineering publications, and "Peer Review is central to the organization of modern science…why not apply scientific [and engineering] methods to the peer review process" (Horrobin, 2001)."

--_{Piotr Konieczny aka Prokonsul Piotrus| talk} 20:59, 25 April 2010 (UTC)

Wiki is victime to its own censorship. The last observation in the archived material makes a presumption on the wrong citation. The correct citation has been censored. When it is restored, progress can be made. —Preceding unsigned comment added by Bourdillona (talk • contribs) 11:55, 29 April 2010 (UTC)

Recent news to consider

There has been a big fuss about the recent text The Spirit Level recently, especially as regards the authors' claims about peer review (they said they would only respond to peer reviewed criticisms). Has anyone considered this for inclusion? I can look into this later myself. Links, for your information: [9], [10]. The Sound and the Fury (talk) 06:43, 17 August 2010 (UTC)

• Thanks for those links, interesting reading :-) However, this is just a limited spat involving one particular book and a few people, nothing much fundamental about peer review, I think. A more useful resource for improving this article could be this month's issue of The Scientist (see here). --Crusio (talk) 12:53, 17 August 2010 (UTC)

But the issues raised there are relevant to 'peer review' generally, right? It's a contemporary representation of how the idea of peer review has been appropriated and then disputed. I would have thought it somewhat relevant. The Sound and the Fury (talk) 16:32, 17 August 2010 (UTC)

Yeah, sorry, didn't want to appear to just brush it off. However, such kind of disputes flare up every now and then at different places. But the links you gave can be used as sources, if needed (well, except for the blog, of course).

Merge proposal

Currently there are several articles dealing with variants of peer-review. I think they should all be merged into this article (merging them would probably allow us to focus the article and give it a global structure). The concerned articles are:

• Anonymous peer review
• Open peer review
• Independent review

Opinions? Headbomb {talk / contribs / physics / books} 17:31, 14 August 2010 (UTC)

• Makes a great deal of sense to me. Independent review is already redirected to this article, although there was a stub previously which might have a couple of points worth carrying across. Anonymous peer review could pretty much be copied over as is. That wouldn't leave much of a justification for keeping Open peer review separate. andy (talk) 18:41, 14 August 2010 (UTC)

• I don't think the current article goes into enough detail to distinguish between the sort of anonymous peer review used by journal editors before publication; and the kind of "independent review" (if that is the right term) of published research and analysis that helps the scientific community distinguish between irreproducible results and pseudoscience on the one hand, and novel, useful discoveries on the other hand. The term "peer review" is used for two significantly different practices or customs:
  1. A scientist's peers, general known to him (and with his knowledge) check his experimental methods, data collection and filtering strategy, analysis, etc., and comment publicly on all this - especially if they disagree;
  2. A scientist's peers, in the strictest confidence (and thus without his knowledge), help a journal editor decide a scientific paper has enough merit to be published. --Uncle Ed (talk) 22:14, 15 August 2010 (UTC)

• Support merge. All these should be merged here (and the recently created Category:Peer review can then be deleted). As for post-publication review: as long as articles are involved, this usually is called something like "(open) peer commentary", not really "peer review" any more. As for doing experiments to replicate somebody else's results, I don't think that has ever been called "peer review" (or anything including the word "review"). I'm not even sure there is a specific term for this (apart from "replication") as this is just part of the normal scientific process. If somebody published erroneous results/conclusions (either as an honest mistake or -much more rarely- intentionally), sooner or later people will come forward saying that they are unable to replicate something and scientific hypotheses will be re-formulated based on that. --Crusio (talk) 12:59, 17 August 2010 (UTC)

  I support merging of any article that can be merged, although some article could remain as standalone with just a summary brought here (see Wikipedia:Summary style). I'm a big fan of "once and only once", and have been ever since I found out how much more productive this makes me as a software developer.

  However, there are many articles on peer review here, and we need to think carefully together about how we can assimilate common elements without losing unique aspects. --Uncle Ed (talk) 16:52, 17 August 2010 (UTC)

I have fully merged Anonymous peer review, which was only a stub, but I've merged only the (corrected) summary of Open peer review, because that article has longer history section which needs some careful integration in this article's global historical view if a full merge is desired; I'm on the fence on that one. Tijfo098 (talk) 00:08, 22 October 2010 (UTC)

I think this article needs some clean up but a merger is not a bad idea at all. The user needs to understand what peer-review is. I do think that we should consider what peer-review is for say people (academic promotions, review within professional organizations) vs. peer review for publications such as journals, grants, etc. --Hosserman (talk) 21:12, 23 October 2010 (UTC)

What was the outcome of this discussion? Merge or not? In the meantime there are dirty big tags up the top of the page. The Sound and the Fury (talk) 00:26, 27 November 2010 (UTC)

• I would say that there was a rough consensus to merge, but apparently nobody has had the time or the inclination yet to do this... Feel free to jump in... --Crusio (talk) 08:31, 27 November 2010 (UTC)

• I think that open peer review does not have much content that needs merging here. Most of it is specific to certain journals and, if not already covered there, should be merged to the articles on the journals. The article itself should be redirected here. The text here should be edited to bring it inline with the other definitions ("a scientific literature concept and process" is not really that informative). --Crusio (talk) 12:31, 18 January 2011 (UTC)

• Support merging all as well as Peer review failure, as I just proposed at Talk:Peer review failure#Merger proposal. Like I said there, reliable sources do not cover this material much, aside from the occasional editorial in a scientific journal. I think Wikipedia probably just needs one Peer review article. –CWenger (talk) 15:35, 27 March 2011 (UTC)

• Lone voice of dissent :) I dunno guys - this article is pretty lengthy already. Could we summarise the sections and bring the supporting subjects in to satellite articles? This would make this article more consise and accessible with beefier support articles? Posting anon as I don't think this will be popular! 90.199.170.119 (talk) 01:17, 7 April 2011 (UTC)

Clean-up needed

I tentatively support merging independent review here as well, but this article needs a good scrubbing first because it often uses peer review to mean just prepublication peer review, which while common in science, is not so common in other fields (in some fields not at all), which use different forms of review. If independent review were merged here now massive confusion will result because of the way the peer review article is currently over-focused on science prepublication reviews. It's a major job to cleanup this article. Perhaps a sub-article on prepublication reviews is needed to achieve balance. Note that we also have physician peer review and medical peer review where the term is used a synonym for performance appraisal. Tijfo098 (talk) 01:36, 23 October 2010 (UTC)

The medical peer review even has this hatnote:

For the review of scholarly papers in refereed journals, see peer review.

So, the scope of this article needs clarification. I have the impression that it was started focusing on prepublication reviews or perhaps academic papers reviews, but it has drifted over time. I'm not sure what would be easier: refocus this article to reflect the current lead, or make it regain its seemingly original focus on the academic publishing while writing a decent overview at independent review. YMMV as they say. Tijfo098 (talk) 01:56, 23 October 2010 (UTC)

• I have had another look at independent review and don't see anything that has not been covered elsewhere or should be merged here. I propose to simply redirect it here. --Crusio (talk) 12:28, 18 January 2011 (UTC)

Terminology: "reviewer" or "referee" ?

This article uses both terms "reviewer" and "referee" somewhat interchangeably, but does not clearly state whether they are synonyms or whether there is some difference. A clarification would be helpful. Boute (talk) 03:54, 11 September 2010 (UTC)

I clarified that in the lead. Tijfo098 (talk) 01:27, 23 October 2010 (UTC)

Article needs a preening for terminology

It often uses peer review to mean prepublication peer review, which is very common in the science vernacular, but sources that write about these matters, e.g. [11] are often more careful with the distinctions. Tijfo098 (talk) 01:26, 23 October 2010 (UTC)

Useful source

"Closing the Climategate [Editorial]". Nature. 468 (7322): 345. 2010. Bibcode:2010Natur.468..345.. doi:10.1038/468345a. besides its main topic has some citable part for the purpose of this article. Tijfo098 (talk) 17:56, 21 November 2010 (UTC)

new section about Citation impact

There are many peer review articles that discuss the importance of citations of peer reviewed articles. So a new a section is needed. Albertttt (talk) 11:35, 18 January 2011 (UTC)

• Absolutely correct. That's why we have several articles about citation analysis, impact analysis, etc. This article, however, is about peer review, and citations are not generally seen as "peer review" (not even "post-publication"). So that new section is not needed here and adequately covered elsewhere. --Crusio (talk) 11:56, 18 January 2011 (UTC)
• Agreed - this material is not relevant to this article. In fact Albertttt has already contributed to Citation impact. andy (talk) 11:59, 18 January 2011 (UTC)
• "See also" seems to the place to put it then. Albertttt (talk) 12:08, 18 January 2011 (UTC)

• As Crusio said earlier citations don't really have anything to do with peer review, so it would be misleading to direct people to the citation impact article. andy (talk) 12:10, 18 January 2011 (UTC)

Citations, in drafts of peer reviewed articles, help determine who will conduct the peer review.Albertttt (talk) 12:16, 18 January 2011 (UTC)

That should be in the "recruiting referees" section — Preceding unsigned comment added by Albertttt (talk • contribs) 12:21, 18 January 2011 (UTC)

• As you say yourself, that is something completely different. Also, I doubt that there is more than some anecdotal evidence for that (even though I agree that this may happen: if an editor does not directly think of suitable reviewers, it's easy to look through the list of references, because that will mostly be people working on the same or similar subject). And my personal experience as an editor suggests that it is the vast minority of articles where this happens (after all, editors are selected for their knowledge of a field and the people working in it). --Crusio (talk) 12:23, 18 January 2011 (UTC)
• Clarification: are you saying that the fact that peer reviewed articles must cite other peer review articles don't belong to the peer review article ?Albertttt (talk) 12:31, 18 January 2011 (UTC)

• Perhaps I don't understand your comment. I thought you were referring to "recruiting reviewers". Apart from that, peer-reviewed articles don't have to cite other peer-reviewed articles. they can cite non-reviewed stuff (newspaper articles, for instance) and can even not cite anything at all (although that is so unusual that I cannot think of any example -at least of a post 1960 paper). However, whether or not peer-reviewed articles need to cite other peer-reviewed articles is, again, not something that belongs here, but in articles on academic articles themselves. This article is about the process of peer review, not about the things that are reviewed. --Crusio (talk) 12:37, 18 January 2011 (UTC)

Who promoted the original peers to "peer" status?

76.120.17.197 (talk) 01:20, 16 December 2012 (UTC)

Revert of merge from Open peer review

@Christian75: I am unclear as to why you reverted the merge, when it has support on the talk page. Although you highlighted in your edit summary that the article will need to be rewritten, I do not see how a complete reversal of the merge is conductive to this goal. --NickPenguin_(contribs) 04:25, 5 March 2014 (UTC)

Dear @NickPenguin:, @Christian75: and @Randykitty:: At the moment the contents of "Open peer review" has disappeared completely. Could you please decide what to do first and only then edit the pages?--RolfSander (talk) 12:32, 5 March 2014 (UTC)

As I said, the merge was reverted, and it appears to have been partially restored. I have fully restored the merge. --NickPenguin_(contribs) 14:07, 5 March 2014 (UTC)

Corruption

There needs to be a section on corruption in anonymous reviewing by journals. Theft of intellectual property through this reviewing process is untraceable and systemic. Likewise, unscientific obstruction of publication by rivals, usually based on such unsubstantiated opinion as is never allowed in articles under review.Bourdillona (talk) 01:30, 26 May 2011 (UTC)

• You'll need reliable sources for that. I doubt those exist, especially if you talk about this kind of abuse being "systemic". For my own publications, I have never experienced this kind of abuse and almost all reviewers were courteous and gave constructive criticisms. --Crusio (talk) 01:37, 26 May 2011 (UTC)
• I don't need those sources - you do. The mafia uses a secretive honor system to the same effect. Many, if not most, scientists have experienced the theft and it has been often discussed. If anonymous reviewing is not corrupt, then you have to say why it is not. Journals which are not transparent are not scientific. Editors do not normally respond to authors' complaints. They use stone-walling stock phrases such as "the referee has given his opinion that your paper should not be published and therefore I must reject it." There is no science in anonymous, false opinions given for whatever motive, without justification or oversight. For the mediocre, it's fine. Bourdillona (talk) 11:46, 27 May 2011 (UTC)

• Nope, that's not the way things work (here or in real life). If you accuse somebody or something it is up to you to prove the accusation, not up to the accused to prove innocence. --Crusio (talk) 11:51, 27 May 2011 (UTC)

• I didn't accuse anyone. Please be real. Look for a moment at the system. You give no justification for your opinion. — Preceding unsigned comment added by Bourdillona (talk • contribs) 12:02, 27 May 2011 (UTC)

• Yes, you do. "Theft", "mafia", "corrupt", "not transparent", "stone-walling", etc. Prove it and it can be in the article. In my experience and that of all researchers that I know, the kind of things you mention are (very rare) exceptions and only the mediocre complain that it is otherwise. --Crusio (talk) 12:23, 27 May 2011 (UTC)

You don't name the person that I am supposed to have accused. One problem is that it is precisely because of anonymity that these things happen. It is impossible to reverse them. I have previously cited a book on this issue. As I remember you misrepresented it and deleted the citation with the result that it was subsequently misquoted. From my point of view the point is made and the case is closed.Bourdillona (talk) 01:52, 1 June 2011 (UTC)

I've never heard of it myself directly, but theft does happen - quoting from a letter on the British Medical Journal: "Furthermore, it is not unknown for original ideas to be purloined while turning down a paper." [12]. I would simply expect that reviewers of high-impact journals wouldn't risk their career doing that. However, Bourdillona, for everything on Wikipedia there are rules about what you can write, which Crusio linked. Basically, there must be reliable sources. For "obstruction of opinion by rivals", that's already discussed in Open peer review, with appropriate citations and studies about it.--Blaisorblade (talk) 01:33, 23 October 2011 (UTC)

Here is some proof that peer review is subject to corruption: http://www.washingtonpost.com/news/morning-mix/wp/2014/07/10/scholarly-journal-retracts-60-articles-smashes-peer-review-ring/ — Preceding unsigned comment added by 67.182.153.76 (talk) 15:47, 10 July 2014 (UTC)

"it's only the predatory journals that claim to have peer review but don't really do a serious review (if any at all)"

Randykitty has said that "it's only the predatory journals that claim to have peer review but don't really do a serious review (if any at all)" [13]. To mention only one, recent and extensively documented counter-example, Retraction Watch has reported an investigation carried out by out by SAGE Publications which revealed a "peer-review ring" where, amongst other things, "[an] author reviewed his own paper under one of the aliases he had created" [14]. This peer review failure resulted in the retraction of 60 papers in the Journal of Vibration and Control, which claims to be peer-reviewed [15] and is not considered a predatory journal (not least because it's not open access, and apparently predatory closed-access journals are non-notable). I'd recommend browsing Retraction Watch for many other counter-examples. Let me know if any further information is needed, otherwise would you please cease reverting related edits. Fgnievinski (talk) 00:58, 12 December 2014 (UTC)

• You are constantly mixing up different things. 1/ Predatory journals: they promise peer review but knowingly do not carry it out (or at best perfunctorily). 2/ Schon affair and others: these were case of fraud by the researchers. You can call that a "failure of peer review" if you like, although peer review was and is not designed to detect fraud. (And without detailed on-site investigation, detection of fraud is almost or even completely impossible). Many of Schon's articles were published in Science. Do you really want to argue that Science is a fake scam like the predatory OA journals? 3/ The peer-review ring that you mention is a case where legitimate journals were the victim of fraudulent authors. 4/ The peer-review ring was discovered and publicized by Sage themselves (and more recently a similar case was unearthed by BioMed Central. That's what a legitimate publisher does: they investigate these case and do not hide them. I still have to see the first predatory journal that comes with an investigation concluding "actually, we're fake". Indeed, I do not know of any non-OA predatory journals and doubt that any exist. The reason for that is very simple, the predatory OA scam can only work for OA journals, because how would non-OA journals make any money? Any librarian worth their mettle would recognize their low quality and none would subscribe to it. Please try to understand the issues before making further edits to this page. Thanks. --Randykitty (talk) 09:58, 12 December 2014 (UTC)
  • Your categorization of types of peer review failures is useful: pretense is but one type. In (1) you introduce a definition of predatory journal which dispenses with the open-access requirement (contrary to Beall's definition). Then you conclude that non-OA predatory journals cannot exist because there would be no money involved. Yet there's another type of currency which can be involved, and that is vanity, much sought-after for career advancement, valued by bean-counters, etc. Now I'll make an assumption which you might reasonably contend: editors-in-chief represent the journal; therefore if the former are corrupt, so is the latter. Finally, below I give several examples where editors-in-chief corrupted the peer review system that they were supposed to oversee. Any reason why these wouldn't serve as bonna fide examples of type-1 peer review failures (pretense of peer review) in non-OA journals? Thanks. Fgnievinski (talk) 07:28, 22 December 2014 (UTC)
    • Elsevier#Sponsored journals: "had the appearance of being a peer-reviewed academic journal but in fact contained only articles favourable to Merck drugs"; Elsevier Published 6 Fake Journals
    • Elsevier#Chaos, Solitons & Fractals: "The journal had published 322 papers with El Naschie [the editor-in-chief] as author". Self-publishing editor set to retire Controversial Math Journal Relauches: New Editors, Focus on Rigorous Review
    • Who ensures the integrity of the editor?
    • Retraction Watch piece: Editor in chief steps down after being found plagiarizing in her own journal: "Gomes [the plagiarism offender] is no longer co-editor-in-chief; the journal now lists her as founding editor."

Sorry, but you keep mixing up things and misunderstanding what I am saying. In (1) I am not defining predatory OA journals, I just say something about the peer-review that they practice (or not, as the case may be). I never claimed that peer review never fails. I do claim that predatory money scams wouldn't work for print journals, at least I do not know of any example of that. Vanity is what indeed is involved from the side of authors and from those academics who lend their names to be editors or members of the editorial boards of these sub-par journals. However, the bean counters are not completely stupid: publishing in a sub-par journal is often not counted as a "bean" :-). In the cases that you list above where peer-review was corrupted by editors, the publishers took immediate action when the problem was brought to their attention (the failure being that they didn't detect these problems themselves, although that often happens, too, such as the recent uncovery of a peer-review scam ring by BioMed Central). Predatory publishers won't try to detect such failures and try to cover things up if they are brought to their attention. Indeed, almost invariably, they knowingly put bad peer-review practices in place, as their interest is to publish as many articles as possible, not to have any quality control. So it boils down to this: "legit" publishers will sometimes have problems, too, but they will try to avoid these and correct when the problems are detected, whereas "rogue" publishers will knowingly engage in bad publishing practices. --Randykitty (talk) 08:33, 22 December 2014 (UTC)

• It is a matter of degree: although peer review failure is a defining feature of predatory journals, occasional occurrences exist elsewhere (e.g., SCIgen#In journals, Elsevier#Sponsored journals). I'm citing these documented cases of peer review failure in non-predatory journals. Let's focus on the text as in the body of the article. Fgnievinski (talk) 21:55, 28 December 2014 (UTC)

• No, it's not a matter of degree, it's a matter of intent. A legitimate publishers strives for high-quality editorial processes, and even though they occasionally may fail to deliver, that is because of honest error (or in the case of the fake Elsevier journals, a rogue employee). A predatory publisher has no intent at all to provide any quality, all they want to do is make an easy buck. It's the same difference as somebody intentionally driving a car into a crowd to kill people or someone getting a heart attack and losing control of the steering wheel. The results may be similar, the events are very different. --Randykitty (talk) 22:35, 28 December 2014 (UTC)

new study of peer review

See: Campos-Arceiz, A, Primack, RB, and Koh, LP: "Reviewer recommendations and editors’ decisions for a conservation journal: Is it just a crapshoot? And do Chinese authors get a fair shot?" Biological Conservation (June, 2015; free for the next 6 months). For a description of the findings, see [http://www.elsevier.com/connect/is-peer-review-just-a-crapshoot here. --Randykitty (talk) 07:27, 15 April 2015 (UTC)

Peer review in mega journals

For a critical note on the quality of peer review in mega journals, see Mega-Journals and Peer Review: Can Quality and Standards Survive?. --Randykitty (talk) 07:34, 15 April 2015 (UTC)

splitting off non-scholarly peer review

There are three small sections about Prefessional, Government, and Medical peer-review. The rest, ~ 90% of the article, is about scholarly peer review -- not only the homonym section, but certainly also section "Improvement efforts" and apparently also "Criticism of peer review". First: do you agree with splitting off the material about non-scholarly peer review? Second: how should the two articles be titled? Thanks. Fgnievinski (talk) 17:02, 29 November 2014 (UTC)

• I intend to split Peer review#Scholarly peer review into its own article and keep the present one as a WP:broad-concept article, with its general definition and overview of specific instances (e.g., medical peer review). Fgnievinski (talk) 04:13, 20 June 2015 (UTC)

• Here's a draft: Talk:Scholarly_peer_review/Draft. Fgnievinski (talk) 01:43, 22 June 2015 (UTC)

• Done: Scholarly peer review. Fgnievinski (talk) 03:39, 24 June 2015 (UTC)

Photo

A reviewer at the American National Institutes of Health evaluates a grant proposal.

Really? Is this how the grant proposals are evaluated at the American National Institutes of Health? Indeed. --Дядько Ігор (talk) 11:13, 6 September 2015 (UTC)

External links modified

Hello fellow Wikipedians,

I have just added archive links to one external link on Peer review. Please take a moment to review my edit. If necessary, add {{cbignore}} after the link to keep me from modifying it. Alternatively, you can add {{nobots|deny=InternetArchiveBot}} to keep me off the page altogether. I made the following changes:

• Added archive https://web.archive.org/20100306064610/http://www.ama-assn.org:80/ama/pub/physician-resources/legal-topics/medical-peer-review.shtml to http://www.ama-assn.org/ama/pub/physician-resources/legal-topics/medical-peer-review.shtml

When you have finished reviewing my changes, please set the checked parameter below to true to let others know.

This message was posted before February 2018. After February 2018, "External links modified" talk page sections are no longer generated or monitored by InternetArchiveBot. No special action is required regarding these talk page notices, other than regular verification using the archive tool instructions below. Editors have permission to delete these "External links modified" talk page sections if they want to de-clutter talk pages, but see the RfC before doing mass systematic removals. This message is updated dynamically through the template {{source check}} (last update: 5 June 2024).

• If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
• If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—^{cyberbot II}_{Talk to my owner:Online} 10:52, 1 January 2016 (UTC)

News

Romano, Aja. «Academic journals are facing a battle to weed out fake peer reviews». The Daily Dot, 21 agost 2015 --Hienafant (talk) 21:22, 16 September 2015 (UTC)

The "Article" says; "peer review is common in decisions related to faculty advancement and tenure.[citation needed]". Here is the fatal flaw in Peer Review, which will be its downfall. Pieraccini wrote to get past peer review, not to communicate science. The founder of Wikipedia said he wanted "peer reviewed" literature. This ignored the Wikipedia entry on T S Kuhn, who discusses "The Structure of Scientific Revolutions". Obviously, Peer Review will block even the suggestion of paradigm change. - Ivor Catt 28 July 2016 http://www.ivorcatt.co.uk/x59b1.htm http://www.ivorcatt.co.uk/x66k.htm — Preceding unsigned comment added by 109.151.32.208 (talk) 10:37, 28 July 2016 (UTC) 1.129.97.125 (talk) I concur. I have attempted to have a paper reviewed that provided a simple and comprehensive solution to a large group of scientific mysteries. I did not expect it to succeed because it would invalidate numerous preposterous theories that are proposed by recognized scientists. Moreover, If the thesis of my paper is correct, I have no 'peers'.

I have had a true copy notarized by a local courthouse so that I can claim priority for the concept when it eventually surfaces and so I can present evidence of submission to respected societies of peers. I recommend this process to others who wish to claim authorship of their ideas. The Notary's/Justice of the Peace's signature establishes the date the copy was made. I also intend to donate a copy to my local library, and register it with our national archives, thereby publishing it.

Bible is first and most peer-reviewed journal of documents

The process used by the early Christian church leaders in the life of the new church established the concept of peer review, later adopted by early church-related academic institutions. This well-documented effort is likely the basis of all peer review. Peer review of the Bible continues through commentators and skeptics. This is the essential process of peer review. (24.165.81.152 (talk) 15:22, 24 January 2016 (UTC)Dr. Dwight B. Reimer, DocReimer@gmail.com, Jan 24, 2016)

That sounds like an odd thing to say. But if it is well documented, please link to some of the documentation William M. Connolley (talk) 15:44, 24 January 2016 (UTC)

Bible criticism is definitely very important to the development of literary criticism, but it has no clear connection to peer review. The Bible is not a "journal," and the texts of the Bible -- today, at least -- are not viewed as potential journal articles to be reviewed by "peers." A peer reviewer's job is to recommend whether or not a publishing body should accept a particular submission for publication and, when possible, to suggest revisions that would strengthen the argument of the submission. "Canon formation" works well as a term for talking about what the early church did, but "peer review" doesn't work as well. Jk180 (talk) 15:26, 10 March 2016 (UTC)

That sounds like a very odd thing to say but it may depend on who you describe as 'early Christian leaders'. The Christian church in the early going did their very best to 'rationalize' Christian teachings, carefully dictating what they considered to be the 'right' teaching, destroying evidence that contradicted these themes, and demonizing those who offered alternative views. While what you describe may refer to a later time and more specific context, it is difficult to believe that the Church until the last century or so (when science forced it into a corner) would have surrendered what it considered to be a political and religious position in the name of anything objective. Isn't objective truth the ultimate purpose of peer review?--50.68.140.76 (talk) 20:07, 24 July 2018 (UTC)

Nothing here on the impact of funding on Peer Review

I read an article recently that bemoaned the decline in the availability of funding for peer review, leading to a startling decline, especially in the social sciences, of studies that were being properly reviewed--as low as 13% in some disciplines. Funding is difficult enough to come by, and peer review studies are just not attractive to funders: if they confirm the original study then the original researchers get all the glory, and if they don't, still nobody gains.

But the impact is worth addressing here. The result of this is that researchers have less incentive to exercise a rigid discipline over their methods (which costs money too), and increasingly, studies that haven't been properly reviewed and may well be wrong, are now quoted by other researchers. --50.68.140.76 (talk) 20:15, 24 July 2018 (UTC)

I think you're confusing peer review and replication. --Wikiman2718 (talk) 09:00, 19 July 2019 (UTC)

Content moved over from Open access

I've moved over a couple of sections from the open access page, witten by nemo bis since I think it fits here better in the #criticism section here than it did at its previous location (discussion). It probably still needs a bit of editing for tone and consistency with the rest of the page. T.Shafee(Evo&Evo)^talk 11:35, 31 December 2019 (UTC)

Einstein

If you find Einstein's reaction to anonymous peer review interesting , read Hate the peer-review process? Einstein did too and Einstein Versus the Physical Review --Error (talk) 10:01, 4 May 2020 (UTC)

Adding sources for edits

Hyland, Ken; Feng (Kevin) Jiang. (2020). 'This work is antithetical to the spirit of research ': An anatomy of harsh peer reviews.  Journal of English for Academic Purposes 46, 1-13. https://doi.org/10.1016/j.jeap.2020.100867

https://doi.org/10.1016/j.compcom.2019.01.005

Ilkos, Melpomeni. (2018). Using Peer Review with Greek EFL College Students.  Journal of Academic Writing 8.2 [Special issue: EATAW Conference 2017], 89-113. https://doi.org/10.18552/joaw.v8i2.447

Warnsby, Anna; Asko Kauppinen; Laura Aull; Djuddah Leijen; Joe Moxley. (2018). Affective Language in Student Peer Reviews: Exploring Data from Three Institutional Contexts.  Journal of Academic Writing 8.1, 28-53. https://doi.org/10.18552/joaw.v8i1.429

Contemporary Peer Review: Construct Modeling, Measurement Foundations, and the Future of Digital Learning, by Ashley Nichole Reese, Rajeev Reddy Rachamalla, Alex Rudniy, Laura Aull, Dave Eubanks Roach Jefferson (talk) 00:46, 2 March 2023 (UTC)