Wikipedia:Reliable sources/Flaws

This is a list of flaws of WP:RS, written by User:Phil Sandifer.

Flaws of WP:RS

As requested, an inventory of the flaws in this page. I hope you will see that the flaws are deeply rooted, affecting the organization and goal of the page. Not only is this an inadequate guideline for sourcing, it is an inadequate foundation for a guideline. David is correct – this needs to be rewritten from the ground up.

There are three basic flaws of this page, and they'll recur throughout the below.

1. The page arbitrarily attempts to rule out subjective judgment in some cases while mandating it in others. The cases where it mandates it are often masked as didactic guidelines that depend on phrases like "reliable X," "common Y," or a mushy definition of fact. These phrases could be defined in the same way that "reliable source" has been, but such definitions would necessarily run into the same problems of subjectivity. It's turtles all the way down.
2. The page is, at numerous points, clearly written for a narrow range of topics. When applied to other topics, it ranges from the merely unhelpful to the completely wrong.
3. Sections flatly contradict each other.

More details follow.

Opening

from Wikipedia:Reliable sources:

Wikipedia articles should use reliable published sources. This page provides guidance about how to identify these. The two policy pages that discuss the need to use sources are Wikipedia:No original research and Wikipedia:Verifiability.

If you can provide useful information to Wikipedia, please do so, but bear in mind that edits for which no reliable references are provided may be removed by any editor. The responsibility for finding and adding references lies with the person adding material to an article, and sources should be provided whenever possible.

What follows is a description of Wikipedia's best practices. Many articles may fall short of this standard until editors devote time and effort to fact-checking and reference-running. (See efforts to identify reliable sources.) In the meantime, readers can still benefit from your contributions, bearing in mind that unsourced edits, or edits relying on inappropriate sources, may be challenged and removed at any time. Sometimes it is better to have no information at all than to have information without sources.

There are many ways in which factual errors can be introduced into reports. Keep in mind that many articles are about characterizing the various factions in a dispute. This means that you will be looking for reliable published reports of people's opinions.

The opening section in general and paragraph three in particular frame the page in an astonishingly flimflam way. The prospect of unsourced information being removed is raised, but no serious suggestion is given as to when "it is better to have no information at all than to have information without sources." The result is to give a vague and nonspecific warning when, in fact, a specific warning is in order. We are, after all, talking (at least primarily) about BLP here.

Definitions

from Wikipedia:Reliable sources#Definitions:

Please note the following terms:

• A fact is an actual state of affairs. To say of a sentence or proposition that it is true is to say that it refers to a fact. As far as the encyclopedia is concerned, a fact is a statement agreed to by the consensus of scholars or experts working on a topic. (New evidence might emerge so that the statement is no longer accepted as a fact; at that time the encyclopedia should be revised.)

Assert facts, including facts about opinions—but don't assert opinions themselves. That a survey produced a certain published result is a fact. That there is a planet called Mars is a fact. That Plato was a philosopher is a fact. No one seriously disputes any of these things, so we can feel free to assert them.

• An opinion is a view that someone holds, the content of which may or may not be verifiable. However, that a certain person or group expressed a certain opinion is a fact (that is, it is true that the person expressed the opinion) and it may be included in Wikipedia if it can be verified; that is, if you can cite a good source showing that the person or group expressed the opinion.

• A primary source is a document or person providing direct evidence of a certain state of affairs; in other words, a source very close to the situation you are writing about. The term most often refers to a document produced by a participant in an event or an observer of that event. It could be an official report, an original letter, a media account by a journalist who actually observed the event, or an autobiography. Statistics compiled by an authoritative agency are considered primary sources. In general, Wikipedia articles should not depend on primary sources but rather on reliable secondary sources that have made careful use of the primary-source material. Most primary-source material requires training to use correctly, especially on historical topics. Wikipedia articles may use primary sources only if they have been published by a reliable publisher e.g. trial transcripts published by a court stenographer, or historic documents that appear in edited collections. We may not use primary sources whose information has not been made available by a reliable publisher. See Wikipedia:No original research and Wikipedia:Verifiability

• A secondary source summarizes one or more primary or secondary sources. Secondary sources produced by scholars and published by scholarly presses are carefully vetted for quality control and can be considered authoritative.

• A tertiary source usually summarizes secondary sources. Encyclopedias, for instance, are tertiary sources.

When reporting facts, Wikipedia articles should cite sources. Wikipedia is a tertiary source. Wikipedia cannot cite itself as a source—that would be a self-reference. (However, when writing in the summary style detailed referencing may only be necessary in the subarticle and not the summary.) There is a wealth of reliable information in tertiary sources such as the Encyclopædia Britannica. Note that unsigned Encyclopædia Britannica, World Book, and Encarta articles are written by staff, who may not be experts, and the articles may therefore not have the same level of credibility, but they are regarded as reliable sources for Wikipedia's purposes. When wikipedians have the ambition to write a better encyclopedia entry than those extant, it does not suffice to rely on the content of such tertiary sources. Therefore, in general, as primary sources are also to be treated with caution (see above), secondary sources are the stock material on which Wikipedia articles depend for their references.

When reporting that an opinion is held by a particular individual or group, the best citation will be to a direct quote, citing the source of the quote in full after the sentence, using a Harvard reference, a footnote, or an embedded link. See WP:CITE for more details. If there is text, audio, or video available of someone expressing the opinion directly, you may include or transcribe an excerpt, which is allowed under fair use.

The definition of fact is too iron-clad, relying ultimately on the idea that nobody "seriously disputes" a claim. By this standard, neither evolution nor global warming are facts. No serious encyclopedia should assert that evolution is not fact, but this definition of fact leads us inexorably to that conclusion.

The problem is exacerbated in the next definition, whereby opinions are yolked to verifiability, or lack thereof. This contrast between fact and opinion leaves a vast no man's land of verifiable information that people still disagree with. Worse, this no man's land is not self-evident. Does the statement "The US war in Iraq was conducted based on false information regarding the presence of WMDs in Iraq" count? It's verifiable, but it's not universally held.

To be clear, THERE IS NO WHITE LINE DEFINITION of fact and opinion that will let us automatically tell if a statement should be phrased as fact or as "X thinks Y." There are white line cases – Mars is a planet, Christians believe that Jesus is the son of God. But there's a vast middle ground that needs to be taken case by case.

Onward, the definitions of sources are based on old language, but have been altered past the point of usefulness. The biggest problem is the idea that primary sources require a reliable publisher. Here the reliable publisher is being used as a sort of surrogate secondary source – we're not using the primary source, but rather the primary source under the imprimatur of Publisher X. All of this is justified under the idea that "most primary-source material requires training to use correctly." (An idea that is untrue. They require care to use correctly. Training is often given to help people become more careful, but it is not the training that is required. There is a difference between using historical archives and running a nuclear reactor.)

The problem is that the same can be said of secondary sources. Even ones published by scholarly presses, which are, oddly, the only secondary sources endorsed, creating a system whereby we are bound entirely to scholarly sources on a topic. (Thankfully, the page is not consistent enough to endorse that position throughout) Secondary sources require just as much care as primary sources, and the privileging of them is nonsense.

The original statement on primary sources from which all of this derived stated that it's original research to organize primary sources in a "novel" fashion. But it never set up such a wide-ranging ban on primary sources, and with good reason.

Finally, the distinction makes no light of the fact that the primacy of sources is contextual. An example of just how bad this can get: There is a poem by Yeats called "Among Schoolchildren." The literary critic Paul de Man has a famous reading of this poem that has been published by scholarly presses. This reading is a secondary source in terms of the Yeats poem. But despite its scholarly status, it's tremendously contested, since de Man is a controversial deconstructionist – widely recognized as brilliant and important, but not always agreed with. Furthermore, de Man's reading is a secondary source ONLY on the topic of Yeats. It's a primary source on the topic of de Man. And due to the nature of the field, secondary sources on de Man's reading are virtually all critical – not because his reading does not have adherents, but because publishing an article reconfirming de Man's opinion is not considered a useful publication, and so only contrary opinions get published.

This is neither an overly convoluted example, nor an unusual one. There is an endless list of topics that require this level of thought to untangle the nature of primary and secondary sources.

The hedging about using other encyclopedias is a pleasant and rare example of actually molding the guideline to reality, but is done in a uselessly clumsy way, amounting to "unsigned articles in encyclopedias aren't good enough, but we use them anyway." The result moves towards a hardline sourcing guideline with a thousand asterisked exceptions. This is not a direction in which a usable guideline can be found.

Unattributed material

From Wikipedia:Reliable sources#Unattributed material:

Wikipedians often report as facts things they remember hearing about or reading somewhere, but they don't remember where, and they have no corroborating evidence. It is important to seek reliable sources to verify these types of reports, and if they cannot be verified, any editor may delete or challenge them.

It is always appropriate to ask other editors to produce their sources. The burden of evidence lies with the editor who has made the edit in question, and any unsourced material may be removed by any editor. However, some editors may object if you remove material without giving people a chance to find a source, particularly when the material is not obviously wrong, absurd, or harmful. Instead of removing such material immediately, editors are encouraged to move it to the talk page, or to place the {{fact}} template after the disputed word or sentence, or to tag the article by adding {{not verified}} or {{unsourced}} at the top of the page. See Wikipedia:Verifiability and Wikipedia:No original research, which are policy, and Wikipedia:Avoid weasel words.

Do not, however, remove statements that you believe to be both true and common knowledge, simply because they aren't sourced. Don't, for instance, remove a reference to "earth's elliptical orbit" simply because the writer has not supported the assertion that planetary orbits are elliptical.

If you do honestly disbelieve a statement, do remove it and request a source on the talk page. If you do honestly think it isn't common knowledge, do tag it as requiring a reference or query it on the talk page. And do be skeptical about claims of "common knowledge" about people, especially living people. Gossip is not common knowledge.

The instruction not to remove material that you believe to be true and common knowledge is important, but too weak. The reference to Wikipedia:Common knowledge is distressing, both because it's unclear to me that the example given (Earth's elliptical orbit) satisfies the criteria there, and because that page seems to me to have a section that tries to depricate WP:AGF in favor of wikiquette. This use of common knowledge is equivalent to the fact/opinion mess above – a situation where there is no white line is being phrased as though there were a white line. In reality, the judgments over what does and doesn't need a source are largely done on a case-by-case basis – not by referring to a definition of common knowledge.

This points towards a larger flaw in our understanding of sources. We are basically set up to defer almost universally to the person asking for a source. In reality, we need to recognize that source requests can be made in bad faith or in error. There are many cases where the answer to a request for a source is "No." Requesting a source needs to not be fetishized as an innately reasonable act. Like adding dispute tags or cleanup tags, it is often a reasonable and helpful act. It can also be a trolling or stupid act.

Beware false authority

From Wikipedia:Reliable sources#Beware false authority:

Look out for false claims of authority. Web sites that have numerous footnotes may be entirely unreliable. The first question to ask yourself is, "What are the credentials and expertise of the people taking responsibility for a website?" Anyone can post anything on the web.

Use sources who have postgraduate degrees or demonstrable published expertise in the field they are discussing. The more reputable ones are affiliated with academic institutions. The most reputable have written textbooks in their field: these authors can be expected to have a broad, authoritative grasp of their subject. In general, higher education textbooks are frequently revised and try to be authoritative. Textbooks aimed at secondary-school students, however, do not try to be authoritative and are subject to political approval.

This section starts off by demonstrating the problem with valorizing secondary sources – the process of vetting them is, in the end, subjective, and often no easier than vetting primary sources. The sole privileging of academic sources continues here, impovershing fields with less academic research. (Popular culture, current events, things related to homelife [Washing machines, cooking, furniture], non-theoretical aspects of computing, etc.).

The discussion of textbooks is valid only for the sciences and to a lesser degree the social sciences (Where it is valid basically on an undergraduate level only). It is utterly useless for the humanities, where few college textbooks exist, and fewer still are assigned.

Exceptional claims

From Wikipedia:Reliable sources#Exceptional claims require exceptional evidence:

Certain red flags should prompt editors to closely and skeptically examine the sources for a given claim.

• Surprising or apparently important claims that are not widely known.
• Surprising or apparently important reports of recent events not covered by reputable news media.
• Reports of a statement by someone that seems out of character, embarrassing, controversial, or against an interest they had previously defended.
• Claims not supported or claims that are contradicted by the prevailing view in the relevant academic community. Be particularly careful when proponents say there is a conspiracy to silence them.

"Reputable news media" is yet another case of a deep well of subjective judgment masquerading as a white line policy. "Reports of a statement by someone that seems out of character, embarrassing, controversial, or against an interest they had previously defended" is similarly flawed.

Evaluating sources

Editors have to evaluate sources and decide which are the most reliable and authoritative. For academic topics, every field has an established system of reviews and evaluations that can be found in scholarly journals associated with that field. In history, for example, the American Historical Review reviews around 1,000 books each year. The American Historical Association's Guide to Historical Literature (1995) summarizes the evaluations of 27,000 books and articles in all fields of history. Editors should seek out and take advantage of these publications to help find authoritative sources. Disagreements between the authoritative sources should be indicated in the article.

See Wikipedia:No original research and Wikipedia:Verifiability for more information.

In many fields, it is nigh-impossible to publish a critical review of anything. This would be one thing if bad books were simply not reviewed, but in practice bad books are badly reviewed, with reviews focusing excessively on positive points while ignoring negative points.

This does not, of course, mean that sources in those fields cannot be evaluated.

Check multiple sources

Because conscious and unconscious biases are not always self-evident, you shouldn't necessarily be satisfied with a single source. Find another one and cross-check. If multiple independent sources agree and they have either no strong reason to be biased, or their biases are at cross purposes, then you may have a reliable account.

However, bear in mind that we only report what reliable publications publish, although of course editors should seek to use the most authoritative sources. In accordance with Wikipedia's No original research policy, we do not add our own opinion.

Certainly nice practice, but is this, practically speaking, ever going to happen?

Issues to look out for

• Have the secondary sources used multiple independent primary sources?
• Do they have an agenda or conflict of interest, strong views, or other bias which may color their report? Remember that conflicts of interest are not always explicitly exposed and bias is not always self-evident. However, that a source has strong views is not necessarily a reason not to use it, although editors should avoid using political groups with widely acknowledged extremist views, like Stormfront.org, Al-Qaeda, or the British Socialist Workers Party. Groups like these may be used as primary sources only, i.e. as sources about themselves and their own activities or viewpoints, and even then with caution and sparingly. Extremist groups should not be used as secondary sources.
• Were they actually there? Be careful to distinguish between descriptions of events by eyewitnesses and by commentators. The former are primary sources; the latter secondary. Both can be reliable.
• Find out what other people say about your sources.
• Have the sources reported other facts reliably, including on different subjects? Cross-check with what you already know.
• Are they available to other editors to check? We provide sources for our readers, so they must be accessible in principle. If not, inclusion is probably not appropriate. Note, however, that they need not be online; availability through a library is sufficient.
• Live news footage during disasters and major incidents are primary sources that often contain highly speculative reports and commentary, some which may later turn out to be false; Such sources should be corroborated with other reliable sources, such as newspaper reports that have gone through editorial process and review.

The Stormfront/Al Qaeda/Socialist Workers Party comment is another example of the thousand-exception style of policy writing.

Independent secondary sources

Independent secondary sources:

• Have separate editorial oversight and fact-checking processes;
• Have not collaborated;
• May have taken their own look at the available primary sources and used their own judgment in evaluating them.

Even given the same primary sources, different analysts may come to different conclusions about the facts being reported. In practice, many secondary sources find and use different primary sources in the course of their research. Conscious biases, unconscious biases, and errors are not always self-evident. The best way to expose them is to cross-check with another secondary source.

This seems, ultimately, to point towards coming up with an absolute account. NPOV neither mandates nor encourages this.

Online Sources: Evaluating Reliability

Evaluate the reliability of online sources just as you would print or other more traditional sources. Neither online nor print sources deserve an automatic assumption of reliability by virtue of the medium they are printed in. All reports must be evaluated according to the processes and people that created them.

Reliability is a spectrum, and must be considered on a case by case basis. Typically peer reviewed publications are considered to be the most reliable, with established professional publications next. Government publications are often reliable, but governments vary widely in their level of reliability, and often have their own interests which will explicitly allow for withholding of information, or even out right deception of the public. Below this are sources which, while not tangible, can be providers of reliable information in some cases, for example websites associated with reliable sources of information.

• With any source, multiple independent confirmation is one good guideline to reliability, if several sources have independently checked a fact or assertion, then it is more reliable than one which is not checked.
• Sources where there are multiple steps to publication, such as fact checking and editorial oversight, are generally more credible, all other things being equal, than those which are not.
• Reliable sources also are ones which differentiate within their own information stream between assertions which are backed by observation, those that are theoretical but highly likely, and those which are speculative, conjectural and rumor.
• Reliable sources have reproducible or verifiable means of gathering information. A fact which could be checked, even if it has not been, is generally more reliable than one which cannot be checked.
• Reliable sources tend to state explicitly who their sources are. Thus quotes with attribution are more reliable than "anonymous sources," particularly when anonymous sources are speaking towards their own interests.
• In general it is preferable to cite the original source for an assertion, as well as important confirming sources. It is generally preferable to cite reliable sources over less reliable sources when given a choice.
• However, while reliability is to some extent fungible, peer reviewed publications make errors, professional publications vary widely in quality and have their own POVs and other sources have to be evaluated based on the particular assertion.
• While reputable and reliable have considerable overlap, one is not a substitute for the other.
• A source is more reliable within its area of expertise than out of its area of expertise.
• A particular source which aims to have credibility beyond a particular POV is generally regarded as more reliable than one whose audience is narrow in terms of its ideology, partisan agenda or point of view.

Note that Wikipedia itself does not currently meet the reliability guidelines; however, nothing in this guideline is meant to contravene the associated guideline: Wikipedia:Build the web. Wikilink freely.

The tone of this section is good, but it's trying to collapse all the common sense into one section, and still suffers from a bias towards academic sources. The main problem here is that, by being collapsed into so small a section, it ends up going in circles, and ultimately reads more as "Evaluating sources is hard" but doesn't really point toward an endpoint.

OS: Bulletin boards, wikis and posts to Usenet

Posts to bulletin boards, Usenet, and wikis, or messages left on blogs, should not be used as primary or secondary sources. This is in part because we have no way of knowing who has written or posted them, and in part because there is no editorial oversight or third-party fact-checking. In addition, in the case of wikis, the content of an article could change at any moment. For exceptions, see the section on self-published sources.

The same reasoning applies to trivia on sites such as IMDb or FunTrivia.com, where the degree of editorial oversight is unknown. (However, film credits on IMDb are provided directly by the Writer's Guild of America and can be considered reliable.)

This section is just nonsense. Usenet, BBs, and wikis are perfectly reliable as primary sources in lots of cases. Spoo is a featured article based almost entirely on BB posts and Usenet, and nobody with any knowledge about the topic would criticize one of them.

OS: Self-publication

A self-published source is a published source that has not been subject to any form of independent fact-checking, or where no one stands between the writer and the act of publication. It includes personal websites, and books published by vanity presses. Anyone can create a website or pay to have a book published, and then claim to be an expert in a certain field. For that reason, self-published books, personal websites, and blogs are largely not acceptable as sources.

Exceptions to this may be when a well-known, professional researcher writing within their field of expertise, or a well-known professional journalist, has produced self-published material. In some cases, these may be acceptable as sources, so long as their work has been previously published by credible, third-party publications, and they are writing under their own name or known pen-name and not anonymously.

However, editors should exercise caution for two reasons: first, if the information on the professional researcher's blog (or self-published equivalent) is really worth reporting, someone else will have done so; secondly, the information has been self-published, which means it has not been subject to any independent form of fact-checking.

In general it is preferable to wait until other sources have had time to review or comment on self-published sources.

Reports by anonymous individuals, or those without a track record of publication to judge their reliability, do not warrant citation at all, until such time as it is clear that the report has gained cachet, in which case it can be noted as a POV.

This policy does not fit sensibly with the previous section. Anyone can start a blog, making the identity verification equally problematic. This renders the two functionally equivalent. In some cases, of course, we can verify a blogger's identity. We can also, in some cases, verify a message board poster's identity. Also, the professional researcher criteria is insufficient. Consider a situation like Ronald Moore, creator of the new Battlestar Galactica. He maintains a blog on the Sci-Fi Channel's website. This is clearly a reliable source of information about the show, despite the fact that he is not a "researcher" in any useful definition.

OS: Self-publication on self

Material from self-published sources, whether published online or as a book or pamphlet, may be used as sources of information about themselves in articles about themselves, so long as there is no reasonable doubt about who wrote it, and where the material is one of the following:

• relevant to the person's notability, or, if the material is self-published by a group or organisation, relevant to the notability of that group or organisation or
• not contentious, such as basic biographical information. All information of a self-published nature should be looked at with a critical eye.

it should also be:

• not unduly self-serving or self-aggrandizing;
• about the subject only, and does not involve claims about third parties, or about events not directly related to the subject;
• subject to verification by other sources.

Self-published material should always be reported as the POV of the publisher, and not as general fact, until such time as there is independent corroboration of that material. The reputation of the self-publisher is a guide to whether the material rises to the level of notability at all.

In general, if a self-published source is reliable, then other reliable sources will cite it, until then, it should be avoided.

The above BSG example (As well as the Spoo example) fail this test too. No independent corroboration is going to be forthcoming on such examples of information being released about a media product. (How would this even be possible? No secondary source is going to base itself on anything other than the statements of the creators.) This is a BLP guideline masquerading as a larger guideline. A more sane guideline would note that when a person's own account is contradicted by other accounts, this should be noted – not to cast de facto suspicion on a person's own account.

OS:Self-published sources as secondary sources

Personal websites, blogs, and other self-published or vanity publications should not be used as secondary sources.

That is, they should not be used as sources of information about a person or topic other than the owner of the website, or author of the book. The reason personal websites are not used as secondary sources—and as primary sources only with great caution and not as a sole source if the subject is controversial—is that they are usually created by unknown individuals who have no one checking their work. They may be uninformed, misled, pushing an agenda, sloppy, relying on rumor and suspicion, or even insane; or they may be intelligent, careful people sharing their knowledge with the world. Only with independent verification by other sources not holding the same POV is it possible to determine the difference.

Visiting a stranger's personal website is often the online equivalent of reading an unattributed flyer on a lamp post, and should be treated accordingly.

The blanket ban on using self-published sources as secondary sources is in flat contradiction with the "professional researcher" clause several sections up.

The websites and publications of political parties and religious groups should be treated with caution, although neither political affiliation nor religious belief is in itself a reason not to use a source.

Widely acknowledged extremist or even terrorist organizations or individuals, whether of a political, religious, racist, or other character, should never be used as sources for Wikipedia, except as primary sources, that is to say they may be used in articles discussing the opinions of those organizations or individuals. Even then they should be used with great caution, and should be supported by other sources.

Why is extreme caution necessary to use, say, the Stormfront website as a primary source about Stormfront's beliefs? This is silly. What we're trying to prevent is allowing Stormfront to write their own biography.

Caution should be used when using company or organization websites as sources. Although the company or organization is a good source of information on itself, it has an obvious bias. The American Association of Widget Manufacturers is interested in promoting widgets, so be careful not to rely on it exclusively if other reliable sources are available, in order to maintain a neutral point of view. Exercise particular care when using such a website as a source if the company or organization is a controversial one.

The note about company websites is not a matter of reliable sources – it's a matter of NPOV. A general note that, when conclusions of primary sources are contradicted by other sources, whether primary or secondary, we do not defer wholly to the primary source would be both sufficient and appropriate.

Full-text online sources are as acceptable as offline sources if they are of similar quality and reliability. Readers may prefer online sources because they are easily accessed.

If you find a print source that is out of copyright or that is available on compatible licensing terms, add it to Wikisource and link to it there (in addition to the normal scholarly citation). Many significant out-of-copyright books have already been put online by other projects.

Many sources are under copyright and only available offline, such as most books, newspaper articles and certain journals. Other texts are only available online through restricted academic and commercial databases. However, this does not mean they cannot be cited as sources, so long as they are otherwise reliable.

[no comments on this section]

Finding good sources

Until more authors publish online, and more material is uploaded, some of the most reliable and informative sources are still available only in printed form. If you can't find good sources on the web, try a local library or bookstore. Major university libraries usually have larger collections than do municipal libraries.

Fact checking and reference-running can be time consuming. Your local public or academic library may not have the work cited by an article on its shelves. Often you can ask for a book through interlibrary loan, but this can sometimes take several weeks to do. Fortunately, new tools are now available online to make this work easier. Services such as Google Books, Amazon.com’s “search inside!” , the Internet Archive’s Million Book Project and the University of Michigan's Making of America allow you to search the full text of thousands of books. In addition, many similar subscription-based services may be available through your public, college, university or graduate school libraries.

To check on facts and citations in Wikipedia, these databases are powerful tools. You can search them the same way that you do in an internet search. Enter the author in quotation marks and the title in quotation marks. If the book is in the database already, the search engine will find it. If it isn’t, you may discover another work that discusses the book you seek. For subjects, enter as many terms as you can recall. The engines will display a list of pages that contain these terms. Often you will be able to verify the fact you are checking or discover a significant point of view not represented in the Wikipedia article.

When you use one of these services, be sure to gather all the information you can find by selecting links such as “About the Book.” You should be able to assemble a citation in exactly the same way you do with a print publication. If there is an ISBN for the book, be sure to include it. Use the ISBN to link to the book, since several of these sites display only selected materials from the books they have online.

Hint: Services such as Google Books often have poorly OCR-ed text. This is especially true for names and words with diacritics where searches often come up with nothing. A way around this is to search for common OCR mistakes.

The Wikipedia special page Book sources will enable readers to click on the ISBN number of your book citation and search a variety of library databases and retailers to find it. For example the citation:

Harvey, Andrew, Songs of Kabir, Weiser Books (January 2002), ISBN 1578632498

Clicking on the ISBN number will enable users to "find this book" at Amazon.com, public libraries, etc.

A bit of a pep talk, and an odd one at that. We should be cautious about suggesting that the ideal Wikipedia editor is going to devote an inordinate amount of time to the task and go do book-based research. It's nice, yes, but it does cut rather severely against the notion that "anyone" can edit the encyclopedia, and is an affront to the volunteer nature of the project. We should make sure this guideline can be followed by the volunteers we have – not by the ones we wish we had.

Sources in languages other than English

Because this is the English Wikipedia, for the convenience of our readers, English-language sources should be provided whenever possible, and should always be used in preference to foreign-language sources (assuming equal quality and reliability). For example, do not use a foreign-language newspaper as a source unless there is no equivalent article in an English-language newspaper. However, foreign-language sources are acceptable in terms of verifiability, subject to the same criteria as English-language sources.

Keep in mind that translations are subject to error, whether performed by a Wikipedia editor or a professional, published translator. In principle, readers should have the opportunity to verify for themselves what the original material actually said, that it was published by a credible source, and that it was translated correctly.

Therefore, when the original material is in a language other than English:

• Where sources are directly quoted, published translations are generally preferred over editors performing their own translations directly.
• Where editors use their own English translation of a non-English source as a quote in an article, there should be clear citation of the foreign-language original, so that readers can check what the original source said and the accuracy of the translation.

No comments from Phil.

BLP

Unverified material that could be construed as critical, negative or harmful in articles about living persons should be removed immediately, and should not be moved to the talk page. The same applies to sections dealing with living persons in other articles. Real people are involved, and they can be hurt by your words. We are not tabloid journalists, we are an encyclopedia.

This is not a guideline related to sourcing.

History

Historical research involves the collection of original or “primary” documents (the job of libraries and archives), the close reading of the documents, and their interpretation in terms of larger historical issues. To be verifiable, research must be based on the primary documents. In recent decades, many more primary documents (such as letters and papers of historical figures) have been made easily available in bound volumes or online. For instance, the Jefferson Papers project at Princeton begun in 1950 has just published volume 30, reaching February 1801. More recently, primary sources have been put online, such as the complete run of the London Times, the New York Times and other major newspapers. Some of these are proprietary and must be accessed through libraries; others such as “Making of America”, publishing of 19th century magazines, are open to the public.

Scholars doing research publish their results in books and journal articles. The books are usually published by university presses or by commercial houses like W.W. Norton and Greenwood which emulate the university press standards. Reputable history books and journal articles always include footnotes and bibliographies giving the sources used in great detail. Most journals contain book reviews by scholars that evaluate the quality of new books, and usually summarize some of their new ideas. The American Historical Review (all fields of history) and Journal of American History (US history) each publish 1000 or more full-length reviews a year. Many of the major journals are online, as far back as 1885, especially through JSTOR.org. A good book or article will spell out the historiographical debates that are ongoing, and alert readers to other major studies.

On many topics, there are different interpretive schools which use the same documents and facts but use different frameworks and come to different conclusions. Useful access points include: scholar.google.com and books.google.com, and (through libraries) ABC-CLIO’s two abstract services, American: History and Life (for journal articles and book reviews dealing with the US and Canada), and Historical Abstracts (for the rest of the world.) Research libraries will hold paper guides to authoritative sources. The most useful is The American Historical Association's Guide to Historical Literature, edited by Mary Beth Norton and Pamela Gerardi 2 vol (1995), which is an annotated bibliography of authoritative sources in all fields of history.

In historical pages the user is assisted by having an annotated bibliography of the best resources. Users will often have to use inter-library loan to obtain books, so a short annotation explaining the value and POV of the book may be helpful.

There are many other sources of historical information, but their authority varies. A recent trend is a proliferation of specialized encyclopedias on historical topics. These are edited by experts who commission scholars to write the articles, and then review each article for quality control. They can be considered authoritative for Wikipedia. General encyclopedias, like the Encyclopedia Britannica or Encarta, sometimes have authoritative signed articles written by specialists and including references. However, unsigned entries are written in batches by freelancers and must be used with caution.

College textbooks are updated every few years, are evaluated by many specialists, and usually try to keep abreast of the scholarship, but they are often without footnotes and usually do not spell out the historiographical debates. Textbooks at the K-12 level do not try to be authoritative and should be avoided by Wikipedia editors. Every place has guide books, which usually contain a capsule history of the area, but the great majority do not pretend to be authoritative.

On many historical topics there are memoirs and oral histories that specialists consult with caution, for they are filled with stories that people wish to remember—and usually recall without going back to the original documentation. Editors should use them with caution.

The general public mostly gets its history from novels, films, TV shows, or tour guides at various sites. These sources are full of rumor and gossip and false or exaggerated tales. They tend to present rosy-colored histories in which the well-known names are portrayed heroically. Almost always editors can find much more authoritative sources.

• See also Historiography

This section is working more towards a notion of "authoritative" sources instead of reliable ones. Authoritative sources are of interest only if we are trying to present an absolute point of view. We're not.

Sciences

Scientific journals are the best place to find primary source articles about experiments, including medical studies. Any serious scientific journal is peer-reviewed. Many articles are excluded from peer-reviewed journals because they report what is in the opinion of the editors unimportant or questionable research. In particular be careful of material in a journal that is not peer-reviewed reporting material in a different field. (See the Marty Rimm and Sokal affairs.)

The fact that a statement is published in a refereed journal does not make it true. Even a well-designed experiment or study can produce flawed results or fall victim to deliberate fraud. (See the Retracted article on neurotoxicity of ecstasy and the Schön affair.)

Honesty and the policies of neutrality and No original research demand that we present the prevailing "scientific consensus". Polling a group of experts in the field wouldn't be practical for many editors but fortunately there is an easier way. The scientific consensus can be found in recent, authoritative review articles or textbooks and some forms of monographs.

There is sometimes no single prevailing view because the available evidence does not yet point to a single answer. Because Wikipedia not only aims to be accurate, but also useful, it tries to explain the theories and empirical justification for each school of thought, with reference to published sources. Editors must not, however, create arguments themselves in favor of, or against, any particular theory or position. See Wikipedia:No original research, which is policy. Although significant-minority views are welcome in Wikipedia, the views of tiny minorities need not be reported. (See Wikipedia:Neutral Point of View.)

Make readers aware of any uncertainty or controversy. A well-referenced article will point to specific journal articles or specific theories proposed by specific researchers.

The claim about "reporting material in different fields" is problematic, due particularly to the rise of interdisceplenary journals.

Multiple studies have pointed to systemic flaws in the scientific peer review system. This becomes a serious issue when dealing with cutting-edge science, which is often difficult at best to summarize in generalizable terms. The combination of these facts poses serious problems for verifiability. (This is a case where we need to be worried about primary sources – where a non-expert could not possibly interpret them.)

The popular press generally does not cover science well. Articles in newspapers and popular magazines generally lack the context to judge experimental results. They tend to overemphasize the certainty of any result, for instance presenting a new experimental medicine as the "discovery of the cure" of a disease. Also, newspapers and magazines frequently publish articles about scientific results before those results have been peer-reviewed or reproduced by other experimenters. They also tend not to report adequately on the methodology of scientific work, or the degree of experimental error. Thus, popular newspaper and magazine sources are generally not reliable sources for science and medicine articles.

What can a popular-press article on scientific research provide? Often, the most useful thing is the name of the head researcher involved in a project, and the name of his or her institution. For instance, a newspaper article quoting Joe Smith of the Woods Hole Oceanographic Institution regarding whales' response to sonar gives you a strong suggestion of where to go to find more: look up his work on the subject. Rather than citing the newspaper article, cite his published papers.

The popular press does not cover science less well than any other academic field. This warning should not be specific to science.

One method to determine which journals are held in high esteem by scientists is to look at impact factor ratings, which track how many times a given journal is cited by articles in other publications. Be aware, however, that these impact factors are not necessarily valid for all academic fields and specialties.

In general, journals published by prominent scientific societies are of better quality than those produced by commercial publishers. The American Association for the Advancement of Science's journal Science is among the most highly regarded; the journals Nature and Cell are notable non-society publications.

Keep in mind that even a reputable journal may occasionally post a retraction of an experimental result. Articles may be selected on the grounds that they are interesting or highly promising, not merely because they seem reliable.

Unless we want to create a general list of reputable publications, the "which science journals" section doesn't actually provide useful guidance.

There are a growing number of sources on the web that publish preprints of articles and conference abstracts, the most popular of these being arXiv. Such websites exercise no editorial control over papers published there. For this reason, arXiv (or similar) preprints and conference abstracts should be considered to be self-published, as they have not been published by a third-party source, and should be treated in the same way as other self-published material. See the section above on self-published sources. Most of them are also primary sources, to be treated with the caution as described in various sections of this guideline.

Researchers may publish on arXiv for different reasons: to establish priority in a competitive field, to make available newly developed methods to the scientific community while the publication is undergoing peer-review (a specially lengthy process in mathematics), and sometimes to publish a paper that has been rejected from several journals or to bypass peer-review for publications of dubious quality. Editors should be aware that preprints in such collections, like those in the arXiv collection, may or may not be accepted by the journal for which they were written—in some cases they are written solely for the arXiv and are never submitted for publication. Similarly, material presented at a conference may not merit publication in a scientific journal.

arXiv needs to be used with extreme caution, but an article on, say, the Poincare conjecture can't really be written without it at present.

There are techniques that scientists use to prevent common errors, and to help others replicate results. Some characteristics to look for are experimental control (such as placebo controls), and double-blind methods for medical studies. Detail about the design and implementation of the experiment should be available, as well as raw data. Reliable studies don't just present conclusions.

and

Statistical information is easily and often misinterpreted by the public, by journalists, and by scientists. It should be checked and explained with the utmost care, with reference to published sources.

See Misuse of statistics, Opinion poll, and Statistical survey for common errors and abuses.

The statistics section encourages exactly the sort of judgment that the page forbids on other topics such as blogs and Usenet.

Law

First of all, remember there are several legal traditions and that laws are only valid in their own jurisdiction. The opinion of local experts is therefore preferred, in general, to that of outside commentators, due to variances across areas of jurisdiction.

When discussing legal texts, it is in general better to quote from the text, or quote from reputable jurists, than to quote from newspaper reports, although newspaper reports in good newspapers are acceptable too. The journalist who wrote the paper may not be trained as a lawyer, although s/he may have access to a wider variety of legal experts than many lawyers do, so judge the quality of the report according to how well that journalist, or that newspaper, has covered legal issues in the past.

No comments from Phil here.

Popular culture and fiction

Articles related to popular culture and fiction must be backed up by reliable sources like all other articles. However, due to the subject matter, many may not be discussed in the same academic contexts as science, law, philosophy and so on. Personal websites, wikis, and posts on bulletin boards, Usenet and blogs should still not be used as secondary sources. (See Wikipedia:Requests for arbitration/Wilkes, Wyss and Onefortyone.)

As said before, nonsense – popular culture articles on contemporary culture cannot be written without reference to blogs, Usenet, bulletin boards, etc. This is a fundamental shift in what a source is, and we need to respond to it.

References

No reputable style manuals or research guidelines are cited – only Jimbo's statements.

External links

• How to Read a Primary Source, Reading, Writing, and Researching for History: A Guide for College Students, Patrick Rael, 2004.
• How to Read a Secondary Source, Reading, Writing, and Researching for History: A Guide for College Students, Patrick Rael, 2004.

Again, a subject-specific guideline is being passed off as a general guideline. These pages on primary/secondary sources are good for history – not for the general case.

That's all I've got. Phil Sandifer 17:29, 10 October 2006 (UTC)

• That was never my intent in adding the links Phil, I simply listed them in the manner of our citation style. I thought they may prove of use. Hiding Talk 15:42, 11 October 2006 (UTC)