Wikipedia talk:Wikipedia Signpost/2016-03-02/Recent research

← Back to Recent research

Discuss this story

Test of 300k citations: how verifiable is "verifiable" in practice?

• Regarding Test of 300k citations: how verifiable is "verifiable" in practice?: Just a few thoughts while reading: Hope no one misextrapolates to ideas like "we shouldn't cite any source that isn't free (costless)". Most high-quality books post-1922 and many (probably most) high-quality journals are not free (costless). Regarding ISBNs, in my experience, most books from before circa 1970 (when ISBNs began) do not have any ISBN retroactively assigned. If a new print run or edition occurred, that's the only way an associated ISBN will exist. As for LCCN and OCLC, those are often available, at least for English-language books. Regarding DOIs, I know from experience that encountering journal articles that don't have a DOI is a common occurrence, especially with articles from before circa 2005. My point with all this is that the goals are laudable (use identifiers when possible, cite free sources when possible), but "when possible" is the key phrase, and it isn't always possible. Quercus solaris (talk) 02:45, 8 March 2016 (UTC)
• Quercus solaris' comment is absolutely spot on. Verifiability is an important policy but it is a means to an end. The Wikimedia Foundation mantra is "Our mission is to provide free access to the sum of all human knowledge." - taking human knowledge that would otherwise have to be paid for and making it freely available is an important part of that. If WP:V, or indeed any of our policies, becomes detrimental to achieving that mission then it is the policy (or its implementation), not the mission statement, that needs to be reconsidered. WaggersTALK 10:30, 8 March 2016 (UTC)
• I don't agree that a paywalled journal article is practically unverifiable. Journals are available through libraries, which can also obtain books on inter-library loan. I am not sure how often readers (as opposed to reviewers) are interested in the sources. Whereas the whole idea is that we are making hard-to-find knowledge widely available. I have spent a lot of time tracking down those hard-to-find sources. Hawkeye7 (talk) 10:32, 8 March 2016 (UTC)
• Actually they are discussing the "accessibility of verification". In their text they originally use the term "practical accessibility" but then jump to "practical verifiability". I think this is a mistake. Likewise their scoring system is unsound, in that accessibility is as much a feature of the querent as of the Wikipedia page, i.e. their research would be much more useful if we could input our own weighting into their scale. e.g. I live in London and free access to the British Library. Lack of an ISBN number in a pre-1970 book is not a barrier to access, although lack of disposable free time could be. This would then show that accessibility is not evenly distributed through society, and should also include access to electricity and access to the internet. This is not to say their research is useless, far from it, but it needs to be put in a social context and a more robust methodological framework. i.e. Wikipedia constitutes an apparatus (see Karen Barad) with which we can examine accessibility and if we then place other factors (power, connectivity, disposable free time, data costs, disposable money) outside and around their methodology, rather than assuming an unexamined concept of the querent embodying unstated a priori assumptions. Leutha (talk) 11:12, 8 March 2016 (UTC)
  • Those are very good points, Leutha. "Anyone" has generally been understood to include only "anyone with the time, energy, and interest to actually make a serious attempt", not just "anyone who can click a link". WhatamIdoing (talk) 04:52, 21 March 2016 (UTC)
• ISBN numbers are nothing more or less than barcodes for booksellers. At best, the inclusion of such trivia is needless duplication of the actual information needed to locate a book (author, title, publisher, publication date). At worst it is publisher spam that clogs footnotes, making essential information less easy to read and internalize. It is ridiculous to posit that ISBN numbers are in any way a metric of verifiability. Carrite (talk) 13:31, 8 March 2016 (UTC)
  • While presence or absence of ISBNs is a weak proxy for verifiability, they are still useful for bibliographic reasons. All the best: Rich Farmbrough, 14:34, 10 March 2016 (UTC).
    
• Agree with all these: "verifiable" was never intended to mean "verifiable by anyone", or we would would be restricted to web refs (and google books previews are often only available in some countries and not others). Especially in places like the Medical wikiproject, or individual talk-pages, you often see requests from those without library or journal access to supply or check things, which if put in the right place are usually successful. There's a central page for such requests somewhere. Johnbod (talk) 14:23, 9 March 2016 (UTC)
• As the reviewer, I'm very happy to see this great discussion (we have also notified the paper's authors of it, as we routinely do with all reviewed publications). I agree that WP:V is often interpreted as being agnostic about what the authors call practical and technical verifiability. However (and that's why the review said that they take the policy "literally"), they correctly refer to its first sentence, which in its current version reads:

"In Wikipedia, verifiability means that anyone using the encyclopedia can check that the information comes from a reliable source."

That directly contradicts Johnbod's comment above (although of course it does not say that doing so has to be equally easy for everyone). Also, in defense of the authors with regard to Leutha's valid point, they do acknowledge when talking about paywalled papers that accessibility depends on the querent ("someone without the additional means").

I agree with Waggers that verifiability is a means to an end: maintaining or improving the quality of the information on Wikipedia. But IMHO that is an argument for taking concerns about practical verifiability more seriously, based on my own experience as volunteer editor who since many years has spent a lot of his editing time on upholding said quality by vetting edits, often by looking up what the cited source says. It is frequently overlooked that the accuracy of information in Wikipedia is not solely a function of the accuracy of the source that was cited initially, but also depends on how effectively this information is subsequently being protected from being adulterated intentionally or unintentionally (or from being mis-cited in the first place - many hoaxers have had success by citing sources that sounded highly reliable but were very hard to access). In that respect, sources that are paywalled or otherwise difficult to access actually do lead to inferior information quality in the long run, compared to open access sources containing the same information.

Regards, Tbayer (WMF) (talk) 02:54, 10 March 2016 (UTC)

See "The Resource Exchange is a WikiProject dedicated to organizing and sharing the vast resources available to Wikipedians, to aid in verification."

The interpretation of "anyone using the encyclopedia can check" has always been pragmatic. It is perfectly admissible, for example, to cite text on a plaque that is displayed in public. That does not mean that it is practical for anyone to go there and read it, just as documents in Kew or the British Library may not be digitally available. But it is certainly possible, in a hypothetical sense. What is not verifiable is "personal knowledge", "personal communication", "what I saw through my microscope", "what a friend told me" or "what I read through magic glasses". In other media these are all perfectly good sources. Not here. That is what the verifiability doctrine is about.

All the best: Rich Farmbrough, 14:42, 10 March 2016 (UTC).

It is also usually interpreted to mean publicly available, generally meaning published, or at least in a public library archive, rather than privately held as papers, private polling data, commercial documents and records, unreleased government papers, unpublished research etc. Johnbod (talk) 15:55, 10 March 2016 (UTC)

Tilman, could you pass a link to Wikipedia:Reliable sources/Cost to the authors? This supplement to the policy explains in a fairly direct way exactly how limited that "anyone" is. WhatamIdoing (talk) 04:52, 21 March 2016 (UTC)

WhatamIdoing, we already pointed them to this review and the talk page (as we routinely do); you can find the email address of the corresponding author on the first page in case you have further input for them.

That said, I'm not super certain how important it is to wikilawyer with them about the exact interpretation of the policy - as already indicated in the headline and first sentence of the review, I think it's more useful take their rather verbatim interpretation as a starting point that leads to some empirical results that are interesting and relevant in their own right. Regards, Tbayer (WMF) (talk) 21:01, 27 March 2016 (UTC)

Speaking again as an editor who frequently attempts to check cited sources, the Resource Exchange is an awesome project and certainly useful, but too cumbersome in many situations. The Wikipedia Library gives more direct access, although its coverage is not universal either. Regards, Tbayer (WMF) (talk) 21:01, 27 March 2016 (UTC)

• See the discussion on "effective use" in Community informatics. Also see this talk by Michael Gurstein (from 18:50). What he says about Open Data also applies to Open Access Leutha (talk) 11:29, 21 March 2016 (UTC)
• It troubles me a bit when people seem to have a simplistic idea that *all* information can/should cost nothing. It amounts to a claim that *all* intellectual property is theft (echoing "Property is theft!"). But realistically, consider people who spend months of time, and travel expenses, researching a nonfiction book, such as a history or biography. How does an enterprise like that get paid for if that author can't get a book advance from a publishing company? An advance that can only be paid for by future sales revenue of the book? Well maybe that author could pay out of his own pocket for his research, people might reply. Yes, maybe; maybe. But maybe it is dampening/constraining to the output/production of good new information if we insist that it *all* must cost nothing. Please understand that in general I am a fan of open access journals and affordable books. But I am also realistic about the downsides of a situation where no one can get paid to research things, write explanations of things, edit such writing, and so on. Quercus solaris (talk) 00:26, 12 March 2016 (UTC)

I may need to reread the paper, but I don't recall the authors advocating for that simplistic idea. (And the whole debate about open access is more nuanced than than for sure, plus your comparison with that anarchist slogan seems rely on a problematic equating of physical and intellectual property.) It's certainly possible to acknowledge that e.g. scientific researchers need to get paid while at the same time not denying that citing paywalled papers on Wikipedia can (even if they may sometimes be of higher quality than freely available alternatives) also have detrimental effects on Wikipedia's longterm information quality, by making the volunteer work of checking the accuracy of edits much harder. Regards, Tbayer (WMF) (talk) 21:01, 27 March 2016 (UTC)

• The paper has "ISBN numbers can be checked numerically for validity using check-digit algorithms for either their 10 or 13 digit versions [23]. ISBNs found with Wikipedia citations in the ‘book’ reference type specified in the Wikipedia markup were tested according to these algorithms. Out of 37,269 book citations, 29,736 book citations (79.8%) had valid ISBNs, while 3,145 (8.4%) of book citations had invalid ISBNs ..."
  I have checked thousands of ISBNs, and in my experience the fraction with invalid check digits is a lot less than 8%. The paper used the WP dump of 7 July 2014, and refers to the article on Glycerol as having an invalid ISBN. ("... , the greatest gain in article rank was a 3,318 spot jump by “Glycerol” from rank 3,891 to rank 573. This article’s only ISBN was invalid ...") The ISBN was added here, and seems to have been unchanged at 7/7/14, and now, as ISBN 3527306730. It is valid. Am I missing something? Mr Stephen (talk) 22:19, 27 March 2016 (UTC)
  • Mr Stephen, the ISBN is correct, and the entry appears in WorldCat[1]. Looking at the paper, it appears that their idea of a "valid ISBN" is any ISBN that has a valid checksum digit ("ISBN numbers can be checked numerically for validity using check-digit algorithms for either their 10 or 13 digit versions"), and that they used http://www.hahnlibrary.net/libraries/isbncalc.html to validate the checksums. This one is correct according to the resource they used to test checksums (and others). Perhaps they had a problem with their script? WhatamIdoing (talk) 00:04, 28 March 2016 (UTC)

PS: the study has just been featured in The Atlantic, where the authors also propose some sort of browser plugin that displays a rating of each citation's practical verifiability. Regards, Tbayer (WMF) (talk) 19:44, 22 April 2016 (UTC)

Paid Labour

• Regarding paid labor, any dialog on this issue should incorporate Hexatekin's great article, "Labor and the New Encylopedia," and her discussion of free digital labor, especially as it relates to Wikipedia. I understand that Wikipedia will never pay its editors but I think there should be less mystery and enthralled devotion to the concept of donating your time and efforts for free -- especially given the complicated issues that Wikimedia faces going forward. -- BrillLyle (talk) 03:04, 8 March 2016 (UTC)

@BrillLyle: @Hexatekin: Indeed, thanks for the link, which would have made a nice addition to the review! There is also an interesting recent blog post (in French) by Alexander Doria which expresses skepticism at the notion that Wikipedia editing tasks (specifically, RC patrol) can be as understood as "digital labor" in the same sense as Amazon Mechanical Turk tasks: https://scoms.hypotheses.org/625

Regards, Tbayer (WMF) (talk) 21:10, 27 March 2016 (UTC)

@Tbayer (WMF): Thank you for sharing this article. I wish I spoke French but I think I get the general idea. Maybe the terminology of "digital labor" has different definitions, iterations, etc. I wonder how best to describe Wikipedia editing. It sure feels like free labor to me! Thanks again for the link and the thoughts... -- Erika aka BrillLyle (talk) 23:45, 27 March 2016 (UTC)

The attention economy of Wikipedia articles on news topics

• Chart (b) looks like a pregnancy scan. I can see a head near the top!  — Amakuru (talk) 09:54, 8 March 2016 (UTC)

And the other bump? Collect (talk) 14:57, 10 March 2016 (UTC)

Life Expectancy

• It appears that most academics do not achieve "notability" until work done well after the person is in their 30s, while most athletes who are not famous by their 30s never achieve fame, and few artists achieve fame (other than Grandma Moses) after their 30s. Thus, one would expect the results reported without even looking at Wikipedia :(. In short, that study appears to verge on the "Captain Obvious" level. What they ought have done was look at people who reached at least (say) the age of 50, and determine life expectancy of groups from that point. Famous academics who died before the age of 30 is close to a null set, as far as I can tell. Collect (talk) 14:57, 10 March 2016 (UTC)

  You mean somebody like Harry K. Daghlian, Jr., Evariste Galois or Henry Mosely? Hawkeye7 (talk) 23:34, 10 March 2016 (UTC)

  By numbers, most famous academics have been older than 30 when they achieve their fame -- that you can find exceptions is wertlos - I did not say "all." The average NFL player in 2013 was under 26 years old - with the average for the oldest team was under 28.USA Today AIP stats have the average age of doctoral recipients in Physics in the US being over 30 in 2011. Average age at which a Nobel laureate is given: 59.NobelPrize.org ] So yes - the average athlete becomes famous at a much younger age than "academics" become famous. Collect (talk) 00:41, 11 March 2016 (UTC)