Wikipedia:Flagged revisions/Trial/Proposed trials

This Wikipedia page has been superseded by Wikipedia:Flagged protection and patrolled revisions and is retained primarily for historical reference.

Conditions

Discuss:Wikipedia talk:Flagged revisions/Trial/Proposed trials#Conditions

The following conditions have been suggested by editors who would like to test flagged revisions, but have concerns about how this is done.

Any test should include a sunset clause, which would restore the present mode of editing.
Before any test, the talk pages of articles proposed for inclusion in the test should be notified; unless there is consensus, or silence, about such inclusion, the page should not be included.
Any test should specify an equal number of pages of similar status which are not subject to flagged revisions, so that the two sets can be compared.
The first test should contain no more than 1000 [exact number subject to discussion] pages, and the number of articles should only be increased only after discussion of the results of the test.
The first test should not last more than 5 [exact number subject to discussion] days, and the length should only be increased after discussion of the results of the test.
Human editors should acquire the ability to sight articles automatically, subject to fixed conditions on time and edit-count, as the ability to move pages is acquired now.
Bots should not be permitted to sight pages. Their edits should always be subject to review.
Edits should be automatically sighted (perhaps unless they contain a few offensive words) in 48 [exact number subject to discussion] hours, if a human has not reviewed them.
This draft has been defended on the assumption, nowhere stated, that sighting will only be used against obvious vandalism, libel, copyright violations, and unsourced contentious edits to biographies of living persons. A successful version should state what sighting is used for, and include some provisions to prevent potential abuses.

The quantities named are suggestions, and should be edited freely.

Proposed trials

Note: Sections for trials 5, 6, 7, 11, and 12 have been moved to the talk page. Please do not reuse the numbers.

Trial 1: Featured articles/portals

In the first proposed trial, a system of flagged revisions known as 'sighted revisions' is enabled on all featured articles and featured portals. The duration of the trial period will be two months.

Under the system of 'sighted revisions', all revisions are to be sighted unless they are found to contain vandalism, copyright violations, or libel. FlaggedRevisions thereby functions as a low-level filter against obvious vandalism. Any registered editor may request 'reviewer' privileges at Wikipedia:Requests for permissions, which will be granted and revoked by administrators in a similar fashion to rollback. Rollbackers and administrators are automatically granted 'reviewer' permissions.

Any editor who believes they were unjustly refused permissions may make an appeal on the administrators' noticeboard; claims that an editor has abused their FlaggedRevisions-related permissions should be made on the same noticeboard, which may result in said permissions being removed.

Trial 2: Semi-protected articles

Under this second proposed system, all currently semi-protected articles (possibly, with some exceptions) will be unprotected and have Flagged Revisions enabled on them. The duration of the trial period will be two months.

Under the system of 'sighted revisions', all revisions are to be sighted unless they are found to contain vandalism, copyright violations, or libel. FlaggedRevisions thereby functions as a low-level filter against obvious vandalism. Any registered editor may request 'reviewer' privileges at Wikipedia:Requests for permissions, which will be granted and revoked by administrators in a similar fashion to rollback. Rollbackers and administrators are automatically granted 'reviewer' permissions.

Any editor who believes they were unjustly refused permissions may make an appeal on the administrators' noticeboard; claims that an editor has abused their FlaggedRevisions-related permissions should be made on the same noticeboard, which may result in said permissions being removed.

Trial 3: Subsection of Biographies of Living Persons

In this proposed trial, a system of flagged revisions known as 'sighted revisions' is enabled on all articles that are in Category:Living people and have names beginning with "Z" (approx 3 to 4 thousand articles). The duration of the trial period will be two months.

Under the system of 'sighted revisions', all revisions are to be sighted unless they are found to contain vandalism, copyright violations, or libel. FlaggedRevisions thereby functions as a low-level filter against obvious vandalism. Any registered editor may request 'reviewer' privileges at Wikipedia:Requests for permissions, which will be granted and revoked by administrators in a similar fashion to rollback. Rollbackers and administrators are automatically granted 'reviewer' permissions.

Any editor who believes they were unjustly refused permissions may make an appeal on the administrators' noticeboard; claims that an editor has abused their FlaggedRevisions-related permissions should be made on the same noticeboard, which may result in said permissions being removed.

Trial 4: Per-article opt-in

In this proposed trial, any article may have a notice placed on its talk page describing the new feature and asking editors of that article to come to a consensus regarding its use. If there are at least two active editors, and if all the active editors come to a consensus that it should be enabled, then it will be enabled on that article for the duration of the trial. Notices will be automatically placed on the talk pages of all featured articles, semi-protected articles, and articles in Category:Living people; they may be manually placed on any other articles. The duration of the trial period will be two months.

Under the system of 'sighted revisions', all revisions are to be sighted unless they are found to contain vandalism, copyright violations, or libel. FlaggedRevisions thereby functions as a low-level filter against obvious vandalism. Any registered editor may request 'reviewer' privileges at Wikipedia:Requests for permissions, which will be granted and revoked by administrators in a similar fashion to rollback. Rollbackers and administrators are automatically granted 'reviewer' permissions.

Any editor who believes they were unjustly refused permissions may make an appeal on the administrators' noticeboard; claims that an editor has abused their FlaggedRevisions-related permissions should be made on the same noticeboard, which may result in said permissions being removed.

Trial 8: Half of Wikipedia

Some Wikipedians have advocated turning on flagged revisions for all article pages. The following trial, to be performed only after the completion of several other trials, intends to test the desirability of turning on flagged revisions for all article pages.

The trial period will be three months. No other trials will be run at the same time as this trial. At the beginning of the trial, a bot with surveyor permissions will turn on flagged revisions for all articles beginning with the letters A-M, the numbers 0-4, and the symbols !, @, $, %, ^, &, *, (, ), and -, and turn off flagged revisions for all articles beginning with the letters N-Z, the numbers 5-9, or any other symbol. The bot will continue to run and to turn on flagged revisions for new pages beginning with A-M, 0-4, or the above list of symbols. The bot will not sight any revision of any article. At the end of the trial, the bot will turn off flagging for all articles.

This trial does not specify the conditions under which an article is sighted. It does not specify which editors are given surveyor or reviewer permissions. It does not specify how these permissions are requested or how denials are appealed. Before this trial is performed, all of these must be determined by community consensus.

This trial also does not specify how its success is to be measured. Before this trial is performed, a method for evaluating the success of the trial must be determined by community consensus. Ozob (talk) 23:03, 3 January 2009 (UTC)[reply]

Trial 9: Logistical test

This trial is meant to test the logistics of flagged revisions. It is intended to precede any test of the effect of flagged revisions; it only tests the logistics.

The trial is divided into two phases. The first phase will last two weeks. During this time, no articles will have flagged revisions turned on. A banner will be placed using MediaWiki:Sitenotice to inform all users of the forthcoming second phase. Requests for surveyor and reviewer status will be taken using a method determined by community consensus. Requests to revoke surveyor or reviewer status will also be taken (though they are unlikely, since no articles should have flagged revisions turned on). Users with questions about the process will be directed to Wikipedia Talk:Flagged revisions or another page set up especially for that purpose.

Immediately after the first phase ends, the second phase begins. The second phase will last one week. During this week, the previous processes continue to run. In addition, a banner will be placed using MediaWiki:Sitenotice directing users to a description of the present test, and a very small number of articles will have flagged revisions turned on. (The exact number and selection of the articles will be determined by community consensus.) Flagged revisions are intended to be used "normally" for this week, whatever the community decides "normally" should mean.

At the end of the test, the banner is removed, flagged revisions are turned off on all articles, and surveyor and reviewer status is revoked for all users (except those who receive it because they have another status such as admin).

The test succeeds if the above processes run smoothly. Requests for surveyor or reviewer status should take a reasonable amount of time, have a reasonable backlog, and are regularly able to reach consensus. Requests to revoke that status should take a reasonable amount of time, have a reasonable backlog, and are regularly able to reach consensus. Flagging of articles should take a reasonable amount of time, have a reasonable backlog, and should not generate excessive confusion. The success of the test will be determined by a discussion on a designated page.

At least two weeks must separate the end of this test from the start of the next test. During this time, the community may implement procedural changes to make flagged revisions run more smoothly. Ozob (talk) 23:02, 4 January 2009 (UTC)[reply]

Trial 10: Flagged protection

Choose 300 semi-protected articles, and notify the protecting admins. If they do not object, convert the semi-protection to FR for the length of the semi-protection, but no longer than two months. All admins will have surveyor status, all autoconfirmed users will have reviewer status, so this may be independent of Trial 9 (it depends on whether this can be implemented by switching the definitions of admin and autoconfirmed in the software). Any admin may unprotect, as they do now.

The control group is another list of 300 semiprotected articles; again, notify the protecting admins - there may be special circumstances which would make them unrepresentative.

This succeeds if the 300 articles, or some definable portion of them (like all but BLP) are in better condition under flagged protection than the control group. We should consider how much sighting has actually been done, but for this test, if sighting is too much trouble, then the test group will in effect be under semi-protection, and the conclusion will be: no significant benefit. Septentrionalis PMAnderson 16:49, 5 January 2009 (UTC)[reply]

Note: this differs from Trial 2 in two respects: it only FRs a sample, and all autoconfirmed users have reviewer status, thus obviating the need for appeals. Since autoconfirmed editors can edit semi-protected articles now, this is no change in policy. Septentrionalis PMAnderson 16:54, 5 January 2009 (UTC)[reply]

Trial 13: Three month trial of all BLPs + flagged protection

Flagged Revisions is enabled on all articles about living persons.(Perhaps half in the trial)
Administrators can enable it on other articles as an alternative to semi-protection.
All rollbackers are allowed to sight revisions.
Users are automatically allowed to sight after 3 months and 100 edits. It can be given or taken away by an administrator. 3 months is the main requirement, 100 edits is to prevent vandals from creating lots of sleeper accounts. The numbers are open for discussion.
After two months a report describing the result of the trial and how to continue will be compiled by consensus editing. At the end of the three month trial the decision in the report is to be implemented. If there is no consensus report, then Flagged Revisions is disabled until consensus is reached.
Less important points:
If an edit has not been sighted nor reverted for one week, then the article is temporarily taken off Flagged Revisions, showing the draft version to anonymous visitors until a revision is sighted.
If the sighting backlog gets long, then high profile BLPs (George Bush etc.) can be taken off the system.
For anonymous users the "Edit this page" link at the top is replaced with "View draft" when there is an unsighted revision.
Sighters are expected to check for obvious vandalism, obvious copyright violations and contentious material about living persons that violates WP:BLP. Sighters are not responsible for checking if the edit actually improved the article, though it is of course appreciated if they do.

Please comment on this proposed trial on the talk page. Feel free to make small changes above or add extra points below. --Apoc2400 (talk) 14:26, 26 January 2009 (UTC)[reply]

Trial 14: Three month trial of selection of BLPs + flagged protection

Largely the same protection mechanisms as WP:Flagged Protection enabled to replace semi-protection and full protection
Flagged revisions enabled on 1000 validated BLP articles with a similarly cleaned-up control group
Proposal page gives more details on how this might specifically work

Proposal is a bit more involved than a summary will allow - please read, comment, and edit as necessary: nothing finxed in stone at the moment.

Trial 15: Cite it or lose it on BLP subrange

Cited version is initially set as default view for articles in Category:Living people with names in T-Z range.
Only some subset of these articles will be actually gain a sighted version, and thus be affected. This will happen as editors with sighting permission explicitly set sighted on a revision of one of these articles.
Citing permission is automatically granted to users with 100 or more edits and registered for 3 months. It may be revoked by admins.
Setting Cited for a revision means that all changes since the last sighted version (the whole article for first cited version) have been checked and found free of obvious vandalism or libel. (It happens automatically for those with citing permission if the previous version is cited; this assumes such an editor will not introduce vandalism or libel.) Editors are encouraged but not required to do more thorough review.
If an edit remains unreviewed for one week, the default view for that article is reset to "current version", thus making the unreviewed edits generally visible. (Currently the mechanism does not support an automated way of doing this reset, so bots or volunteers will be needed to do this housekeeping.)
For articles (in the trial range) which have been reset to "current", an editor with sighting permission may sight the most recent version and then request that the article have the "sighted is default view" attribute restored. Volunteers with the power to do so will try to get this done within a day.
The trial will run for three months.

Trial 16: BLP Protection, User Vandalism Protection

Set up a bot for name recognition and have that bot scan all the new pages of the day.
Pages with names should be limited for edit, by users only, and that limitation should not be removed.
If it is discovered that a user is vandalizing the page anyway, and this is irrefutable, then the user should be banned.
If any user creates an account from the IP of the banned user, then another bot should inform the new user that their account is being subjected to increased scrutiny.
At this point, all edits that that user makes, should then be reviewed within a short (couple of hours) time frame.

SCmurky (talk) 23:33, 5 February 2009 (UTC)[reply]

Trial 17: Variant of flag protection

Discuss this trial

For a period of three months, it is proposed to implement this flag protection version, the scope is limited by the semi-protection and full-protection policy. The implementation would be progressive, by replacing existing protections (temporary or indefinite) with flag protections if the protecting admins agrees or if there is consensus to do so, and try to use flag protection for new protections. This can be limited to 1000 articles for the first month, 2000 for the second one and 3000 for the third one. Autoconfirmed users are automatically reviewed, meaning that when they edit a reviewed page, the new revision is automatically reviewed. A group of reviewers, granted manually and automatically (requirements to be determined), is created to review contributions by anonymous and new users. The proposed reviewing guidelines are located here. Autoreview may be deactivated for non-reviewers on a certain page to handle persistent sockpuppetry. Full flag protection is enabled for pages meeting the full protection requirements (typically, disputes). Administrators, and possibly users in a group 'moderator', as admins would likely be overburden, can validate revisions when consensual or non-controversial.

Other trials can be run in parallel.

Trial 18: Shadow flagging

Implement one of the other suggested trials (I am pushing for #Trial 13: Three month trial of all BLPs + flagged protection), but as an initial trial, show the latest revision (sighted or not) to readers by default. With this "shadow flagging" we can try flagged revisions in full scale, but with no negative effects if it fails. If flagging is done in a timely and good manner, then we can switch to displaying only flagged revisions by default. Even during this trial it would help make sure no edit falls between our checks. --Apoc2400 (talk) 10:08, 5 March 2009 (UTC)[reply]

>>Discuss

Possible metrics to measure success of trials

Discuss: Wikipedia talk:Flagged revisions/Trial/Proposed trials#Possible metrics to measure success of trials

Median number of reverted vandal edits on articles with flagged revisions turned on. (An edit may be judged vandalism if it is undone by rollback or an anti-vandalism tool such as Huggle.)
Median number of reverted vandal edits appearing in a flagged revision of an article with flagged revisions turned on.
Median number of reverted vandal edits on articles with flagged revisions turned off.
- The same figures for articles averaging less than one edit a week, between one edit a week and one edit a day, more than one edit a day, and more than five edits a day.
- The same figures for BLPs.
- The same figures measured daily instead of over the whole trial period.
Median time to reversion for vandalism of articles with flagged revisions turned on.
Median time to reversion for vandalism in flagged versions of articles with flagged revisions turned on.
Median time to reversion for vandalism of articles with flagged revisions turned off.
Maximum time to reversion for vandalism of articles with flagged revisions turned on.
Maximum time to reversion for vandalism in flagged versions of articles with flagged revisions turned on.
Maximum time to reversion for vandalism of articles with flagged revisions turned off.
Median and Mean number of page views of live vandalized articles with FR on. (as percentage of total page views)
Median and Mean number of page views of live vandalized articles with FR off. (as percentage of total page views)
Number of protected articles (at the end of the trial period) with flagged revisions turned on.
Number of protected articles with flagged revisions turned off.
Number of semi-protected articles with flagged revisions turned on.
Number of semi-protected articles with flagged revisions turned off.
Number of oversights or office actions performed on articles with flagged revisions turned on.
Number of oversights or office actions performed on a flagged version of an article with flagged revisions turned on.
Number of oversights or office actions performed on articles with flagged revisions turned off.
Percentage of IPs and users blocked due to vandalism who vandalized only articles with flagged revisions turned off.
Median number of edits to articles with flagged revisions turned on.
Median number of edits to articles with flagged revisions turned off.
Median number of IP edits to articles with flagged revisions turned on.
Median number of IP edits to articles with flagged revisions turned off.
Median percentage of IP edits to articles with flagged revisions turned on.
Median percentage of IP edits to articles with flagged revisions turned off.
Median number of IP users editing articles with flagged revisions turned on.
Median number of IP users editing articles with flagged revisions turned off.
Median number of logged-in users editing articles with flagged revisions turned on.
Median number of logged-in users editing articles with flagged revisions turned off.
Median percentage of IP users editing articles with flagged revisions turned on.
Median percentage of IP users editing articles with flagged revisions turned off.
Median number of hours until an edit is sighted.
Median number of hours until an edit by an IP user is sighted.
Median number of hours until an edit by a logged-in user without reviewer permissions is sighted.
Maximum number of hours until an edit is sighted.
Maximum number of hours until an edit by an IP user is sighted.
Maximum number of hours until an edit by a logged-in user without reviewer permissions is sighted.
Percentage of edits sighted within five minutes, fifteen minutes, one hour, six hours, twenty-four hours, seventy-two hours, one week, two weeks, and four weeks.
Percentage of edits by IP users sighted within five minutes, fifteen minutes, one hour, six hours, twenty-four hours, seventy-two hours, one week, two weeks, and four weeks.
Percentage of edits by logged-in users without reviewer permissions sighted within five minutes, fifteen minutes, one hour, six hours, twenty-four hours, seventy-two hours, one week, two weeks, and four weeks.
Daily median number of articles receiving at least one edit requiring sighting.
Daily median number of outstanding requests for granting reviewer status.
Daily median number of outstanding requests for granting surveyor status.
Daily median number of outstanding requests for revoking reviewer status.
Daily median number of outstanding requests for revoking surveyor status.

Polls

A poll was conducted at Wikipedia:Protecting BLP articles feeler survey/Implement Flagged Revisions for all BLPs approximately December 2008 to February 2009 which closed with 60% support with 348 participants.
A poll was conducted at Wikipedia:Protecting BLP articles feeler survey/Implement Flagged Revisions for all articles (FR-all) / content pages approximately December 2008 to February 2009 which closed with 75% opposition out of 250 participants
A poll was conducted at Wikipedia talk:Flagged protection and patrolled revisions/Poll approximately March 17–31, 2009, for the proposed flagged revisions trial Flagged protection and patrolled revisions.

Discussions

Discussions have been moved to the talk page. Please use the talk page for further discussion to avoid clogging this page.