Wikipedia:Wikipedia Signpost/2012-09-03/Op-ed

Op-ed

Dispute resolution – where we're at, what we're doing well, and what needs fixing

Share this

Steven Zhang is a community fellow with the Wikimedia Foundation researching dispute resolution.

The views expressed in this op-ed are those of the author only. Readers are invited to respond or offer critical commentary in the comments section, while those wishing to author their own op-ed may use the Signpost's opinion desk.

Since May 2012 I've been a Wikimedia Foundation community fellow with the task of researching and improving dispute resolution on English Wikipedia. Surveying members of the community has revealed much about their thoughts on and experiences with dispute resolution. I've analysed processes to determine their use and effectiveness, and have presented ideas that I hope will improve the future of dispute resolution.

How far have we come in 11 years?

Dispute resolution has existed on Wikipedia from day one. Until late 2003, Jimmy Wales acted as the arbiter for all major disputes. Following the founding of the Mediation and Arbitration Committees, Wales delegated the mandate to resolve these disputes to those bodies.

A number of informal, community-created processes were developed over time to resolve disputes, including third opinion, requests for comment, informal mediation and a variety of noticeboards targeted at issues surrounding biographies of living persons, the use of reliable sources, the neutrality of content, and the presence of original research or fringe theories in articles. Most recently, a noticeboard was created to address a variety of disputes, the dispute resolution noticeboard.

Most of my four years on Wikipedia has involved the development and management of dispute resolution processes. While I've made some progress, it became increasingly clear to me that to make the improvements to these processes that such a large, complex, and often fractious project has needed, more information was required.

Where we are now

I presented some of the findings of my research in a presentation at Wikimania 2012, in Washington DC. From the survey I conducted in April, there were four key findings I learned from editors who had participated in dispute resolution:

The main problems given for dispute resolution are its complexity, its inaccessibility, and the number of understaffed resolution processes available. Respondents want stricter action taken against problematic editors, and a simplified, more accessible process where closure can be brought to a dispute quickly
94% of respondents have requested assistance from a dispute resolution forum at some point, and are generally unhappy with their experiences (only one in five were satisfied); 62% have participated in dispute resolution in the last year.
Opinions of dispute resolution are overall negative – while respondents rate arbitration as the best dispute resolution forum (one in three respondents rate it as good or very good), Wikiquette assistance is regarded as the worst (only 1 in 12 rate it as satisfactory).
Dispute resolution volunteers do so because they feel the process is critical to Wikipedia's functioning, like helping people, or are motivated by the fact they've already received assistance as parties in the forum. Some respondents haven’t volunteered to help run the forum due to the unpleasantness of disputes, the prolonged nature of dispute resolution, poor past experiences or a lack of knowledge of how disputes can be resolved.

In summary, while some aspects of dispute resolution are viewed positively, there are many areas for improvement. The participants graded both their personal experiences and their perception of the effectiveness of the processes poorly. Experienced forum volunteers and co-operative fellow editors contributed to a positive outcome, but the lack of guidance and support from forum volunteers, and uncooperative fellow editors, made the experiences unpleasant. 70% had volunteered with dispute resolution at some point; however, only 40% did so in the month before the survey. This reluctance appears to be due to the complexity of dispute resolution or a lack of understanding on how to resolve disputes effectively. Particular issues are the complexity of requesting dispute resolution, with a lack of uniform handling, along with the time-consuming nature of the processes. The lack of forum volunteers to run the forum is also a problem.

To follow this up, I undertook an analysis of a few dispute resolution forums in May (full results are at this page). Among aspects I examined were forum volunteer counts, response times by volunteers, the time a thread was open, and success rates. Response times ranged from 5 to 24 hours, thread times from 2 to 28 days, and resolution rates from 0 to 100%.

Since the dispute resolution noticeboard was the most used forum in May, I focused my efforts there during August. We made some changes to the setup of the noticeboard – creating a robot to help manage cases, a template to keep track of disputes and a form (see right) to make filing disputes easier. We also set some goals: decrease the first response time by 40% to 10 hours or less, increase the success rate by 22 points, to 70%, increase the number of active volunteers by at least 17% (to 30), and decrease the resolution timeframe by at least 19% (to seven days).

The results for August showed some encouraging stats – a reduction of 67% to first response times, 60% reduction in discussion times, 25% reduction in thread size, an average of 2.85 volunteers to a thread (up from 1.5), and a success rate of over 64% – however the number of volunteers decreased by 20%. This shows that while the forum volunteers handled disputes in a quicker timeframe compared to May, it was from a small group of volunteers – so essentially they had just worked harder this month. This emphasises the need for more volunteers – if the existing volunteers burnout, the processes will suffer.

How did we get here?

It's been somewhat clear to me for some time what the problems with dispute resolution are. From the perspective of a participant, they need to wade through a bunch of dispute resolution forums to find the correct one, and then find the process unstructured, inefficient, confusing and lacking support from experienced volunteers. From the perspective of a volunteer, discussions are too long to read, consume too much of their time, and result in little recognition for the amount of work involved. There is also a lack of support and guidance for new volunteers – they essentially need to jump into it from the deep end. The results of the survey suggest that the community's views mostly resonate with my prior understanding.

There are other things to consider. Due to a lack of structure in some dispute resolution forums, discussions can easily get out of hand, and become extremely long and unproductive. Discussion often is retrospective as opposed to prospective, and thus does not focus on resolving the dispute. Every day, our volunteers are learning how to tackle these problems, but participants need to focus on the issues at hand and work together to find a way to resolve them.

Where we need to go, and how we can get there

Changes need to be made to make dispute resolution more effective and efficient. The alphabet soup of noticeboards needs some sort of universal request form like the one demonstrated above to help foster structured, productive discussions, while making them clearer and easier to use for volunteers and participants. We're on the right path with the closure of the mediation cabal earlier this year, and we continue to discuss changes to other forums – but we also need to make participation less ambiguous and time-consuming, and make it easier to actually get involved in resolving disputes. Volunteers are the lifeblood of the system, without whom it would not function.

But we can't make these changes alone. We need your input. We want you to share with us your experiences with dispute resolution, and how you think it can be improved. We'd also love to hear advice you have for volunteers, whether they're working within a DR process or if they're just editors on a talk page trying to cool down a discussion. Also, if you're interested in volunteering, please drop by the dispute resolution noticeboard or leave us comments here and we will help you get started.

Over the next month, I'll also be soliciting input from the community through an RFC to work towards further streamlining dispute resolution. I envision a dispute resolution system that is efficient and effective, and which has an abundance of volunteers. I hope you will join me in my efforts.

See also

In this issue

3 September 2012

News and notes

Technology report

Discuss this story

These comments are automatically transcluded from this article's talk page. To follow comments, add the page to your watchlist. If your comment has not appeared here, you can try purging the cache.

Hi, peanut gallery here: Unless there is a normalised version of that graph, it will be not very useful, as the absolute values cannot be compared between categories. Also, its not fair to make comparisons akin to: "1 in 3 people thought apple was aweome, but 2 in 3 people thought orange was OK." 46.115.17.33 (talk) 17:14, 3 September 2012 (UTC)[reply]

Fixed - thanks. Szhang (WMF) (talk) 18:17, 3 September 2012 (UTC)[reply]

On my monitor the words on the image(when zoomed in) are not readable without a magnifying glass. Regards, Sun Creator^(talk) 12:43, 4 September 2012 (UTC)[reply]

Here ya go. Rich Farmbrough, 20:30, 4 September 2012 (UTC).[reply]

I've uploaded a higher resolution graph - I hope this helps. Steven Zhang ^{Help resolve disputes!} 05:22, 5 September 2012 (UTC)[reply]

I can't agree anymore that the biggest problem with resolving disputes is that I have no idea where to go. If I had a dispute I would've clicked around on blue links for "are you in the right place", if not here's five other places you might want to check out. Ugh. Dan653 (talk) 02:49, 8 September 2012 (UTC)[reply]

Well, yes, but we need to think from the perspective of someone that's not too familiar with the system. Steven Zhang ^{Help resolve disputes!} 00:57, 9 September 2012 (UTC)[reply]

I just had a great outcome that solved all my problems, so it works at least some of the time. MathewTownsend (talk) 00:37, 17 September 2012 (UTC)[reply]

Those who "win" a dispute tend to rate the resolution process more favorably than those who lose. It is extremely difficult for most people to separate their opinion of the quality of a process from their opinion of the result.—Finell 21:10, 7 November 2012 (UTC)[reply]

Keep up with The Signpost on Twitter, Facebook or Mastodon.

Home

About