Wikipedia talk:RfA Review

Archive Pre-reflect Reflect

Recommend

I know we want to add a section offering background on adminship - sort of a where we are, where we've been section - but what other prose items do we need to add? I'm reading through, and wondering if we can kill two birds with one survey by drawing up conclusions and using those conclusions as the basis for the Recommend phase survey. Thoughts? UltraExactZZ ^Claims ~ Evidence 16:19, 25 August 2008 (UTC)

That's exactly the plan. We should be able to detail conclusions from the current process, and use those as the basis for constructing recommendations. On that note though, what do you think the conclusions are? Also, I'd like to do some images for the perceptions/attributes element, in order to illustrate these topics. Hope this helps, Gazimoff 12:35, 26 August 2008 (UTC)

We've got a graph for each, listing top responses. Are their any contradictory pairs you'd like to compare, or do you have something else in mind for those images? UltraExactZZ ^Claims ~ Evidence 13:02, 27 August 2008 (UTC)

<- I think we will want to branch out in two directions. The Reflect Report should end with statements such as this:

• "There is evidence that editors see RfA as a negative process, with issues such as badgering of opposes, personal attacks, and a general lack of civility being prominent concerns."

For the recommendation questionnaire, We'd frame the question this way:

• "There is evidence that editors see RfA as a negative process, with issues such as badgering of opposes, personal attacks, and a general lack of civility being prominent concerns. How would you recommend changing the RfA process to address these concerns?"

We'll either get concrete so-fix-it style responses, with policy proposals that could be turned into policy, or we'll get "Once the culture of RfA changes, these problems will go away", or we'll get "I don't see those concerns as a problem." UltraExactZZ ^Claims ~ Evidence 12:59, 27 August 2008 (UTC)

I agree with this. I also think we should specifically look at areas that contributors have stated doesn't work well for them, and try to address them by building up a set of requirements to initiate the recommend phase with. Areas that people feel work well can have less of a focus. Hope this makes sense Gazimoff 18:51, 27 August 2008 (UTC)

Agreed. I've tried to start posting some drafts, along those lines. Recall will be a fun one, and I tried two sides of the same coin with that one. UltraExactZZ ^Claims ~ Evidence 15:24, 28 August 2008 (UTC)

Wikipedia:Perennial proposals

I'm seeing several recommendations from the RfA Review that are listed as Perennial Proposals. For example, 16 editors suggested that admins should stand for reconfirmation periodically - and this is listed as a perennial proposal, which fails largely due to the scale of the task (over 25 reconfirmations per week for the 1,500+ admins currently on the list). Do we still forward this as a proposal? Or do we cite WP:PEREN, and go in a different direction? De-bundling the tools is on there as well, with its failure cited as a technical issue. Is that worth revisiting? UltraExactZZ ^Claims ~ Evidence 14:24, 27 August 2008 (UTC)

I say it is worth revisting. Perhaps if a number of editors still think it's a good idea, such as the 16 mentioned above, maybe some changes could be used to the alter the idea to make it more feasible, such as making the reconfirmation RFA only every two years or only applicable to "active" admins, in order to cut down on the load. Since "information is king" I think as many proposals as possible would be the best. Useight (talk) 16:31, 27 August 2008 (UTC)

I would agree, though I think we should give as few details as possible, so that the meat of the proposal comes from the editors. Maybe something like:

• "Some editors favor requiring admins to have their administrator status confirmed periodically, in order to ensure that they have the trust of the community. What do you think? How often should such reconfirmations take place? How complex - or simple - should the process for reconfirmation be?"

I think that would hit the highlights. It also leaves as much as possible to the editor. Thoughts? UltraExactZZ ^Claims ~ Evidence 18:59, 27 August 2008 (UTC)

Discussion of negative consequences of failure?

On my RfA Review page, I said:

...I believe there are many people who hang out at WP:RfA would can tell with 80% or 90% accuracy whether a candidate is likely to pass or not. It would save the time of people at WP:RfA, improve the quality of their advice (since they wouldn't have as much work to do), and save the time and feelings of the candidates, if people would hold their fire, and the process would start with someone knowledgeable saying something like "Please look at the following 3 previous candidates whose contributions remind me of yours...notice they didn't succeed, and consider the advice they got. Are you sure you want to do this now, or would you like to prepare a bit first?" The question should only come from someone who has a track record of making the right call and of having the best interests of the candidate at heart; asking a candidate to withdraw for any other reason is like to either be or be perceived as bullying.

I haven't seen discussion on other pages about the negative consequences of failure. If someone thinks they'll do great at RfA if they just do X, Y and Z, and they spend 3 months jumping through what they think are the right hoops, and fail, it's likely to sour them on Wikipedia. We don't need to say "You can't run for RfA unless you've been active for 6 months", but having a few people act in the role of "trusted advocates", with the job of giving sound advice to any candidate about their chances based on the track record of similar candidates at RfA, would probably streamline the process, save people's time, and help morale all around. - Dan Dank55 (send/receive) 16:23, 27 August 2008 (UTC)

But who will be on this "filtering" team and how do we decide who goes on the team? Does this extra layer actually provide enough benefit to justify its creation? Useight (talk) 16:41, 27 August 2008 (UTC)

Some of that occurs at Admin Coaching, with coaches themselves evaluating candidates. This is good and bad; it creates that mentor relationship, but it also focuses the coaching on RfA, which is problematic. That's one of the criticisms of self-noms, that no one has vetted a candidate before they go to RfA. In theory, a nominator has reviewed the candidate's work and thinks that they can pass an RfA. UltraExactZZ ^Claims ~ Evidence 16:54, 27 August 2008 (UTC)

First, this is what the nom shold be doing. The nom should be asking 1) Will the candidate make a good admin? 2) In the current environment, will the candidate pass an RfA? But, let's ignore the nom. Your proposal is, in part, what admin coaching is intended to do. If a coachee fails an RfA, then I put the blame for that failure on the Coach not the candidate. (Conversely, if the coachee passes an RfA, I put the credit for passing on the candidate.) Unfortunately, we've had a large number of coachees that failed in May/June that gave coaching a black eye. This is a work in progress but I was taking a look at what caused coaching to fail a few months ago. It failed because suddenly people started treating coaching as a "get by RfA" pass and rushed candidates through the process and not truly vetting them... or the candidate pushed the envelope and decided to run without their coaches support. I think some people do a better job at vetting candidates than others. Some people can predict the fate of an RfA better than another, but you can never know. I like to think I vet candidates pretty well and have a sense of the way the wind is blowing. Many of my candidates/coachees get 100+ supports because I don't nom people I don't think will be good admins and with one notable exception I don't nom them if I don't think they will pass. That being said, I recently had a candidate get SNOWED on me... I THOUGHT he would pass, but his RfA didn't even last a day! You never really know what is going to happen.---Balloonman ^PoppaBalloon 17:07, 27 August 2008 (UTC)

Agreed - we had several editors criticize coaching as "Teaching for the Test". You know, it might not be a bad idea to do a mini-RfA Review with the 5 or 10 best coaches and see what they're doing right, then use that as fodder to expand/formalize/rennovate the current coaching process. Best Practices are usually called that for a reason, after all. UltraExactZZ ^Claims ~ Evidence 17:24, 27 August 2008 (UTC)

Actually, that is what I am kind of working on... reviewing what is being done... and who is doing it. I know that there are some people whose coaching I DO NOT respect. There are others who do a good job. I'm happy when the big criticism for my coachees is that they "might be over prepared." It means that I did a good job.---Balloonman ^PoppaBalloon 17:30, 27 August 2008 (UTC)

My fault, I'm not being clear, I'm talking about self-noms. We currently have 3 open self-nommed RfAs, all of which are on the minus side of 70%. All of these guys chose, for whatever reason, to go it alone. I don't want to say there's only one system that would work, because I'd be wrong, but just for instance, suppose that everyone has the "right" (if they're moving in the right direction) to get assigned to an advisor, who will give them a quick assessment every couple of months until they're ready...nothing too exhausting or too fancy, just "tell me what you've done good and bad...nope, look at these 3 other noms, their contributions remind me of yours, and they didn't make it...look for yourself and see if you agree; you may want to study them and see what went wrong, but if you think you can make it anyway, go ahead"...that kind of thing. Suppose we have a sign-up sheet for this role of advisor, and take only the first 10 volunteers (because if it's a long list, no one advisor will be able to establish the kind of track record of impartiality that would lead skittish candidates to trust them, although the list of 10 might change over time). At the top of WP:RfA, we have an infobox that says, if you want to self-nom, here's a list of people who have a good track record in this kind of thing. (My guess is it would work better to assign people to advisors, otherwise potential candidates will think...wrongly..."Oh, Balloonman is famous, I'll do a lot better if I pick him".) Granted, if the candidate doesn't give honest information to the guy they're relying on, the results will be bad; you can lead a horse to water, but you can't make them drink. This is another reason for a more lightweight advisor–advisee relationship than what's available now; it doesn't suck up too much of the advisor's time with a candidate who might be less than honest. - Dan Dank55 (send/receive) 18:02, 27 August 2008 (UTC)

Some would say I'm infamous... not famous...---Balloonman ^PoppaBalloon 18:11, 27 August 2008 (UTC)

I'm not sure that the advisor role couldn't be filled by a coach, if only as a form of pre-coaching. The caveat there is that we're already short on coaches, so adding more burden to the process as-is would be problematic. An alternative might be to show the last 20 RfAs with self-noms, and the last 20 RfAs with regular nominations. These lists would show the editor's editcounts, length of experience, and support/oppose %. This might be a good way to illustrate minimums without actually debating and setting them; the average of the last 20 RfAs would be given in terms of successful and unsuccessful, so editors considering it could see the trends. if they're below the average for successful RfAs, they might consider waiting. Oh hell, I'm going to have to do stats on this, aren't I? UltraExactZZ ^Claims ~ Evidence 18:31, 27 August 2008 (UTC)

←Absolutely! I complained about style guidelines, and all it got me was the "honor" of doing the monthly WT:UPDATES :). That's actually not a bad idea; people would trust statistics even more than they'd trust an advisor. Excellent approach. - Dan Dank55 (send/receive) 18:46, 27 August 2008 (UTC)

I also strongly encourage editors to try out Editor Review before going for RfA. It might identify problem areas or pitfalls beforehand. It might also gain you interest from an experienced editor or administrator willing to nominate you for adminship at some stage in the future, or who would be prepared to coach you. And Ultra, while I'd love to help you out on the coaching side of things, I'm still way to inexperienced as an admin to be useful there. I'm happy to perform editor reviews and assesments though inorder to provide people with useful feedback. Hope this helps, Gazimoff 18:57, 27 August 2008 (UTC)

And maybe Editor Review is something that can provide insight into coaching and pre-adminship mentoring... Maybe there are best practices there that would work well elsewhere. Too... many.... ideas.... UltraExactZZ ^Claims ~ Evidence 19:35, 27 August 2008 (UTC)

As a point of information, based on the last 25 successful Candidates for Adminship (from Lifebaka to Ice Cold Beer), the average support ratio is 94.08% (Support/Total, discounting Neutrals). The average editcount at the beginning of the RfA was 10963. If we take J.delanoy and Good Ol'factory out, both of whom had well over 45000 edits at the start of their RfAs, the average is still 8003. The average age of account for each candidate, as measured from first edit to the RfA being transcluded, is 22 months. Haven't looked at Unsuccessfuls yet, but I suddenly found this fascinating, so I thought I'd post it. UltraExactZZ ^Claims ~ Evidence 19:35, 27 August 2008 (UTC)

More details - The range of editcounts was 45000 on the high end (noting that the actual counts are much higher) to 2974 on the low end. Account age ranged from 45 months to 2 editors with only 5 months of experience. The averages for unsuccessful candidates are actually not that different from the successful candidates; The average edit count for the last 23 unsuccessful RfAs (not counting NOTNOW closures, but counting SNOW) was 7495 edits. Average account age was 19 months. UltraExactZZ ^Claims ~ Evidence 20:13, 27 August 2008 (UTC)

Interesting. As a side note, I have User:Useight/No Support, which is a tally of the last ~400 RFAs that got snowed with 0 support and how many edits the candidate had (along with what month it was and how many opposers there were). I also have the same data in a spreadsheet if that would help for any reason. Useight (talk) 20:35, 27 August 2008 (UTC)

Draft Recommendations

OK, so I've taken the most common (or strongest) sentiments from the responses and tried to draft recommendations around them. These took the form of "Editors said RfA should be X and Y, how should we accomplish this?" The dilemma - I have 16 of these already. For reference, we had 15 questions. Please review the draft, and see if we're missing any major items from the responses. Please also rephrase my wording wherever I was unclear; the simpler we can be, the easier it'll be for editors to respond. Thanks, UltraExactZZ ^Claims ~ Evidence 13:48, 29 August 2008 (UTC)

I know there's discussion going on about inactive and active admins; if there's a question that can come from that, I'd like to add it; otherwise, if there are no concerns about this list, I'll start setting up templates. Please have a look - thanks! UltraExactZZ ^Claims ~ Evidence 13:36, 2 September 2008 (UTC)

I'm reading over it now. As well as talking about what reccomendations for further examination we have, I'd also like to include suggestions about what currently works well so that we can feed back some positive messages. I'll make some tweaks for you to look at. Hope this helps, Gazimoff 10:18, 3 September 2008 (UTC)

Agreed - My only concern would be that recommendations about things that work might end up changing what works about them. Maybe some general baseline-style questions, like the ones we had at the end last time around. UltraExactZZ ^Claims ~ Evidence 12:59, 3 September 2008 (UTC)

Kinda. Reccomendations should form a starting point to generate a set of requirements - a list of items that a process needs to fulfil in order for it to be considered fit for use. It may be that through the reccomendations discussion, some requirements will be added and a priority list will form. It's then easy to identify where the current process misses those current requirements snd should be fairly straightforward to work out if any proposed replacement will meet them. Hope this helps, Gazimoff 14:41, 3 September 2008 (UTC)

I'm still having trouble sorting out what to add here - do the responses we already have not form a set of requirements for characteristics of admins and, by extension, what points RfA needs to address? Or am I missing something? UltraExactZZ ^Claims ~ Evidence 18:59, 5 September 2008 (UTC)

<-- I've added two questions, intended to review RfA and adminship overall. The first refers to the characteristics of admins, and asks if the process could be geared to focus on the trustworthiness issue (being the sole criteria cited in policy) or a set of desirable characteristics. The second asks the respondent to list parts of RfA that work, and contrast them with parts that don't. This question, I think, will be useful in weighting the recommendations - a strong response that questions are bad, for example, would lend greater weight to recommendations for changing the question portion of RfA. How does this work? UltraExactZZ ^Claims ~ Evidence 19:57, 8 September 2008 (UTC)

I've posted a notice at WT:RFA for additional eyes; if the current questions are sound, I think we can begin the Recommend phase within the week. UltraExactZZ ^Claims ~ Evidence 18:55, 9 September 2008 (UTC)

I took a look at this (saw the link at WT:RFA). Overall I have to say that the questions are looking pretty good, although there might be some concern regarding the scope and number of them. It has been my experience that when a problem is broken down into so many constituent parts and people are asked to address the individual concerns and attempt to find a resolution to them, that it becomes somewhat onerous to figure out any kind of broad consensus. I don't want to discourage the addition of new questions but such should be approached with caution, to ensure the process doesn't become so daunting that it winds up collapsing in on itself. Shereth 15:40, 10 September 2008 (UTC)

I think we have all the questions we can handle; I certainly can't see adding any more. UltraExactZZ ^Claims ~ Evidence 14:16, 11 September 2008 (UTC)

<-- How long do people have to prepare their answers to the questions? I think this should be placed on the page, not sure how long it was in regard to the previous steps. Also i have changed the message at the top of this page to reflect the stage where it is at now, feel free to tweak or change it. Thanks Monster Under Your Bed ^(talk) 12:46, 13 September 2008 (UTC)

The last questionnaire ran from 12 June to 1 July - Since this one is a bit longer, I figured we'd give it a month - but there has been very little discussion on the point. A good target date might be 8 or 15 October, both of which are Fridays. As for the top of this page - I totally missed that, so thanks! UltraExactZZ ^Claims ~ Evidence 12:56, 14 September 2008 (UTC)

Complete rejection

The questionaire makes the request for adminship process look worse than choosing and getting into university. The questionaire should be redone by a different group of people. --Gerry Ashton (talk) 23:06, 22 September 2008 (UTC)

I went to this discussion page wondering if anyone had the same reaction to the questionnaire as I. I have my answer. Like most group activities, wikipedia has a core which takes this way too seriously for my taste. But, happily, the I am still welcome to play around at the margins. For that I am thankful. Philhower (talk) 03:04, 23 September 2008 (UTC)

Statistically speaking, it is, actually. The percent of editors who become admins is far below the percent of high-school students who enter college (based on my own anecdotal evidence, and in the US; I could dig up actual percentages if asked), and there is no equivalent of colleges/universities below the Ivy League.

I'm not sure I agree with your analysis of why it's this way, however. Having looked over quite a few of the responses to the Question phase, there are a lot of editors who believe that RFA is hopelessly broken, as well as a lot who think it works just fine (an argument which we won't be getting into here). The questions in the Recommend phase (where we currently are) show this, in that they all ask if anything can be or needs to be done to correct problems. The way the problems were detected is through a statistical analysis of the data gathered in the Question phase (found here) to identify what a lot of editors believe are problems with the RFA system.

Also, while you may not agree that the Recommend questionnaire is relatively well balanced given the initial input into it, you are of course free to state on it that the RFA process as it stands works just fine, regardless of how exactly the questions are worded. Cheers. lifebaka++ 12:24, 23 September 2008 (UTC)

Lifebaka hit the points I was going to mention. I'd only add this - the whole point of the review, as I saw it, was to move from "Rfa is bad" to "These are the ways in which RfA is Bad" to "These are specific things we can do to make RfA less bad". If we had received 209 responses that uniformly said "RfA is good", then that would have been that - but some editors had concerns, and this list of questions is a result. Of the 114 responses we've received to date (!), several reject the questions outright - I've seen quite a few "This isn't a problem at all" responses, which is fine. The reason I'm spending so much time on this project is that it's different - it's editors stepping up and offering their insights, rather than Support or Opposeing the first guy who speaks up. UltraExactZZ ^Claims ~ Evidence 02:34, 24 September 2008 (UTC)

Individual questions in the questionnaire are all optional. If you only have the time/inclination to answer one question, please do so. Even a brief answer is helpful to the project. [Actually I admire the commitment of the volunteer who is going to review these answers and then collate a summary.] Axl ¤ [Talk] 10:48, 28 September 2008 (UTC)

Some statistically significant factors for adminship success

This should be of interest in the current discussion: Academic_studies_about_Wikipedia#Obtaining_administratorship. VG ☎ 18:36, 25 September 2008 (UTC)

Hmm, that's interesting. Not terribly surprising, though, about what does and doesn't help. Thanks for the link. Cheers. lifebaka++ 18:57, 25 September 2008 (UTC)

Something people will be interested in

If your looking at this page then the following discussion starting from here:Wikipedia_talk:Requests_for_adminship#7_admins_created_in_Sep_08._Crat_happy_to_nominate_.22unusual.22_RfAs. should be of great interest. 211.30.12.197 (talk) 07:54, 4 October 2008 (UTC)

Evaluation

As we begin to wrap up the Recommend phase, I note that 273 editors have formatted subpages for the RfA Review Recommendations. Are there thoughts on how to find the most common recommendations from that list? I am inclined to use a statistical model, as we did last time around, but wanted to get some input first. Thoughts? UltraExactZZ ^Claims ~ Evidence 18:09, 12 October 2008 (UTC)

Whew, this got lost in my watchlist, otherwise I would've been here much sooner. I don't know how well a statistical model will work here, as the questions were much more open, but it's at least worth a shot to see if anything meaningful comes out of it. Drop me a line if you need any help. Cheers. lifebaka++ 20:53, 27 October 2008 (UTC)

I've read through the first 24 responses, and they're good ones. About the only clear consensus at this point is that automatic reconfirmation sucks - but we already have more than 100 distinct recommendations on everything from RFA Clerks (!) to hybrid New Admin School/Coaching/Editor Review programs to neutral canvassing (!). Hopefully, next week I'll start posting tally lists. UltraExactZZ ^Claims ~ Evidence 14:25, 7 November 2008 (UTC)

RfA is not broken! ... not really...

• Blah blah blah RfA is broken. Nope. Changed my mind. It is NOT broken. I'd love to see changes that either make it harder to get an RfA, OR easier to get an RfA PLUS easier to be desysopped. But whatever. The system is what it is and it works reasonably well. The REAL problem here is that it takes years of abusive behavior for vested edtitors to be desysopped.
• But now we probably have a set of editors with a vested interest in the "reform RfA" meme. Ling.Nut ^{(talk—WP:3IAR)} 00:58, 28 December 2008 (UTC)

Progress?

I'm not sure who's managing this process, but it seems to have stalled :( Stifle (talk) 12:23, 13 January 2009 (UTC)

UltraExactZZ had been footing a lot of the work, and he hasn't edited since December 22nd. I'll shoot him an email later today, to see what's up. lifebaka++ 13:42, 13 January 2009 (UTC)

Any response? John Carter (talk) 22:21, 15 February 2009 (UTC)

I'm here John, I've read the pages in Category:Wikipedian Recommendations to RfA Review that were written by people whose names I recognize from RFAs, which IIRC was a little less than half of the responses. Because there were so many entries from people I didn't recognize and from people who I did recognize who don't generally participate at RFA, I think that adding up totals isn't going to be helpful. I'm currently working in userspace on a list of all the proposed ideas that haven't been generally rejected at WT:RFA since the RFA Review process started. - Dan Dank55 (push to talk) 22:48, 15 February 2009 (UTC)

Thank you for the work. :) I have to believe it is probably a rather daunting task, and can well understand how it might take a while to gather together all the information. John Carter (talk) 23:03, 15 February 2009 (UTC)

Given the recent developments over various topics and threads ... I'd like to kick this thing in the rear, and get some communication flowing in all directions. I'm interested in how RfA has an impact on those who have endured it. How has it affected your point of view in regards to both WP and RfA in general. How has it affected editing patterns. — Ched :  ?  17:10, 25 May 2009 (UTC)

Second. ~ Amory (user • talk • contribs) 18:37, 23 June 2009 (UTC)