Wikipedia talk:Article Feedback Tool/Version 5/Archive 1

Archive 1

Archive 2

Archive 3

→

Archive 5

More Like Product Review Engine

i'd like this extension to alllow users to rate a product review article but with 10 stars as an overallrating instead of 5points scale. 10 stars could have a custom size and AFT should have a setting to count ratings from all the time, not last 30 days - display overall rating changes in time (all time/year/month/week) — Preceding unsigned comment added by 213.134.163.135 (talk) 22:08, 6 November 2011 (UTC)

That's not going to happen; are you the chap who poked me on IRC? We are designing tools so that they can be best used by the Community, not so that they can be best used by third-party commercial websites. Okeyes (WMF) (talk) 07:51, 7 November 2011 (UTC)

Talk page integration

Note: ideas from this section have been accepted by the developers

I don't understand how the feed of comments is going to be: shouldn't those comments be merged to the talk page, so that there's a single point of discussion and feedback can be discussed easily (for instance, how to be acted upon)? Nemo 21:24, 27 October 2011 (UTC)

So, the initial suggestion is to maintain it distinctly - a dedicated thread of comments. However, some users (yourself included) have suggested integrating it with the talkpage. I've brought the point up with the devs handling this, and will let you know when they provide some feedback :). Okeyes (WMF) (talk) 22:14, 27 October 2011 (UTC)

Okay, hey again! I've spoken to Fabrice - they're going to look at talkpage integration in the second stage of development, which starts in January-ish. At the moment it's a question of fixing up the basics, then we can look at how to integrate stuff without making talkpages horribly clogged. Okeyes (WMF) (talk) 19:48, 28 October 2011 (UTC)

That would be very nice. I realize this may be a problem for high-traffic pages, but for the vast majority of pages, Talk: is a barren desert with a few weathered obelisks of WikiProject banners protruding. Getting an actual conversation on article improvements started in such places, even a somewhat banal one, would be welcome. Choess (talk) 02:11, 31 October 2011 (UTC)

Choess, I don't think you understand what will happen. If you understand a "conversation" to mean what the dictionary says it means—I say something, you listen to me and say something, and I listen to your reply and say something back—then it's not going to happen. This is not a "discussion" or "conversation" page. This is a "post a drive-by comment and go away, never to be seen or heard from again" page. WhatamIdoing (talk) 15:01, 31 October 2011 (UTC)

Well, the plan is to try and engage them further - it's in its early stages right now (the devs are still working out the process) but I'll let you both know when I find out the specifics. Okeyes (WMF) (talk) 15:18, 31 October 2011 (UTC)

That'd be nice, of course, but let's keep in mind that 97% of current AFT users are not logged in, which means that 97% of users will be difficult or impossible to contact the next day. Furthermore, you propose below that all AFT comments disappear after 24 hours, which is also not compatible with a conversation. WhatamIdoing (talk) 15:33, 31 October 2011 (UTC)

As said below, the 24 hours thing was merely an example, and it'll be [time period] without action, not just [time period] by default. I know the devs have thought of one way to provide contact details, but I'm going to check with them first just to avoid putting words in their mouth (hence the reticence to give specifics). Okeyes (WMF) (talk) 16:12, 31 October 2011 (UTC)

Okay, Fabrice has given me permission to spill :P. Basically the plan in phase 2 or 3 is to experiment with an automated email contact when someone's comment is addressed or replied to. In phase 1 (that's where we are now) the new "call to action" would ask non-registered users if they'd like to be contacted with followups, and, if so, prompt them to enter their email addresses. Okeyes (WMF) (talk) 09:48, 2 November 2011 (UTC)

The comments page, even if technically separate, could be transcluded into the regular talk page to increase it's visibility. It could also be transcluded into a collapsible box, like the {{FAQ}} template does, if you didn't want it to take over the page. Since it's IMO unrealistic expect to start a proper discussion with these comments, some degree of separation might be preferable. WhatamIdoing (talk) 01:33, 31 October 2011 (UTC)

That'd be a good middle ground :). I'll add it to the list of options. Okeyes (WMF) (talk) 15:18, 31 October 2011 (UTC)

• Okay, having spoken to Fabrice, it looks like talkpage integration is going to come in the "second" phase, which starts in January - first phase is about getting the barebones stuff together. Make no mistake, I'll keep your comments for when the second phase starts, and give you a poke at that point :). Okeyes (WMF) (talk) 09:37, 2 November 2011 (UTC)

Previous experience

I was for a long time active in Russian Wikipedia, and there the feedback tool page exists for I believe two years, ru:Википедия:Сообщения об ошибках. (Before Russian Wikipedia, it was introduced in Polish Wikipedia, but there I had no experience with it.). The link to the feedback form is found in the Toolbox field. It is a long-standing pain-in-the-ass page, since it creates a considerable backlog and requires a constant attention of several dozens of dedicated editors. I do not have any quantitative statistics, but my impression is that all comments are at least meaningful (possibly the meaningless ones are blocked by a filter, I am not sure) in the sense it is clear what the user actually wants. About 50% (in my estimate) are about typos which are easy to fix; about 10% are bogus (when the user misread smth or wants to add smth which clearly does not belong to the article or is plain wrong) and do not require any reaction, but the remaining 40% or so is addition of some unsourced information which looks credible but may be not credible (especially what concerns the BLP, the press-secretaries usually go the feedback form for whatever reason). The users who leave messages never come back and it does not make any sense to ask them questions. In many cases the dedicated users who work on the page can not handle the feedback, and then eventually it gets archived and goes to the talk page (and dies there). So I would say such feedback requires a lot of coordination (which should be thought out in advance - for instance, my contribution over there was to suggest to close every single discussion once it does not require any action anymore, then it could be automatically archived by a bot) and actually has very little useful output - but if you want to try, just try. The page should be popularized between the active users from the very beginning, otherwise in two days it will have a thousand messages's backlog.--Ymblanter (talk) 08:42, 28 October 2011 (UTC)

For the record, this "Report an error" feature (which is provided by ru:MediaWiki:Wikibugs.js) was mentioned previously during the discussion about AFT:

• mw:Thread:Talk:Article feedback/Delete/reply (9)
• mw:Thread:Talk:Article feedback/Justin Bieber article not complete?/reply (6)
  • mw:Thread:Talk:Article feedback/Justin Bieber article not complete?/reply (11)

By the way: its use here on English Wikipedia is still pending and, as noted on Polish Wikipedia, there is also a rewritten version on Portuguese Wikipedia, which is easier to translate to other languages. Helder 12:33, 28 October 2011 (UTC)

Awesome; thanks for the additional information. I would say that one of the aims of this tool is to provide an "on-ramp" for editing. It's not just "provide feedback", but "provide feedback...or why not fix the error yourself?" - if we can get the second bit right, hopefully it won't overwhelm people, since the readers will be taking a chunk of the work. Okeyes (WMF) (talk) 15:06, 28 October 2011 (UTC)

From my experience, it never worked. We even had a number of "regulars" who were willing to point out some inconsistencies in the articles on a regular basis but unwilling to correct them themselves (or actually to edit articles at all). It might be good to find someone who had an experience in Polish and/or Portuguese Wikipedia working on the feedback pages, it might be different doe to the differences in the interface.--Ymblanter (talk) 15:17, 28 October 2011 (UTC)

Interesting! I'll look around for someone. I would distinguish "report this error" for the new version of the AFT, though - the latter will hopefully actively encourage readers to edit, while "report this error" really doesn't (I don't think?). Okeyes (WMF) (talk) 01:02, 30 October 2011 (UTC)

Thanks for this rather troubling info, this reinforces my concern that this could divert people from fixing things to flagging them for others to fix. ϢereSpielChequers 16:58, 9 November 2011 (UTC)

Volunteer needed?

I'm willing to be a lackey alpha tester, or help out in any other way you need. I've been following this issue more closely than I have most issues, and want to help make it a success. Sven Manguard Wha? 11:04, 28 October 2011 (UTC)

Great! So, if you look on the schedule you'll see a load of dates for community involvement. The earliest "alpha testing" date is 10 November, when the first feedback forms drop - I'll leave you a note when they're out. In the meantime, the devs are still churning away at mockups and prototypes; looking at the existing version, do you have any ideas on how to make them better, or any opinions on the general themes so far? Okeyes (WMF) (talk) 15:22, 28 October 2011 (UTC)

My personal opinion

K, let me start. The reason I hate the current article feedback tool is it lets people who don't like the subject rate it 1 star for every category, when, say, it's extremely comprehensive, well-referenced, etc. That's the problem with the stars - no specific advice is really given. "Okay, I've added two hundred reliable sources from all these news agencies and Celtic F.C. is still rated 1.01 star for sources." That's basically the problem with the stars.

The current article feedback tool is also disadvantaged by the limited selection of the aspects - sourcing, bias, comprehensiveness, and quality of prose. I simply propose just to scrap using specific categories, as well, they don't help too much either, as I explained above. My proposal is outlined below:

• Make it have three buttons: "Did you find what you were looking for? Yes/No/Some of it".
• Make giving comments optional, but "highly recommendeded"
• You can make suggestions under two categories - just "suggestions" and "criticism". Praise and thanks doesn't really help.
• Text response - no stars, and it will be placed automatically at a subpage of the talk page, like Talk:Cat/Feedback or something. However, it should sort of be like a certain special page - admins can redact content without a trace.

HurricaneFan25 21:53, 28 October 2011 (UTC)

Pretty much all of that is already in the proposal. One question: why "yes/no/some of it"? Would it work better if it was yes/no but "did you find all of what you were looking for?" (italics mine)? Okeyes (WMF) (talk) 22:01, 28 October 2011 (UTC)

Yes, the second option seems better than mine - and if you select "no", there should be an automatic option to ask "what were you looking for?" or "why not?" HurricaneFan25 11:55, 30 October 2011 (UTC)

That'd be good; it would offer a prompt to explain to the user what we want to go in the text box. I'm not sure if it would artificially limit the range of responses, though - we'll find out! I'll suggest it to the techs. I'm just going to wait until I see if the other users down below like the proposed wording as well; hopefully we can hash something out that everyone's happy with before forwarding it to staffers. Okeyes (WMF) (talk) 02:00, 31 October 2011 (UTC)

Anonymous feedback

When giving feedback, users should have the option to not have their ip shown (except to checkuser or admin). Feedback on controversial articles may elicit unwanted responses...Smallman12q (talk) 21:59, 28 October 2011 (UTC)

Good point; I'll run it past the developers and get back to you with their thoughts :). Okeyes (WMF) (talk) 22:02, 28 October 2011 (UTC)

Tech are debating it; it's a "do we have transparency or anonymity?" question. They'll definitely consider it, though, but probably for phase 2 (which starts in January. Phase 1 is about general ideas). Okeyes (WMF) (talk) 22:39, 28 October 2011 (UTC)

Placement

Note: multiple ideas from this section have been accepted by the developers

One worry I have about the current version is that it appears at the bottom of the article. I suspect that in most cases readers who have serious frustrations will never get far enough down to see the Feedback request -- and the longer the article, the more likely that is. Looie496 (talk) 22:11, 28 October 2011 (UTC)

Agreed. The devs are talking about seeing if the idea of placing it somewhere on the left or right of the article (in a more compact form) can hold water. The plan is to gauge the community's opinion of the idea and then turn it on for a limited set of articles for a short period - A/B testing, basically. Then we can see if it makes an impact on numbers, and if so, does it make enough of an impact to offset the distraction. Okeyes (WMF) (talk) 22:23, 28 October 2011 (UTC)

Cool. Looie496 (talk) 22:31, 28 October 2011 (UTC)

On the other hand, it's not a bad idea to prioritize comments from readers who actually read the entire article. Besides, 99% of all Internet pages has their comments or surveys at the bottom. Internet users know this and all know how to scroll to the bottom of a page to post a comment. Placement of the AFT should be secondary to our primary goal: To serve readers information, not soliciting help or comments. --Bensin (talk) 00:52, 29 October 2011 (UTC)

Sure, but does a different placement harm the goal of providing them with information? And is there something to be said for making it easier and more obvious for users to say "you're missing X" so that we can pursue the primary goal? Okeyes (WMF) (talk) 01:35, 29 October 2011 (UTC)

I think it would be harmful if it is placed in the very top of the article at the expense of lead sections and infoboxes. That said, there is nothing wrong with placing something somewhere someone would expect it to be. Content on top, comments at the bottom. Seems right to me. --Bensin (talk) 02:27, 29 October 2011 (UTC)

That's fair enough. As said, I think the idea is to see if we can test it for a short period of time on a limited number of articles and see what the reaction is - does it get increased views? If so, are they all people going "this sucks, I can't find what I want, why is this box here?!". If the reaction is overwhelmingly negative, the idea stops there. Would that be acceptable? Okeyes (WMF) (talk) 00:53, 30 October 2011 (UTC)

I'm not really sure what you mean by "test it". Is the question to find out whether placing the AFT at the top of articles will generate more comments? I'm pretty sure it will. As will placing it in the middle of the lead section or making it yellow and blinking. But, as I said, our goal is not to harvest comments or ratings, and if that task in any way conflicts with our task of sharing knowledge, my loyalty is to the second.

There is much talk (which is probably good) that we want to make it easy for readers to comment/contribute. Why not make it just as easy for readers who don't want to comment to opt out from doing so? If the AFT is to be moved or made even more obtrusive, I'd be anxious to see it be fitted with an "X" in the top right corner. Clicking the "X" would dismiss it the same way a reader/user can dismiss top sitenotices. It would

a) be really interesting, for everyone, to see the numbers on how many readers and users choose to close it

b) be a service to the readers who don't like it. Let the 90% lurkers be lurkers.

To answer your question: I personally don't want the AFT to be placed anywhere else but at the bottom. It's the logical place for it and it is right there anyone would expect it to be. Suggesting placing the AFT at the top of an article seems to me, to be honest, vain. However: Fit it with an "X" and you may place it wherever you want without any objections from me.

I think last time I heard "test it for a short period of time on a limited number of articles" was at the first roll-out of AFT... But if the testing is biased, then I don't really see the point of testing at all. I had to push to get the "What's this?"-link in the AFT-box to even give users a fair chance to get involved. I don't understand how anyone can expect to get negative feedback on something if people don't know where to post it. The same goes for readers who dislike the AFT. Make it as easy to say "no thanks" as "yes please". --Bensin (talk) 16:38, 30 October 2011 (UTC)

All good points. By "test it" I meant "stick it somewhere else for a short period of time on a limited number of articles to see not only if we get more comments, but if the comments outweigh the dissatisfaction". I like your suggestion of a closing button as a way of gauging if readers hate the placement or not - I'll put it forward. Can you explain what you mean by biased testing? Make no mistake - this is just a/b testing. Even if the result was "readers love having it in the [insert new placement here]", it'd still be too early in the process for that to become permanent - we'd just go back to the drawing board keeping that idea in mind. Having said that, if by some weird freak anomaly the Foundation goes halfway through the consultation process "this works perfectly! Lets turn it on permanently and cull further discussion!" without a damn good reason, I will (quite justifiably) resign. I can't see that happening, though; the devs are genuinely making a far greater effort to engage editors. Okeyes (WMF) (talk) 02:04, 31 October 2011 (UTC)

As an addendum; would you agree that having a "hide/collapse" box would be a good idea permanently, not just for testing? I'd be interested in proposing that, but I don't want to put words in your mouth. Okeyes (WMF) (talk) 02:04, 31 October 2011 (UTC)

Regarding "biased testing": I think the test "for a short period of time on a limited number of articles" the foundation did when rolling out AFT was biased because to remove AFT and thus "vote no thanks", as I understand, you had to have an account and edit your skin (and that you could only turn it off if you used vector?). The low number of users who opted to do this was then used as an argument in favor of AFT. If it's not as easy to vote "no" as it is to vote "yes", the test is biased. Calling this a "test" was, from where I stand, little more than an alibi for the roll-out.

I would agree that having a hide-option permanently (like in the table of contents-box) would be a good idea. It's also really important that you can log the use of a hide/close-option, especially if you plan to move the AFT-box to an obtrusive position. You really shouldn't move it there though. Here's why I think so:

Pitting "more comments" against "reader satisfaction" is in my opinion a good example of how an interest is conflicting with our task of sharing knowledge. I do understand that constructive comments will eventually increase content quality (and consequently also increase reader satisfaction), but we also have a responsibility to share knowledge here and now. If the reader experience is not given the respect and attention it deserves we run the risk of losing readers to sites who just mirror the content but present it prettier than we do. Or someone might design a browser extension/plugin that "removes spam-crap from Wikipedia" like an ad-blocker. Losing readers means a decreased base for recruiting new editors. Nobody wants that! We want the readers right here. With us. Seeing what we see. Repeated attempts to manipulate readers, into rating or commenting or editing, will eventually backfire. The swarm is smart. Smarter than all of us. We, of all people, should know this :-) --Bensin (talk) 03:59, 31 October 2011 (UTC)

An excellent point; what I meant by "reader satisfaction" is precisely that - whether or not readers are pissed off by it. If they are, it shouldn't go through. Hopefully the test this time will not be "biased"; any a/b test on this feature will (or should) have an option to kill the tool. Okeyes (WMF) (talk) 04:05, 31 October 2011 (UTC)

Sounds really good. Designed right, used right and placed right this could be a powerful tool to accelerate improvement of our content without impairing the reading experience. If it does, everyone wins. --Bensin (talk) 04:47, 31 October 2011 (UTC)

Agreed! Thanks so much for your good ideas and feedback so far :). — Preceding unsigned comment added by Okeyes (WMF) (talk • contribs) 14:26, 31 October 2011

You are welcome. This time it was a pleasure. --Bensin (talk) 15:16, 31 October 2011 (UTC)

• Okay, Bensin, I've checked with Fabrice; he loves the idea of a close button, and it'll be included in any new placement tests :). Such tests will occur in Phase 2 or 3, apparently, not immediately, so it's safe for now. Fabrice has instructed me to thank you for this idea, although as usual I'd have done so anyway of my own volition :P. They are planning on adding a "feedback" button on the left or right side of the screen that jumps you down the page to the AFT box, though - thoughts? My feeling is it might not be too useful; it'll either be static, in which case most users will miss it, or moving, in which case it'll piss most people off :P. Okeyes (WMF) (talk) 09:17, 2 November 2011 (UTC)

I'm glad you and Fabrice like the idea of the "close" button. I agree with you fully on the "feedback"-button: It will either be not useful enough or pissing people off. --Bensin (talk) 02:55, 4 November 2011 (UTC)

To ability to close it:

• An gadget I want to hide/collapse all AFT windows. should be made for editors who get tired from this tool. I am not sure if this should just collapse all AFT windows to single line (as suggested below) or don't show them at all.
• It should not disappear completely after closing - there must be an easy way to reopen it. For example you close it: it collapses itself to a single line Make an suggestion/comment. (or something like that); you click on this: it reopens again.

To another placement:

• I would like to know results from different placement testings. But IMHO it should be left on the bottom of the page.
• If you want to make it more visible for not-whole-page readers what about a link after every title to AFK down there? Like

== History == [Feedback] [Edit]

What do you think about it? --18:47, 30 October 2011 (UTC), Utar (talk)

I think for editors, there's already a function to hide the AFT: Preferences > Appearance > Don't show the Article feedback widget on pages. I like your idea of sticking a link at the top of each section (or the bottom, maybe? That way feedback can be clearly linked to the section the reader has an issue with) but placing 5 or 10 or 20 feedback links on a page might be too intrusive. I guess we can test it in the same way we'd test placement :). Okeyes (WMF) (talk) 02:07, 31 October 2011 (UTC)

OK, it is already there. I didn't go through all preferences but just through Gadgets. Maybe feedbacking of parts of the article could be done the similar way: Enable section editing via [edit] links => Enable section feedbacking via [feedback] links. That way we won't mess pages for people who don't like it. And if you have those edit links enabled you already have lots of links in articles so making a change from [edit] to [feedback][edit] should not be intrusive at all. --07:25, 31 October 2011 (UTC), Utar (talk)

Yup; I imagine any new feature will come with an option to disable it. I'll bring the [feedback] idea up with the devs. Okeyes (WMF) (talk) 14:26, 31 October 2011 (UTC)

Okay, Utar; Fabrice finds the [feedback] idea really interesting, and he's going to bring it up at the next team meeting :). I think the plan is "wait on instituting placement tests and suchlike until we confirm the AFT itself doesn't make people carve their eyes out with a melon baller" but it's definitely under consideration :). Okeyes (WMF) (talk) 09:17, 2 November 2011 (UTC)

My gut reaction is that any user who knows how to activate "section feedback" is able to, and should, fix the problem in the article themselves. Not just give feedback or report the problem. --Bensin (talk) 02:55, 4 November 2011 (UTC)

Placement of feedback link

Regarding the proposals of placement of a feedback link, I want to revisit some things I brought up earlier regarding placement. Don't we prefer edits over feedback? If that is the case, then why does all of those suggested placements of the feedback links instead put an emphasis on feedback? Will the proposed links also have an "x" so readers can dismiss them? The proposal says "we expect to [...] implement the best of these solution in the final version." Best as determined by which parameters? Generating most comments? Generating best comments? Least intrusive the the 90% lurkers? (I feel comfortable suggesting the lurkers on Wikipedia are more than 99%) Again: our mission is to serve the world information, not solicit comments on it. Most readers are simply not interested in editing or commenting. And that is totally fine! Any of the suggested placements of the link will increase the volume of comments at the expense of the lurker-reader's experience and we risk losing those readers to other sites who prettier present our content. That would be a disaster! We need to look at the bigger picture: What's good for the Wikimedia projects as a whole, not just what's good for the AFT-projekt. --Bensin (talk) 00:51, 8 January 2012 (UTC)

Option 1 vs. Option 2

Note: Ideas from this section have been accepted by the developers

I think I prefer the clean and simple option 1, but that the question be rephrased as "Did this article contain the information you were looking for?" (or perhaps "the information you expected to find?") because that is what we want to know. My thoughts on option 2: "Suggestions" are what we want, "Questions" are better answered on the reference desk, "Problems" should be phrased as suggestions. "Praise" may be nice, but is not necessary. If you go for option two it would be better if the options were checkboxes. --Bensin (talk) 02:27, 29 October 2011 (UTC)

We'll have hard data to judge the effectiveness of the two options once this first trial is finished. I'd support, in later trials, comparing your proposed wording and others against "Did you find what you were looking for?" and basing the final choice of wording on their measured effects. --Anthonyhcole (talk) 10:21, 29 October 2011 (UTC)

When doing that, please remember to consider that the messages may be translated to other languages and that the rater may have selected another language in its preferences (or, the tool may be in use in wikis in other languages). Different translations may also have different effects. Helder 12:48, 29 October 2011 (UTC)

That's a good point, Helder. I'll pass it on to Fabrice, see what the response is - I imagine it'll probably be similar to Anthony's comment (we're going to engage in a/b testing and get hard data). Okeyes (WMF) (talk) 00:58, 30 October 2011 (UTC)

How do you measure if someone interpreted the question the way you intended? The question "Did you find what you were looking for?" can be interpreted in at least six ways:

1) "You were looking for this article. Did you find it?"

2) "Did this article correspond well to what you expected it to be?"

3) "Did this article contain the specific piece of information you were looking for?"

4) "Was it you, yourself, who found this article with the information you were looking for, or did you get help from someone else?"

5) "Did Wikipedia have an article on what you were looking for?"

6) "Last night you were wandering the streets. Did you then find what you were looking for?"

The first time I read "Did you find what you were looking for?" I interpreted the question as alternative 1 and I thought "Of course I found the article. Otherwise I wouldn't be reading this question related to this specific article." Ask the question you want answered. --Bensin (talk) 17:14, 30 October 2011 (UTC)

Nice list, indeed. Here is number 7 which could happen in some long articles:

"Did you find what you were looking for?" ~ "Oh yes, I finally found this AFT window. What a long way…"

I personally like option 1 better because even here on enwiki (and definitely not on smaller language versions) there is not enough editors to answer all questions. And we are an encyklopedia not an Q&A system. So asking readers to ask could be dangerous. Suggestions will do the work too (and some part of people will ask through first window they see whether it really is for questions or not, too; see some talk pages).--19:02, 30 October 2011 (UTC), Utar (talk)

Noted! I think we need to come up with a wording that makes clear what we're asking, and that this is to be used to suggest improvements to the article, not to ask general questions. Perhaps two different things? "did you find all of what you were looking for?" by the button, and then "do you have any suggestions for the article's writers?" or something by the comments section? I'd love to hear some proposed new wordings being bandied about, since I think we agree that the current one is too ambiguous to be useful. Anyone have any ideas? Okeyes (WMF) (talk) 02:10, 31 October 2011 (UTC)

Before wording the yes/no-question someone should propose what it is we want to know from the reader. My number one choice, however, is that we limit ourselves to one single question in the entire box: "How can this article be improved?" One question from us, one answer from the reader. Succinct and easy. --Bensin (talk) 04:09, 31 October 2011 (UTC)

That's true; are we assuming we're working on the first design, not the second here? Okeyes (WMF) (talk) 04:15, 31 October 2011 (UTC)

I prefer option 1, yes, but with only one question: "How can this article be improved?". The simpler the better. --Bensin (talk) 04:38, 31 October 2011 (UTC)

Come to think of it, I guess I'm just less interested in readers' evaluations of articles than their constructive comments on how it can be improved. --Bensin (talk) 22:01, 31 October 2011 (UTC)

That's a fair point; I can understand why both approaches would provide some advantages, it's a question of which one provides the most. I've just send off my notes and suchlike to the staffers, so should have some responses to ideas and queries very soon. Okeyes (WMF) (talk) 22:10, 31 October 2011 (UTC)

• Okay, responses from staffers! So, Bensin, Fabrice likes phrasing it as simple a request for pointers rather than a general "got anything to say?" - he's working on a question along the lines of “how could this article be improved?”. We'll test it out after the A/B tests confirm which of the feedback forms is most liked by users. Okeyes (WMF) (talk) 09:06, 2 November 2011 (UTC)

Sounds good. Thanks for the feedback! --Bensin (talk) 02:26, 4 November 2011 (UTC)

I'd like it to simply ask "Is there any content that you would like to see added to this article?" It's simple, to the point, does not prompt an emotional outburst, and has a high potential to improve article content esp. when a large piece of info is missing/requested :) ~User: Hestiazfire

I like that :). I think a problem is it has the potential to miss some things. Like, would adding more citations be adding more content, for example? Would readers grok that? Thanks for choosing to comment, btw - it's great to see new users getting so involved :). Okeyes (WMF) (talk) 22:33, 11 November 2011 (UTC)

Comments

Note: multiple ideas from this section have been accepted by the developers or are under discussion

The idea of having a written feedback form is probably an improvement on the current polling method in that clarifies the particular concerns and will probably expose any reviewer bias. However, there is a concern that the reviewer may later regret their comments and want to have them removed. In terms of data collection, the previous type of form may be more useful on heavily trafficked pages, where the number of reviewers will tend to wash out a small number of biased votes. It may also be useful to track individual votes and apply robust statistical methods.

I would like to request that the new output form have a link from each comment to the matching article history version that was being commented upon. It'd also be good to have a standard page watch option for editors, if it doesn't already. Thank you. Regards, RJH (talk) 19:39, 29 October 2011 (UTC)

That's a great idea! If feedback is being acted on we do need to know what article version there is. I'll also ask about the comment removal and see what can be done. In terms of removing bias, the current plan is firstly to invite readers and editors to "rank" existing comments by usefulness, and second to have a cutoff. We're not sure what the cutoff will be yet, but if (for argument's sake) we assume it's 24 hours without action or 1,000 other comments or whatever - after a comment is 1,000 comments old, or has been around for a 24 hour period without any sort of commentary, upvoting or downvoting going on, it'll be killed and won't appear. Okeyes (WMF) (talk) 01:01, 30 October 2011 (UTC)

Hey again, RJH; I've spoken to Fabrice. His words were "great minds think alike" :P. Apparently some version of that is already in the works, although they have it structured as a "show a diff" link rather than a "show the original version" link. I'm arguing for it to be the latter, and I'm sure you'd prefer that; if the objective is to see what they saw, well, diffs don't allow that. Thanks for the great suggestion :). Okeyes (WMF) (talk) 10:24, 30 October 2011 (UTC)

Comments:

• • I've replied inline just for consistency. Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
• The yes/no buttons for "Did you find what you were looking for?". Yes, but you might consider a Likert scale, something in the middle of yes and no. The binary option assumes readers have something definite in mind before they visit an article, which may not always be the case. Even a middle response button of "partially" would be an improvement. That might prompt more posts.
  • Good point! I'll bring it up with the devs, see what their take on it is. Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
• Is it worth accepting feedback from registered/logged-in users? They're a tiny proportion, and the target is outside the project. I always feel a little awkward using the AFT as an insider.
  • I think "yes". There's a distinction between "registered users" and "editors"; they could be registered because they want to turn appearancey things on or off, or they could be researchers, or they could be refdesk users. A lot of editors are readers as well. I know there's currently a function to turn off the AFT if you're logged in - would that work, or do you still want to be able to see the actual "box" to evaluate the data returned? Note that the ratings system is going to disappear under both proposed versions. Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
• The drop-out rate between starting the edit process and completing the edit is disappointing. So is the drop-out between completing the edit and joining up. What stops people from proceeding on both counts?
  • Now that is an excellent question. I don't know if we have any data on it, but we should; it would seem silly not to establish what the problems are before working on an improved "call to action". I'll find out if we have that data, and poke people to gather some if we don't :). Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
• In any case, could the system avoid asking logged-in editors whether they'd like to edit the article? Bit annoying when you just want to see the previous results.
  • Good point; as said, though, the actual "ratings" system is going to be removed, so there's not really going to be a need to see any results (or rather: any results will be listed on an article's feedback page, distinct from the actual AFT box). Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
• Will the system still dispense with data after X number of entries? (Is it 30? I can't recall.) I'm struggling to see why we don't value ever-larger samples, unless they'd be contaminated by the evolution of the article itself. Tony (talk) 14:04, 30 October 2011 (UTC)
  • I think the idea is to avoid a system where readers can "game" the AFT (you know, "I don't like Justin Bieber, so I'll rate it down and get all my friends to do it too", although why on earth they read an entire article about someone they loathe I don't know); the ratings expiring means any attempt to bias it in one direction or another swiftly vanishes. That being said, the ability to game should die with the death of a "rating" system. I have actually asked the devs to hide comments after a certain age, or after a certain number of comments have been submitted more recently (to avoid creating a backlog), but there's no reason why we can't keep them on the toolserver or something; are you looking to be able to grab the comments themselves, or the yes/no/maybe ranking attached? Or both? Okeyes (WMF) (talk) 02:18, 31 October 2011 (UTC)
    • Thanks for taking on this task. It's good to know there's movement on this. I didn't keep abreast of the fact that the numerical ratings will be entirely ditched. Possibly fewer readers will post verbal comments than click on the current scales, but I guess you're prepared for that (the feedback might well be more useful, even if from fewer people). Tony (talk) 09:26, 31 October 2011 (UTC)
      • I think that's the idea :). Feedback will be actionable, which is currently isn't - I think on the article improvement front, 20,000 (or whatever) comments > 40,000 non-actionable ratings. On the "getting readers to become editors" front, obviously fewer people is bad, but we may be able to offset that by improving the "calls to action" (and we don't actually know what loss in numbers we'd get yet. The devs will do a/b testing and compare the results). Thanks for your suggestions thus far, Tony :). It's great to see community members responding to foundation calls for action despite the historical problems with the process! Okeyes (WMF) (talk) 14:31, 31 October 2011 (UTC)
• Okay, response for the devs on the "did you find what you were looking for?" point - some surveys show that a majority of readers reach WP through things like search engines, or are generally going to pages intending to read about a specific thing, so the current yes/no approach is the best on that front. Now, I'm questioning whether or not "the majority of users reach a page intentionally" is the same as "the majority of pages are reached intentionally" - "six degrees of Wikipedia" exists as a game for that very reason. I've dropped him a note asking if we have any data on pages as opposed to readers. Okeyes (WMF) (talk) 09:10, 2 November 2011 (UTC)

Several questions

• 1. So there won't be only Yes/(Maybe)/No scale and place for text?
  2. Won't we lose some of the feedbackers this way for whom numbers where OK but writing a comment is too hard/don't have time to express themselves.
  3. Do we need those analfabetic or busy reviewers?

• In today version there was some sort of questionnaire after filling (all? really don't know; found it only once) ratings in which I was also asked to fill suggestions to improve. I didn't found them anywhere showed after completing the survey.
  1. (to version - probably - 4) Are those suggestions somewhere?
  2. (to version 5) Will they be merged with suggestions we get from this questionnaire?
  3. So there would be a new tab for suggestions and later it would be probably merged to talk pages?

These should be all my ideas for now. See you later. --19:28, 30 October 2011 (UTC), Utar (talk) So, in order:

1. Yup, there'll be a "comments" box as well;
2. Possibly. It'll be fairly easy to test, actually; we can just pick a high-profile article for testing the new forms on. Compare the numbers they get to the numbers the current version gets on an average day, see what the loss is. I would say that from an article evaluation point of view, 20,000 actionable comments tops 40,000 ratings. From a "giving readers an on-ramp to editing" point of view, it'd be a loss, but hopefully improvements made to that aspect will lead to a higher percentage of readers trying their hand, and will make up for any drop. There might not be much of a drop, of course; we'll find out when it's tested.
3. I'm not quite sure what you mean by this; could you explain further?
4. Hmn. I'll ask the devs about that and see what's going on; should have a reply for you within 24 hours.
5. As above.
6. It's an either or; either it's merged to the talkpage or, if editors aren't comfortable with that, it gets its own distinct subpage (presumably linked from the talkpage in some way). Okeyes (WMF) (talk) 01:46, 31 October 2011 (UTC)

3: It should have been something you answered in 2: if having (probably) fewer text comments is better/worser than (probably) more number ratings. --07:13, 31 October 2011 (UTC), Utar (talk)

On an "improving articles" front, it's acceptable (imo) to have fewer comments if the comments are actually actionable - that is, editors can look at them and go "aha, I know what needs fixing" - which they currently can't. On the "getting readers to become editors" front, obviously fewer people is bad, but we may be able to offset that by improving the "calls to action" (and we don't actually know what loss in numbers we'd get yet. The devs will do a/b testing and compare the results). Okeyes (WMF) (talk) 14:34, 31 October 2011 (UTC)

Yes, that's fine with me. --19:05, 31 October 2011 (UTC), Utar (talk)

Feedback form if used for reporting incorrect scientific content

Option 1 (also if rephrased "Did this article contain the information you were looking for?") would possibly restrict the feedback options too much. What if the information was there, but was given incorrectly? As a user I may think, well I have been looking for the correct information, and not for nonsense. If I like to report a mistake or incorrectness, shall I answer yes or no?

The feedback tool could be used to improve articles in scientific disciplines (my field is biology). The basic problem there is that WP contributors have cited sources that contained incorrect information - my experience is that these kinds of mistakes are extremely difficult to get corrected in WP (because the authors argue that the source was given and that for changing this passage, other sources showing the contrary must be provided - background is that those authors are often not experienced experts in the involved very special subdiscipline and do not have the skills to see that the source from which they took their information contained mistakes or unubstantiated arguments). I made bad experiences in the German and English WPs, a colleage of mine had the same problems in the French WP. It gets very difficult to remove mistakes when the authors are experienced WP authors. I have largely given up editing pages for these reasons, and I am not the only one. It's usually 2 or 3 bad experiences (experienced WP authors being successful in defending scientific nonsense because they know the tricks) and then you practically never contribute again.

I can imagine that a tool designed to report nonsense or incorrect information (in a button-like form without the WP author having the chance to answer) could possibly have the effect that the WP authors could somehow be encouraged to change their behaviour (especially if bad ratings could be accumulated for each responsible WP author). My suggestion would be to use the feedback form to rank the scientific skills of those WP authors who feel responsible for certain pages (meaning that they have these pages on their watchlists). The objective is not to force scientists to improve pages by getting involved in endless discussions with experienced WP authors, but more indirectly, by downvoting the scientific value of the content without even knowing which WP author would currently feel responsible for it.

The result again, could be shown at the corresponding page of content. "The WP users who have this page on their watchlists, have in average obtained the following rating score for all the pages they feel responsible for". This would eventually provide some help to judge the reliability of the scientific content. Experienced WP authors who survey hundreds of pages would not get voted down by someone who dislikes a certain football team and gives a bad score for one page. But WP authors who repeat the same mistakes over and over again in their pages would sooner or later obtain bad ratings.

The present tool had this checkbox "I am highly knowledgeable about this topic (optional)". Was this successful? I never used it. I do not know if the things I am raising here have already been known or discussed.

As a user I don't care much on the cited sources if I have the feeling I can rely on the given information. If I see that I cannot rely on the given information, my experience is that the fact that a source was cited, does not at all improve the content. Citing of sources and reliability of information are disconnected from each other. If sources are cited, this can at some occasions even mean "the content of page is under dispute, and this is why various conributors needed to cite sources, for their edits not getting immediately reverted". If no or few sources are cited, this can mean, "the information is reliable, nobody has disputed it and forced others to cite many sources". -- FranciscoWelterSchultes (talk) 23:04, 30 October 2011 (UTC)

I see your concerns about Option 1 - I think the devs are planning on a/b testing, but rephrasing the question would be a good idea. I'll bring it up with them. Re scientific content, I'm not sure if that would fly. The goal of this is to provide feedback on the articles, not the contributors, and to do so over the entire wiki - while there is a secondary application to scientific articles, this tool isn't designed to apply just to them. We're also trying to move away from simple ratings, and towards comment boxes or other mechanisms whereby readers can not only say "something is wrong" but say "X is wrong", allowing editors to (hopefully) fix the issues. Okeyes (WMF) (talk) 01:31, 31 October 2011 (UTC)

IMO

This is so different from the previous versions that it's hard to compare them. However, I expect that one set of complaints will go away ("He rated my great article as all ones just because he hates the actor") and be replaced by another set of complaints ("He complained about my great article just because he hates the actor").

Do we have a plan for dealing with libel in the comments? WhatamIdoing (talk) 01:43, 31 October 2011 (UTC)

This'll be dealt with in several ways (hopefully). First, comments will be filtered through the Abuse Filter, amongst other things, for obvious badwords. Second, there's going to be a "hide" or "remove" function, in the same way that the MoodBar has one (and presumably for the same reason). Third, one of the things I insisted on when I started looking at this was that the devs work in some sort of cutoff point for comments - top 1,000, or the ones posted in the last 24 hours, or whatever - and everything outside that range gets dropped. This was intended to stop the AFT creating an extra pile of work editors feel obliged to deal with (or at least to keep the workload small) but there's no reason it can't be used for defamatory edits too. If something somehow gets past all the editors and the abuse filter, it dies in a relatively short space of time anyway. Nevertheless, if you have any other suggestions for how to deal with that problem, I'd love to hear them and pass them on to the devs :). Okeyes (WMF) (talk) 01:50, 31 October 2011 (UTC)

I'm not sure I understand that. Aren't comments going to be listed per article? Looie496 (talk) 03:02, 31 October 2011 (UTC)

They are - what's the point of confusion? Apologies if I didn't explain too well :). Okeyes (WMF) (talk) 03:16, 31 October 2011 (UTC)

It seems to me that it will take a very long time for most articles to get 1000 comments, and it doesn't seem to make sense to only keep the comments from the past 24 hours, so your numbers are confusing to me. Looie496 (talk) 06:29, 31 October 2011 (UTC)

Me too. Judging from the number of readers presently rating articles I work on, I'd expect 5 to 10 comments per week on 5000 hit/day medical articles. --Anthonyhcole (talk) 07:14, 31 October 2011 (UTC)

Indeed; I was more giving an example number; I imagine it could be determined on a per-article basis (obviously things like Justin Bieber are going to get a lot a day, less prominent articles not so many). We could structure it to scale: "if [number of comments] exceeds [specific number a day], archive every [number of comments]" or something. Thoughts? Okeyes (WMF) (talk) 14:36, 31 October 2011 (UTC)

I don't really care about the details as long as the comments about articles I maintain stick around long enough for me to see them. Looie496 (talk) 16:47, 31 October 2011 (UTC)

Okay; that's fair enough. We'll work something out. Okeyes (WMF) (talk) 17:37, 31 October 2011 (UTC)

I think that you'd normally default to a week as the minimum time period; we run PRODs and such on that schedule with the idea that some people visit Wikipedia once a week. However, that's too long for libel, and I honestly don't think we can assume that someone will come around and patrol the comments just for the sake of weeding out libel and spam.

So—Pending changes on the comments pages? Visible only to logged-in users? At least set them all to be NOINDEXed? WhatamIdoing (talk) 22:58, 1 November 2011 (UTC)

NOINDEXed certainly sounds like a good idea - I'll add it to the list. The visibility to logged in/not logged in users is something I have to discuss with Erik, Howie and Fabrice on Thursday; we'll see what comes out of the meeting. Okeyes (WMF) (talk) 05:54, 2 November 2011 (UTC)

• Yes, Fabrice confirms that NOINDEX tagging is being discussed with the development team :). Okeyes (WMF) (talk) 14:20, 16 November 2011 (UTC)

My humble opinions

I'm new to the conversation on this page and I haven't taken the time to read all of the recent conversations about the subject, so I'm sure I'll be covering territory already discussed, but I've been asked to come and voice my opinions/concerns, so here I am. I've been in several heated discussions regarding the AFT and I've honestly gotten burned out on the subject, so I'll try and summarize my views succinctly.

I'm not a fan of the tool for several reasons. The first and most important being what's already been stated by others hundreds of times - From what I've seen, much of the time the ratings people leave are based on the subject of an article rather than on the merits of the article itself. I've found this to be a particular problem with "celebrity"/"pop-culture" pages, which includes actors, singers, politicians - basically, any well-known public figure's "biography" page, as well as pages for TV shows, movies, video games, etc, etc, the list goes on and on. I've seen "featured" articles rated very low because simply people don't like a politician and "stub" or "start" articles rated very high simply because 12-year-old girls think the new tween heart-throb-of-the-month's new haircut is "cute", so the ratings are often meaningless as far as I'm concerned. When I've pointed this out in the past, I was dismissed and told this tool was supposedly created to lure casual users into editing and not to provide any real indication of an article's quality but then I honestly just don't see the point of a ratings system if its results are intended to be ignored from the outset.

One of my other main problems with the tool (I have too many to recount them all again here) is the four criteria we're being asked to rate. In my opinion, even the well-intentioned "average reader/rater" (this includes myself when it comes to 99.999% of pages) is not qualified to judge 3 of the 4 criteria. If I come to an article with 40 or 50 or 100 or more sources, I'm not going to sit there and take 2 or 3 hours to read and verify every single source before rating the page. Unless it's a subject I'm very knowledgeable about, the one criteria I'm reasonably qualified to judge is "well-written". The rest of the criteria, "trustworthy", "objective" and "complete" are criteria that I would have to sit and check every single source before being qualified to rate (short of an obvious lack of sources, glaring bias, or obvious "stub" status.)

As I've stated before in previous conversations, I would personally vote to just scrap the tool - However, if we are to keep it, I would suggest adopting IMDb's system of making ratings "hidden" on the page until the article has received at least 10 votes. This way, a good article doesn't sit with one or two stars for months based on one or two 10-year-old girl's opinions. If the average reader sees a page is rated very low, it's human nature to suspect there must be something specious about the article as a whole since the average reader most likely will not realize that the ratings are simply being manipulated based on people's opinions of the subject rather than the quality of the article, again, unless they intend to sit and research/verify all the references/sources on a page (which sort of defeats the whole purpose of Wikipedia for the average reader.) My other suggestion would be to move the ratings off the article page altogether (perhaps to the "talk" page) - again, so as not to "taint" a well-written article with low ratings, or conversely, give a poorly-sourced "stub" article the air of legitimacy with disproportionately high ratings.

I could go on and on, but these are my basic problems/suggestions regarding the tool. I realize a whole new set of conversations and changes are already in the works, but I'm burned out on the subject, so I admit I haven't taken the time to read all of the most recent threads/pages discussing the tool, but I was asked to share my opinion about my experience with the tool, so I thought I should chime in. --- Crakkerjakk (talk) 05:44, 31 October 2011 (UTC)

Thanks for the comment :). Have you read the project page (WP:AFT5) yet? If not, you'll like it; the plan is to scrap the ratings system entirely and replace it with actionable comments. Give it a read and let me know what you think. Thanks for choosing to participate :). Okeyes (WMF) (talk) 05:55, 31 October 2011 (UTC)

I glanced at the page, but thought I'd already read it when I saw the top section regarding the "current version", so I didn't check to see the section below about version 5. It sounds like a good idea to me to change to comments. However, I see there is still a rate "up" or "down" feature. I'm not exactly sure how this addresses the #1 problem I've found with the tool, which is fan/hate votes. Will the up/down ratings still be viewable on the main article page? If so, I don't really see how that's much improvement over the current 5-star ratings system. --- Crakkerjakk (talk) 06:34, 31 October 2011 (UTC)

Do you mean the rating system for comments on File:Article-Feedback-Page-Simple-Wireframe-V5-10-26.png? Okeyes (WMF) (talk) 14:37, 31 October 2011 (UTC)

Crakkerjakk, I don't see any 'rate "up" or "down" feature'. I see (in option #1) a question, "Did you find what you were looking for?" (and a free-form text box), but "Did you find what you were looking for?" is not an assessment of overall article quality. The answer could be "yes" for a really lousy article, if it happens to contain the one thing the person wanted, and it could be "no" for a great article if it happens not to contain some obscure or irrelevant detail.

Option #2 gives only the free-form text box. There is no "up" or "down" or "Is this a good article" or anything like that in either of the proposed versions. Can you be more specific about what looked like an "up" or "down" feature to you? WhatamIdoing (talk) 23:05, 1 November 2011 (UTC)

• We do actually have plans to have such a system, but for rating the comments themselves; it's not in the wireframe yet. Okeyes (WMF) (talk) 05:51, 2 November 2011 (UTC)

Invitation to edit

Earlier this year, I and others ran a trial. We put an invitation to edit at the top of 19 medical articles for a month. Readers who clicked the invitation were taken to a mini how-to-edit tutorial. The intention was to compare relevant editing stats for that month with those for the same month in previous years, and, finally, compare those results with the same results from a similar sample without the template. It had the support of many members of WikiProject Medicine and regular editors at each of the trial articles.

The trial was constantly harassed by editors objecting to its existence, the template was frequently removed from trial articles and, mid-trial, the template was proposed for deletion.

It was meant to be a practice run for a much bigger sample. But, after the experience of defending the templates on 19 articles, and arguing the merits of the trial with a seemingly endless stream of editors on that tiny trial, I couldn't face attempting to persuade the community to back a larger trial, and the nonsense, literally, that running and defending such a trial would entail. I was also profoundly dispirited by the mini-tutorial: in the end, I believe it had all the essentials for a first-time medical editor to do a reasonable job, but it was still far too long. That is, as was mentioned in your recent office hours discussion, no matter how slick the invitation is, the UI (for editing and especially creating citations) is a huge impediment to new editors. I'm ashamed to say I just walked away from the trial, and went back to editing. I have no idea whether those stats would reveal anything, the number is so low.

I just thought I'd point interested editors to this experience, in case any ideas canvassed during its development and trial stages (WT:ITE) are of relevance to this wonderful enterprise. --Anthonyhcole (talk) 05:56, 31 October 2011 (UTC)

I'll give it a looksee :). I agree that the barrier is too high - for some wider data, 17 percent of the people who see the "you know you can edit, right?" sign on the current AFT version try to edit. That's fantastic. We're getting 30-40,000 comments *a day*; imagine 17 percent of those people becoming editors! Unfortunately, out of the 17 percent of people who try to edit, 2 percent complete an action. It's just too confusing for a newbie. Hopefully the visual editor will improve things, although I think we need to improve the guidance the AFT gives to new editors in the meantime. I imagine that's something we'll look at slightly later on in the development process. Okeyes (WMF) (talk) 14:41, 31 October 2011 (UTC)

Yes. That 27% : 2% speaks volumes about accessibility. On that, is a simple citation generator part of the brief for the visual editor, Okeyes? --Anthonyhcole (talk) 14:56, 31 October 2011 (UTC)

I'll add it to "list of things to ask the devs" and find out for you :). Okeyes (WMF) (talk) 15:01, 31 October 2011 (UTC)

Okay; the response, to rephrase - the initial version won't contain a citation generator, because that's hella-complex. However, citations are displayed in templates, and templates will be laid out in a "fill in this form" fashion - so there will be a generator eventually in that form. Okeyes (WMF) (talk) 18:04, 2 November 2011 (UTC)

We need to push ahead with this trial and crunch the numbers. Raising money alone is not going to keep Wikipedia alive and vibrant. We need to raise editors... I have tried a number of measures as listed here [1] with limited success. --Doc James (talk · contribs · email) 09:38, 24 November 2011 (UTC)

The AFT trial? We be doing so; I have a big message and newsletter to type up now and post here now, which should say where we are, what we're doing next, so on, so forth. Okeyes (WMF) (talk) 13:01, 24 November 2011 (UTC)

Central point

I think it would be good to have some easy way to get to all comments. Imagine you don't know what to do on WIkipedia (really hard to imagine :D ) and so you will go and get to comments that haven't been used yet. It could be either some central repository with all comments posted there or a special link. That central page was my first idea but now it seems to big, the link version would be better.

This link would work as "Random article with comments none worked up yet". This way both comments from well-know pages (where will be lots of them) to outer not-so-usually-visited articles (where is none to use them) could be worked up.

I am not sure where this link should be placed here on enwiki. Maybe to Help or in that template in Recent Changes. Or (which sounds better) to CleanUp project or something like that. --19:34, 31 October 2011 (UTC), Utar (talk)

That's a really interesting idea! I'll throw it at the devs (just writing my roundup of suggestions now). Special:ArticleFeedback or something :). Okeyes (WMF) (talk) 19:41, 31 October 2011 (UTC)

I like this idea too! I would like to propose a modified version of it: A "best-comment-dispenser" where editors can request the "most upvoted comment (on the entire wiki) not marked as done". When a comment is dispensed to a user, it is assumed the user is working on the article and no other comments from that article is dispensed to anyone else. If the comment is not marked as "done" within 24h(?), comments from the article can again be dispensed to someone else who requests a comment to work on. This should reduce edit conflicts between users who are trying to take care of the same article all at once, but at the same time give priority to the readers' most pressing issues. --Bensin (talk) 03:46, 4 November 2011 (UTC)

That sounds really great. --09:46, 4 November 2011 (UTC), Utar (talk)

I think that'd be presumed with a central thread - I'll poke the devs with this idea. Okeyes (WMF) (talk) 17:38, 4 November 2011 (UTC)

Office Hours

Hey, everyone!

So, first off, thanks for all your great suggestions so far :). I should have responses from the devs by tomorrow morning. Second, although I've tried to notify everyone, just in case anyone got missed - Brandon Harris, Howie Fung, Dario Taraborelli, Fabrice Florin and I will be holding an Office Hours session on Thursday at 24:00 UTC. I'd love if you lot could come along and get some direct, realtime feedback from the devs - as with the last occasion, I'll stick around afterwards and chat :). It'd be great to see you there! Okeyes (WMF) (talk) 19:39, 1 November 2011 (UTC)

UI snafu

I note that the tool collects data as red stars and displays ratings as blue squares. I'd dearly like to see the log of whichever design discussion came up with that decision. --Tagishsimon (talk) 03:11, 2 November 2011 (UTC)

That I'm not sure of; however, if you look at the new version that feature isn't present :). Okeyes (WMF) (talk) 06:03, 2 November 2011 (UTC)

Ah, I almost said something about that (hoping it would not be red as shown). Thanks, Oliver. Tony (talk) 13:22, 4 November 2011 (UTC)

Notifications

Is there any way i can be kept up to date on discussions, office hours requests etc, involving the article feedback tool? I'd like to recieve messages like this in future. I looked over the relevant pages, but couldn't find a place to add my name to a notification list. The Cavalry (Message me) 22:23, 3 November 2011 (UTC)

I'm compiling a list now - I'll post it on the AFT5 main page later, and anyone who isn't on it can add themselves. I'll be providing notifications to people who have commented here, and people who are wanting to be informed. You were not notified yourself because you haven't (in AFT-related discussions) appeared or shown any interest in the past. Okeyes (WMF) (talk) 22:44, 3 November 2011 (UTC)

Thanks :-) The Cavalry (Message me) 13:16, 4 November 2011 (UTC)

Accessibility

Oliver et al., I hope you don't mind: I asked User:Graham87 for his feedback about accessibility. Tony (talk) 13:29, 4 November 2011 (UTC)

I've been asked to comment here about accessibility with screen readers. While it's hard for me to comment on the accessibility of something that doesn't exist yet, I'd say that in general, if standard HTML forms are used with labels attached to every field, the tool should be accessible. I'll let you know if I find a problem with the accessibility of any prototype/s once they are released. Graham87 13:42, 4 November 2011 (UTC)

Brilliant! Tony, thanks for bringing Graham into this - Graham, thanks for the pointer :). I'm pretty sure the plan is to use pure HTML, and we normally make such labels necessary - but I'll double-check. Okeyes (WMF) (talk) 17:18, 4 November 2011 (UTC)

Resolutions/Phases of comments

Undone and done are minimum. Bensin proposed merged resolution. But why not to use more of them as in Trac system? ("~" tilde means a response)

1. New/Undone - new comments start here ("The article needs more links.")
2. Merged/Duplication - a comment was closed because there is another one which pleades for the same thing; that one would be upvoted because more people want that thing ("It should have more links.")
3. Done/Fixed/Completed - job was done (~ "Links added.")
4. WorksForMe - comments pleaded about something that editor can't see there f.e. browser dependant errors ("That table is so messed." ~ "Seems good for me. Which browser are you using?")
5. Invalid - comments about the subject of article not about article itself ("I hate this actor.")
6. WontFix/NotHere - comments about the article but that should be somewhere else - maybe with the tool to move comments for one page to another one ("This article about X needs more complex history." ~ "Yes, but that was moved to "History of X". Comment there/I will move this comment there.") - or are just asking for something not doable/not allowed (~ No, you can't use Java here., ~ No, that photos are copyrighted.)

Is something like that being prepared or have I found the gold vein? Or you don't like it and it's only pyrite? --13:17, 5 November 2011 (UTC), Utar (talk)

A good point! I'll bring it up with the devs - I think they may not like something that detailed, because it makes the system a bit complex, but we could have "done" and "notdone because [box for additional comment]" or something. I'll find out :). Okeyes (WMF) (talk) 11:42, 6 November 2011 (UTC)

In this system, how would you respond to a comment that says something non-actionable due to vagueness, like "Incomplete article"? WhatamIdoing (talk) 05:52, 8 November 2011 (UTC)

(2, 3, 4) 5, 6 or just ask for more information. --07:15, 8 November 2011 (UTC), Utar (talk)

General comments

I was requested to comment and am happy to do so.

I think that the change to providing specific feedback is much needed and very good. I think that the "ratings" method of the previous versions does not provide good info. First, the obvious, it's just a rating rather than specific comments. But second, there is no framework/standard to interpret the meaning of any particular rating. Even if there was an attempt to develop it, I don't think it would work due to the wide variations in articles, the reader situations that they create, and the readers.

I must confess that I dislike distracting clutter on article pages, and due to it's size, position etc., I think that version 4 was certainly clutter. I found myself substantially irritated by it/ actively disliking it. Suggest (only) a brief but clear and obvious few words to click on on the article page, which would take the reader to a screen which shows the detail.

Sincerely, North8000 (talk) 21:35, 5 November 2011 (UTC)

I think the plan is definitely to look at trying to make the form as compact as possible :). Utar's idea above - to have them as drop-down links on section headings - would mean that the forms aren't visible, and the devs are actively considering that. Thanks for the comment :). Okeyes (WMF) (talk) 11:41, 6 November 2011 (UTC)

New developments, new opinions needed

Note: ideas from this section have been accepted by the developers

Hey guys! Okay, first off, thanks to everyone for their work so far. People have come up with a lot of useful suggestions, all of which have been happily ferried to the devs. If you look at the top of the page you'll see a new box with a load of ideas in - this is the "status box", showing what happened to community ideas the devs accepted. Each of the ideas are going to be looked at in more detail and hopefully built into the programming, either in phase 1 (normalspeak = between now and January) or phase 2 (January to March-ish). I have no doubt the box will expand as we go on - all the suggestions since I formatted it have been just as awesome :).

In the meantime, there's one open issue which (imo) is of paramount importance - that's this: what level of "authority" should be necessary to access and use the feedback page itself, particularly the up- and down-voting elements? Fabrice had a meeting last week, and they drew up some basic ideas. One of these (and I think, one of the most important bits of the entire system) is that only registered users should be allowed to up- and down-vote questions, to avoid the possibility of gaming the system. Personally, I think IP addresses should be allowed to vote - if we limit it to just registered users we're limiting it to (mostly) just editors, who may not be interested in the system, and excluding most readers. Any clearly gamed posts can just be hidden, and I don't think they're likely to be that big a problem. Fabrice has asked me to throw this out and gather editorial perspectives and opinions on this - what do you lot think? Okeyes (WMF) (talk) 14:27, 6 November 2011 (UTC)

• I think that everyone should have access. The majority of readers are not registered users. Plus, as long as we keep this feedback page as being for useful feedback (and not degenerate into a competition via things hinging on the result) there shouldn't be too much drama. I mean, otherwise we would be saying that the reader is allowed to edit the article but not allowed to give feedback on it ! North8000 (talk) 14:44, 6 November 2011 (UTC)

• Maybe all should be able to tweak the comments. Because if we limit it to registered ones there would be not a few readers who will make accounts just to be able to vote here. If it would be restricted to logged users than not to all but to autoconfirmed ones (or how they are here called users with at least 10-20 edits and 3 days, something like that). That way you either make it for all users, both readers and editors, or just for autoconfirmed ones (than those who would like to made an account just for voting will have to make some edits, hopefully good ones). --15:59, 6 November 2011 (UTC), Utar (talk)
  • By "tweak" do you mean "up and down-voting"? That's what's being discussed (there are other permission levels which I'll bring up, but I thought the access/no access issue for IP addresses would be a crucial and contentious one). Okeyes (WMF) (talk) 16:03, 6 November 2011 (UTC)
    • Yes. I have chosen a bad word. It is more about enhancing than changing/modifing/voting for which was what I wanted. --20:43, 6 November 2011 (UTC), Utar (talk)
      • Gotcha :). Okeyes (WMF) (talk) 21:11, 6 November 2011 (UTC)
• I would be surprised if this aspect turns out to be important. I expect that the biggest problem with the system will be that most of the comments will not be actionable -- they will be things like "I did not find the information I was looking for" or "this article is badly written" or "this article contains an error". My handling of feedback will be much more determined by how concrete it is than by how many people agree with it. Looie496 (talk) 16:39, 6 November 2011 (UTC)
• If an unregistered user can't "access" the page, then how could the reader find out whether there's been a response to his or her comment? What about the significant number of unregistered users who are regular editors?
  My initial belief is that everyone should be able to use the page, unless and until we prove that unrestricted full access isn't functional. WhatamIdoing (talk) 20:51, 8 November 2011 (UTC)

  The proposal is to let them access it, but not use the voting system (although I agree on your point). Okeyes (WMF) (talk) 21:19, 8 November 2011 (UTC)

• I still say full access to provide feedback by everyone. Same reasons apply as before the clarification. The majority of readers are not registered users. Plus, as long as we keep this feedback page as being for useful feedback (and not degenerate into a competition via things hinging on the result) there shouldn't be too much drama. I mean, otherwise we would be saying that the reader is allowed to edit the article but not allowed to give feedback on it ! Sincerely, North8000 (talk) 21:38, 8 November 2011 (UTC)
• Looie496 makes an interesting point. We might overestimate the will of readers to post and vote on comments. However, I think transparency is preferable whenever possible. All our projects is built on a foundation of transparency and it would be a little odd if this project, designed to invite to participation, did not. Should gaming become an issue I think it's better to just block those IPs. Also, there can be no +1:ing without readers seeing each other's comments, and I kinda like the idea of a really lazy reader who gets excited by contributing with constructive up-voting of comments for high-profile articles and thus helping editors identifying critical flaws. --Bensin (talk) 01:15, 9 November 2011 (UTC)
• I don't personally care much for voting about individual comments, but I understand that others might. Make it accessible to all, until and unless there's a problem that requires us to restrict its use. Graham87 01:44, 9 November 2011 (UTC)
• If it is implemented, all readers should have access to the voting feature. Perhaps voting can be disabled or restricted to autoconfirmed on articles where it is obviously being gamed, but I seriously doubt that'll be an issue on 99% or more of articles. AGF. --Anthonyhcole (talk) 07:34, 10 November 2011 (UTC)

  Okay, thanks to this feedback, it'll be accessible to IPs by default. Thanks all for participating :). Okeyes (WMF) (talk) 14:58, 10 November 2011 (UTC)

Will submitted feedback appear in a user's contributions?

Will feedback show as an edit when looking at someone's contribs? One reason for asking this is that it would be handy if, when looking at a feedback item, it were possible to click something that immediately took you to all other feedback provided from the same IP or by the same logged in user. For one thing it would be a fast way to see if someone is misusing their feedback rights in some way or, more positively, to follow a very helpful user who provides useful feedback and maybe edit articles based on their detailed and actionable concerns. --bodnotbod (talk) 15:32, 7 November 2011 (UTC)

A good question; I'll find out. One concern, I would say, is that most feedback is likely to come from IP addresses, which are mostly dynamic. As such you could end up with a lot of feedback tied together to one "identity" where in reality dozens (or hundreds) of people could've contributed it. Okeyes (WMF) (talk) 15:40, 7 November 2011 (UTC)

The answer is "we should seriously consider listing your feedback posts on My contributions for phase 2. It's an important feature, which has been requested several times. In enables people to check out each other's comments, will help tremendously with spammer control -- and also seems helpful for community building." - so in other words "yes, assuming doing so doesn't make anything melt or piss people off" ;). Okeyes (WMF) (talk) 14:59, 10 November 2011 (UTC)

Info page for users

Let me add a point that I don't think has been mentioned so far. In conjunction with the feedback system there ought to be a page that users can read explaining the purpose of the feedback system and giving hints about what sorts of feedback would be most useful. There probably should be a link to this explanatory page on the feedback form (labeled "Info" or "Help" or "About" or something). Looie496 (talk) 15:47, 7 November 2011 (UTC)

That's true; such a page already exists, actually, but it should definitely be updated to give a clearer explanation of what the form is for. Okeyes (WMF) (talk) 19:24, 8 November 2011 (UTC)

Okeyes, you have invited me to participate in these discussions, and at last I find here in this section a topic where I have something to add. Looking through much of the preceding, I think many excellent ideas and suggestions have been offered. When we get into "Info page for users", I think Wow!. I'm pretty new to Wikipedia. I've never yet created an article, but am presently working with a far more experienced editor in preparing one. I have occasionally made a few corrections or additions to existing articles, but mostly I have limited myself to either posting questions or suggestions on talkpages for articles (which generally tend to go unnoticed and unanswered); otherwise I have looked for specific editors of articles or for commenters on article talkpages, and posted on their personal talkpages. Most of the time when I have been "bold" and not asked first, I have gotten in trouble and been reverted. My trouble is that there are millions of words (I haven't counted) on Wikipedia's policies, procedures, styles, etc, that to me are an impenetrable thicket.

So now we are proposing yet another. Absolutely, an Info page for users is a necessity for feedback. But how to make it 1) findable, and 2) clear, for noobs? I certainly don't know the answer to this. Here's my own feedback: somehow, someone needs to step back and take a good hard look at all of Wikipedia's internal information on how, when, where and why, and try to construct as simple and logical a tree as is possible. As I've said on my userpage, I still consider myself to be a user much more than an editor here. Wikipedia is a wonderful encyclopedia, with an incredible amount of useful and helpful information on every conceivable topic. But beyond just passive reading, it's confusing as hell. That's my take. Milkunderwood (talk) 21:43, 8 November 2011 (UTC)

I might add that I have run across the following quotation, very obviously offered entirely in jest, but still containing a kernel of truth, because long-time editors do have a different perspective from casual users: Remember, the reader is the enemy. Milkunderwood (talk) 22:45, 8 November 2011 (UTC)

If I were a volunteer, I'd be leading a drive to fix it all up and simplify it - and when I'm a volunteer again, I hope to do so. Unfortunately most of my time these days is spent on Foundation work, and the Foundation can't directly intervene; the structure of internal content and help documentation is an area the editors traditionally have control over. Once my contract is up, hopefully I'll be able to fix things.

What's being discussed here isn't actually a new help page - it's a page explaining what the AFT is and what it's there to do. We've currently got one, linked through the existing AFT box, which is at WP:AFT. I think Louie is just talking about improving and updating that :). I care deeply about help pages themselves, though, and if you've got any ideas to help improve things which I can assist you with, in a foundation capacity, do drop me a note. Okeyes (WMF) (talk) 22:47, 8 November 2011 (UTC)

This doesn't really go to Looie's point, but it addresses Milkunderwood's, I think. I'd like to see this tool include an invitation to edit. That's been mooted but I'm not sure where it's at at the moment. Rather than the invitation taking the reader directly to the article's edit box, I'd like the reader to be presented with a simple beginners' tutorial first. Something like this. It could drop down from the AFT when the reader clicks the invitation.

Getting started

• Click any of the [edit] links within the article and type into the box near the bottom.
• When you're finished, briefly describe what you've done in the "Edit summary" box and click "Save page".

Be guided by these important principles

No original research

• Wikipedia does not publish original thought: all material must be attributable to a reliable, published source. Articles may not contain new analysis or synthesis not fond in reliable published sources.

Reliable sources

• Ideal sources for Wikipedia articles include scientific papers, books and newspapers.

Use your own words

• Accurately reflect sources but do not copy chunks of copyrighted text. In terms of expression, your contribution should bear little resemblance to your source.

Citing sources

• Content not accompanied by an inline "citation" (a footnote marker like this^[14] linked to a book, article, web page, etc. listed at the bottom of the page, that supports the claim, might be deleted.
• Copy the style of other references in the article and type your reference between <ref> and </ref>. When you save your work, the footnote marker^[14] will appear in the text where you typed your reference, and the full reference will appear at the bottom of the article in the list of references.

I believe this tool is the ideal place to introduce interested readers to the basics, and a well-thought-through mini-tutorial will increase the frequency, quality and success of first attempts at editing. --Anthonyhcole (talk) 08:11, 10 November 2011 (UTC)

Ditto! That's the sort of thing I'd like to see too - speaking personally, working on the various calls to action and prompts and what-the-reader-is-presented-with-when-they-indicate-they-want-to-edit is the bit of the design process I'm most looking forward to. I think that reader engagement is important, but that if the AFT is going to live up to the second part of its intended purpose (driving readers towards becoming editors) we also need to focus on improving the info we give to those people who indicate they want to participate. I'll add this to the list of community suggestions I update every day and get dev feedback on - although this'd probably be "phase 3" (in the English language you and I know and love - "we'll work on it in the new year after we've checked that the new designs don't melt and make us jettison the warp core"). Okeyes (WMF) (talk) 14:41, 10 November 2011 (UTC)

You need to test multiple approaches to this to find what works best, also I'd drop the bit about stuff without an inline cite may be deleted on sight. That's really only for stuff that is both contentious and about living people. Anything else people can {{fact}} tag it if they think it is implausible. But no-one should be reverting newbie edits simply because they are unsourced. 11:20, 11 November 2011 (UTC)

Agreed. --Anthonyhcole (talk) 10:12, 24 November 2011 (UTC)

Office Hours

Hey guys; we're holding another office hours session on Thursday at 19:00 UTC (you can check when that is for you here). I hope to see you all there :). If you can't make it, I will be posting the logs as usual. Okeyes (WMF) (talk) 23:02, 8 November 2011 (UTC)

Speed/bandwidth

Hi. Have we checked this out to see how much it slows things down for people on dialup and other slow connections? One of the perennial problems of new developments is that they are run and tested on setups with fast connections and they neglect the large parts of the global south where connection speeds are way less than in San Francisco. I should elaborate that this is me whinging against programmers everywhere, and I've no reason to suspect that our devs are any more likely to make this mistake than any other group of first world programmers. ϢereSpielChequers 17:09, 9 November 2011 (UTC)

I'll find out :). Okeyes (WMF) (talk) 21:34, 9 November 2011 (UTC)

Ta. ϢereSpielChequers 00:21, 10 November 2011 (UTC)

Yes, the system is being explicitly tested on and for low-bandwidth systems. Okeyes (WMF) (talk) 14:20, 16 November 2011 (UTC)

KPIs

My principal concern with this feature is the risk that it will compound our existing problem of a declining editorship by exacerbating the trend from improving the pedia to that of tagging articles in the hope that others will improve them. If like me you buy the theory that the rise of templating is a principal cause of the decline in the editing community then something that furthers that trend is inherently undesirable. So I'd like to suggest that we set some Key Performance Indicators that give us meaningful measurements of the impact of this software. The most logical ones to my mind are the number of new editors/amount of editing in mainspace, (the raw count of new editors includes templating and some people who never make it out of userspace). The difficult thing is knowing what the count would otherwise be as it is declining anyway, and if this software turns out to be neutral rather than harmful it would be unfair to criticise it because the number of new editors continued to decline at the same rate. For the previous version I suggested that we do some A/B testing - put the feature on a hundred thousand articles, pick another 100,000 as a control sample and see how many new editors each group attracted, that wasn't done for the first version but I'd appreciate it if we did it for the second version. Obviously we'd need a way to ignore edits that just added templates.

Ideally I'd like to see the testing done prior to implementation and with a commitment that we wouldn't roll out until we had a system that didn't just work technically, but from the testing was likely to be neutral or only marginally negative to the pedia. ϢereSpielChequers 00:21, 10 November 2011 (UTC)

The AFT is primarily aimed at readers, not editors; it's rather difficult to test impact on editor status directly, as you say, because numbers are already declining. How can we establish a causal link between a decline in numbers and a single event when there are so many variables? There is already a plan for A/B testing (as shown on the AFT5 page, and associated and linked features pages). There's definitely going to be tracking of people who start editing (or try to edit) through the calls to action, but I believe what you're suggesting is instead testing how many people start editing despite the software rather than through it. How do you even test such a thing? We could have less IP addresses editing on the pages where the new AFT feature is, okay. Are the new editors on the more highly edited pages instead just returning users with dynamic IP addresses? Are, because of dynamic IP addresses, some editors appearing to be "old" ones when they are in fact new?

To illustrate: lets say we have Obama under the new template, and Bieber under the old one.

1. I go to edit the Justin Bieber page as an IP. I have edited wikipedia many times before, but my IP address has just changed (most ISPs use dynamic addresses). As such, I'm counted as a new editor when I am, in fact, a relatively old one.
2. I go to edit the Obama article. I've never edited before, but I have a dynamic IP address. This means that my edits appear to come from an address with hundreds of contributions. As such, I'm not counted as a new editor even though I am one.

And that's without getting into current affairs. What does or doesn't happen in the world of media reporting has an impact on the views on our most high-profile pages, which are traditionally celebrity or popular culture entries. By definition, any impact on viewing figures is going to impact on the number of potential editors and the number of AFT submissions, and AFT submissions are fairly heavily weighted towards articles with high viewing figures. Any big media stories could cause an unexpected or disparate spike in stats in either the control group or the group of articles with the new version on.

So, I think measuring this will be fairly difficult, and subject to a lot of variables. But I'll pass it on to Dario and Fabrice and see if they think it's workable. Okeyes (WMF) (talk) 01:38, 10 November 2011 (UTC)

I appreciate that there are complications to this, hence my suggestion of a 100,000 sample. Trending articles are an issue, but they could of course be filtered out by ignoring the articles with the largest change in the number of edits by longterm editors. As for the bit about it being aimed at readers not editors and it being difficult to test impact on editing directly, "About 17 percent of users presented with the invitation to "Edit this page" after completing their rating (aka "edit call to action") tried to edit, and 2.7 percent completed their edit." And "The Article Feedback Tool has therefore succeeded, to some degree, in promoting both quality assessment and reader engagement." in my view read as a claim that the feature is having a positive effect on getting readers to edit. I think that at the least we need to add the caveat of "Though the total number of new editors has continued to decline since the introduction of the AFT." An alternative way to measure things is to look at the steepness of the decline in the number of new editors and see if this has been affected by AFT. Better still test a controversial and risky piece of software like this on multiple smaller wikis before implementing it on arguably our most important one. ϢereSpielChequers 11:55, 10 November 2011 (UTC)

Your suggestion has been forwarded to the devs. Okeyes (WMF) (talk) 16:28, 11 November 2011 (UTC)

• Fabrice tells me that's already been taken into account - Dario has, apparently, also talked to you about this repeatedly. Okeyes (WMF) (talk) 14:19, 16 November 2011 (UTC)

Commentspam

One thing we have to consider (and one thing the developers have asked me to bring up - how do we deal with comment spam? My personal thinking is "if it's technically feasible, we should just plumb in the existing edit filters, abuse filter, spam blacklist and reCAPTCHA" - do people have other suggestions? Okeyes (WMF) (talk) 15:09, 10 November 2011 (UTC)

Might also be worth getting some means of proactively flagging spammers picked up by the above for admin attention, much like AIV. Rich Farmbrough, 19:52, 10 November 2011 (UTC).

Just use the code from edit/abuse filters and the blacklists. I'd highly dis-recommend using reCAPTCHA, as well, it's feedback. Admins should be able to suppress the comments, wherever they go. HurricaneFan25 19:54, 10 November 2011 (UTC)

Feedback vs. Call to Edit

I would like to maintain a distinction between the two. Feedback ought to be a very lightweight process for the reader, requiring zero knowledge of Wikipedia procedures. Editing, on the other hand, is not a lightweight process and cannot be made into one -- edits made without thought, preparation, and knowledge of procedures are likely to be reverted. I don't believe we do a service either to readers or to ourselves by giving the impression that editing is a trivial act: it is an important act that needs to be taken seriously, and we should try to convey that impression to readers. In short, I am opposed to including a call to edit as part of the feedback tool. Readers who give feedback that leads to a response will be much more encouraged to participate than readers who make an edit that is promptly reverted.

I would have much less objection to a link from the feedback tool to WP:Tutorial/Editing (although that page currently sucks -- clearly written by people with no teaching experience). Looie496 (talk) 15:16, 10 November 2011 (UTC)

Well, the calls to action are already included in the existing tool - but they appear after feedback has been submitted, not before. I'll submit your thoughts to the devs. Okeyes (WMF) (talk) 15:38, 10 November 2011 (UTC)

With respect I'd disagree with Looie. Goodfaith edits usually don't get reverted, and fixing a typo is or should be a trivially easy act. Yes I appreciate you have to pick out the word you want to change from a wall of text and code, and that needs to be made easier. But "preparation, and knowledge of procedures" are really not needed. If this system succeeds it will be because someone manages to make the call to edit effective - giving up on that and concentrating on Feedback alone means that system can only work if you also find a way to recruit the editors you need to answer the feedback requests. Remember however plush the carpet and however fresh the flowers in the display vase, any helpdesk will be judged by the helpfulness, accuracy and speed of the help it dispenses. The Article Feedback Tool needs to put more emphasis on the difficult bit, handling responses from readers and make sure that our ability to respond to the Feedback this generates is ramped up in sync with the increased workload this will generate. ϢereSpielChequers 11:11, 11 November 2011 (UTC)

WSC, I don't think I've ever indicated that there will not be attention paid to the call to action. Indeed, I've repeatedly indicated that there will be. The focus at the moment is creating a tool that doesn't make editors and readers alike want to claw their eyeballs out, which, by definition, involves working on the actual tool rather than surrounding material - that is why attention is being paid to the feedback designs. When there's a layout everyone is comfortable with that works, then there'll be a focus on what happens "after". Okeyes (WMF) (talk) 16:27, 11 November 2011 (UTC)

Perhaps not, but my remark was addressed to Looie's comment that he was "opposed to including a call to edit as part of the feedback tool". Do you regard Looie's proposal as compatible with the idea of AFT as a method to encourage readers to edit? ϢereSpielChequers 17:14, 11 November 2011 (UTC)

Ahh, I misread it. Blegh. This is what happens when you spend 12 hours a day on data analysis :P. Yes, we need to include calls to actions - if we don't, one of the tool's primary purposes is simply mooted. The focus has to be on how we phrase them and what guidance we give to new editors. Okeyes (WMF) (talk) 17:52, 11 November 2011 (UTC)

AFT and attracting new WPians

I've flicked through some of the discussion above. Exactly what are the objectives of the AFT? Has anyone produced a bulleted list? For example, is it the following?

• To produce article-specific feedback that enables editors to improve an article.
• To produce feedback that can be interpreted more broadly over whole topics, categories, and even the project as a whole.
• To produce other data that is useful for academic research on WP.
• To give readers the satisfaction of "having a say", with spill-over into the brand, reader loyalty, public attitudes.
• To create opportunities for dialogue with readers (in which case, should they be encouraged onto the talk page of the article?)
• To attract more readers into anon editing and registration.

Are there other objectives? Do these six need to be amended or reshaped, or some withdrawn? What should be their order of priority?

On the last bullet, I can't help feeling that the good start made by the Foundation's Summer Research program into looking at data on anon and newbie editing hasn't yet come to fruition. Do we yet have a way of identifying the editing patterns of newbies and anons that are associated with people who are worth the investment of allocating to a "mentor"? Real-person mentoring is clearly the most effective way to shepherd in more editors. Is the design of AFT5 relevant to such identification? Tony (talk) 11:36, 11 November 2011 (UTC)

The aims can be found here; "This version was intended to do two things: first, give the Wikipedia community a new tool for assessing article quality; second, provide a way for Wikipedia readers to get involved as editors". These intentions are the same for the new version. The point on what we do with the new editors is a good one; I assume it'd be covered later on, when we're discussing the "calls to action" and the help material we give them, which is personally the bit I'm most looking forward to. I'll poke the staffers and see if they can offer any sneak peeks on their thinking and intentions for this area :). Okeyes (WMF) (talk) 16:24, 11 November 2011 (UTC)

And re the "do we yet...", Fabrice's response is "Great question. We plan to track the quality and survival of edits made by people who first start editing through AFT5. This will provide us with some useful data on individuals whose edits were not reverted and therefore may be of sufficient quality to warrant mentoring. I am glad Tony brought up this important method for encouraging newbie editors to develop their skills, and will start keeping mentoring in mind as we plan our next steps for this project." Thanks for the idea :). Okeyes (WMF) (talk) 17:20, 29 November 2011 (UTC)

Given how few effective mentors we have available, I'm not sure that trying to point people at an individual mentor is likely to be successful. We'd be better off pointing them at the Help desk or some such general forum. WhatamIdoing (talk) 02:03, 11 December 2011 (UTC)

Office Hours

Hey guys; another office hours session, coming right up! 22:00 UTC, #wikimedia-office, tomorrow. Hope to see you all there :). Okeyes (WMF) (talk) 20:14, 17 November 2011 (UTC)

The log is here. My thoughts:

• re: this mockup. It just says "2 votes". 2 votes of what? Upvotes?
• re: Fabrice's question "would it be useful to provide a link to show who voted a feedback post up or down (or who flagged it for abuse), for transparency reasons?" It's not uncomplicated, but transparency is the rule, not the exception. Therefore I say yes.

Sorry I couldn't be there. Maybe next time. --Bensin (talk) 17:37, 23 November 2011 (UTC)

Some quick suggestions

• I made attempts to increase readership with an idea called Wikipedia:Sticky notes. At least one idea from there applies here too: After users leave feedback, that is a good time to ask them if they want to create an account (give them the WP:WHY pitch)
• Don't use option 2: comments are normally a combination of praise, suggestions, and so on. If anything, let users tag the comment box as including any of the above (e.g. by clicking checkboxes).

Just some ideas - I don't check here too often -Tesseract²(talk) 00:14, 22 November 2011 (UTC)

Totally agreed on both points :). Personally, I think option 2 is likely to cause more problems than it solves - we should keep it straightforward, with as few diverging paths as possible. Re creating an account, I'm pretty sure that's already included, but I'll check in just to make sure. Okeyes (WMF) (talk) 17:40, 22 November 2011 (UTC)

Yup, Fabrice says the calls to action (or some of them) prompt readers into creating an account to contribute :). Okeyes (WMF) (talk) 17:20, 29 November 2011 (UTC)

A vote for Option 1 and a question

I am glad that you are dispensing with the system of four ratings. Many readers clearly did not understand them - I have seen stubs with no references given ratings of 5 for trustworthiness. Also, the questions were slanted towards the sort of thing that editors think about and discuss anyway.

I like the idea of focussing on text suggestions. I had no idea what to do with the numerical ratings, but I would like to know what changes to content readers want. Therefore, I prefer Option 1 as modified by suggestions above: "How can we improve this article?" It needs to be as simple as possible - after all, this box is for people who haven't noticed the Discussion tab.

That leads me to my question: Is there going to be a comparative analysis of the contributions on the talk page and those on the feedback page? RockMagnetist (talk) 18:08, 23 November 2011 (UTC)

An excellent question; I'll find out. I know there is quite an extensive set of a/b tests going on, and that this covers use of the edit button as opposed to the feedback page, but I don't know if it looks at talkpage/feedback page. It really should; that's a great way of working out if we're attracting new people, or just giving the old people somewhere easier to go. I'll drop Dario a line, then get back to you. Okeyes (WMF) (talk) 00:45, 24 November 2011 (UTC)

Response is "we hadn't planned it, but that's a great thing to find out - we'll stick it in the testing plans as soon as the tool is mature enough to roll out and gather some data" :). Okeyes (WMF) (talk) 17:18, 29 November 2011 (UTC)

Newsletter II - tests, prototypes and office hours

Hey, all! A quick update on how version 5 of the Article Feedback Tool is developing.

So, we're just wrapping up the first round of user contributions. A big thank you to everyone who has contributed ideas (a full list of which can be found at the top of the page); thanks almost entirely to contributions by editors, the tool looks totally different to how it did two months ago when we were starting out. Big ideas that have made it in include a comment voting system, courtesy of User:Bensin, an idea for a more available way of deploying the feedback box, suggested by User:Utar, and the eventual integration of both oversight and the existing spam filtering tools into the new version, courtesy of..well, everyone, really :).

For now, the devs are building the first prototypes, and all the features specifications have been finalised. That doesn't mean you can't help out, however; we'll have a big pile of shiny prototypes to play around with quite soon. If you're interested in testing those, we'll be unveiling it all at this week's office hours session, which will be held on Friday 2 December at 19:00 UTC. If you can't make it, just sign up here. After that, we have a glorious round of testing to undertake; we'll be finding out what form works the best, what wording works the best, and pretty much everything else under the sun. As part of that, we need editors - people who know just what to look for - to review some sample reader comments, and make calls on which ones are useful, which ones are spam, so on and so forth. If that's something you'd be interested in doing, drop an email to okeyeswikimedia.org.

Thanks to everyone for their contributions so far. We're making good headway, and moving forward pretty quickly :). Okeyes (WMF) (talk) 16:33, 29 November 2011 (UTC)

Sign up for prototype testing

1. HurricaneFan25 16:40, 29 November 2011 (UTC)
2. Looie496 (talk) 16:35, 5 December 2011 (UTC)

Editing previous feedback

On mw:Article feedback/Version 5/Feature Requirements it is said that "Unlike with the AFT v4 form, users will not be able to edit previous feedback", but I think this may be problematic, since people may want to fix a typo in their comments. See e.g. bugzilla:18379. Helder 14:38, 5 December 2011 (UTC)

That's a point. We've finished the features requirements for 1.0, but I'll see what the response is. Okeyes (WMF) (talk) 08:02, 6 December 2011 (UTC)

Version 5 Test Pages

Do we know which articles will get the first deployments of the new feedback tool? Thanks, epicAdam^(talk) 03:37, 9 December 2011 (UTC)

We're about to get the manual list. It'll be 30,000 pages - less than 1 percent of all articles - just for the duration of prototype testing and research :). They'll all be in a special category; I'll let people know when it is available. Okeyes (WMF) (talk) 12:36, 9 December 2011 (UTC)

Great, thanks. If I may make a suggestion, the selection criteria should favor high-traffic, semi-protected pages. I am sure that many IP editors have good thoughts on how to improve articles, but are unable or unsure how to do so. People are used to providing comments at the end of an article (like on news websites) and I think the new Feedback Tool may be just the thing to get people engaged without inviting vandalism. Best, epicAdam^(talk) 16:41, 9 December 2011 (UTC)

Info page redux

Apologies if this is information I should have been able to find, but where is the question-symbol icon on the upper right of the form going to take a user? Is that page ready to roll? I might be able to help with it if it needs work. Looie496 (talk) 17:36, 9 December 2011 (UTC)

• It'll eventually take a user to Wikipedia:Article Feedback Tool/Version 5/Help, which is being worked on now :). Okeyes (WMF) (talk) 19:47, 9 December 2011 (UTC)

Cool, thanks. Looie496 (talk) 20:21, 9 December 2011 (UTC)

Prototype testing, phase 1.0

I ran through the Phase 1.0 tests, exercising every major code pathway I could identify. Here are the issues I spotted:

• In option 1, the comment box needs some sort of label or hint -- the lack of one immediately jumped out at me.

• In all options, clicking the Edit link on the Call to Action takes me to a "view source" page along with an explanation that the page is protected and I need an account and four days of editing to do anything. The messages that I see also have some obvious wikisyntax problems, if it matters.

• I believe the feedback box should have a dismiss button even prior to feedback having been submitted, so that readers who don't want to see it can get rid of it.

• Once the "You can edit" message appears, clicking on the question mark icon seems to have no effect.

I know how to use Bugzilla and will be happy to file reports, but thought it would be a good idea to mention these issues here first so as to avoid cluttering the database with things that would turn out to be NOTABUGs. Looie496 (talk) 16:42, 10 December 2011 (UTC)

Those all sound like bugs to me! Just submit them; we can downgrade if it turns out it's not actually a problem. I particularly like the dismiss button idea, although it may not get included in 1.0. Okeyes (WMF) (talk) 23:47, 10 December 2011 (UTC)

Done -- submitted as bugs 32963, 32964, 32965, and 32966. Looie496 (talk) 17:29, 11 December 2011 (UTC)

General comments

I haven't read everything here, so there may be some overlap. Generally I think the focus should be on the reader who comes to the article for information, and not the knowledgeable reader/potential editor. Feedback responses have to be separated between those from experienced readers and editors (who are there to evaluate the content, style, organization etc.) and those from readers seeking information on a subject with which they are completely unfamiliar. In my view the purpose of an encyclopedia is not as a repository of information, it is as a communication means. When information at a suitable level for the information seeker can not be located easily, the tool fails. So here is what I think the feedback section should look like (I'm not going to take the time to think this through thoroughly because, given that most contributors are thinking along other lines, it is unlikely that it will be taken seriously.):

• What is your primary interest in this article?

 1) I know a lot about the subject and I'm here to judge how well the information is presented.
 5) I know nothing about the subject and I'm here to learn the basics.

• What is your level of expertise on the subject of this article?

 1) I know most of the material presented and am formally trained in the subject.
 5) I know almost nothing about the subject.

• Is the presentation of the article's information organized in a way that let's you find what you are interested in?

 1) The information is organized logically and I was able to find what I was looking for very quickly.
 5) The information is illogically organized and I was not able to find what I had hoped to find.

• Relative to your level of understanding of the material, what level of difficulty did you have in understanding the article's content.

 1) The content was too difficult for me to understand and the links to more basic material were not helpful.
 5) I was able to understand the content easily and do not feel the article should be expanded to include more difficult material.

• Relative to other sources you may be using to understand the article's material, is the writing style of this article comparably precise, logical and engaging?

 1) The material is presented in a manner that I find so poor that it distracted me from concentrating on the article's content.
 5) I found the material's presentation to be everything I could have hoped for.

• How old are you?

 1) I'm 13 years old, or younger.
 2) I am between 14 years old and 17 years old.
 3) I am between 18 years old and 22 years old.
 4) I am between 23 years old and 30 years old.
 5) I am 31 years old or older.

More could be added at the risk of becoming tiresome. Perhaps the better approach would be to allow the reader to zero in on perceived difficulties with the article using a comment box. 74.89.141.180 (talk) 05:43, 11 December 2011 (UTC)

My personal view is that we should avoid any clutter that isn't absolutely necessary, and it doesn't seem to me that the answers to most of those questions are directly useful for improving an article. Looie496 (talk) 17:34, 11 December 2011 (UTC)

I disagree; I think they could be useful. The risk is that the usefulness will not outweigh the increase in how difficult it makes the questionnaire to answer. My gut says we'll get more feedback (and more useful feedback) from making it as easy as possible. Still, we'll see how it works in practise, and look at experimenting later; thanks for the suggestions :). Okeyes (WMF) (talk) 13:44, 12 December 2011 (UTC)

Well, thanks for the tepid support, I guess. I just don't see how a survey/feedback mechanism that doesn't distinguish between an inner city 13 year old student seeker of basic information and a college professor seeking/intending to edit content can possibly provide useful reliable direction for improvement. Personally, I wouldn't waste two minutes trying to guess what to do with the limited information available from simple dumbed down oversimplified feedback. Think of it this way: the more opportunity you give interpreters of the results to misinterpreting the collected responses, the more opportunity you'll give to editors who'll seek justification in the responses for their POV. I'd like to see us move away from the zoo. blackcloak (talk) 05:56, 12 January 2012 (UTC)

That's true, but "editors soliciting comments to maintain a POV" would be, I would hope, an edge case. Detailed surveys that substantially drive down the number of comments would be universal ;p. Okeyes (WMF) (talk) 11:52, 12 January 2012 (UTC)

Nice goin'

I'm a sparodic visitor here. The idea of specific comments vs. ratings is very good.

One likely pitfall to be careful of. The comment listing will be in the "hot seat" on contentious articles. To whatever extent the comments can be modified (promoting certain comments to more prominent, finding wiki-lawyer excuses to suppress comments that oppose their view) by the combatants, they will. North8000 (talk) 11:08, 12 December 2011 (UTC)

This is true; that's unfortunately always going to be the case. I think if we have a choice between useless-feedback-and-no-drama and useful-feedback-and-some-drama, we should always go for the second, unless the drama outweighs the useful feedback :). During the testing phase we will be keeping an eye on things like this, and I'd really like to do some later experiments on what happens with the feedback. Okeyes (WMF) (talk) 13:36, 12 December 2011 (UTC)

I was focused more on the regular editors at a contentious article messing with the feedback to whatever extent they are able to. Possibly you could say that any moving-into-prominence or deletion must be done by uninvolved experienced editors or admins. North8000 (talk) 13:47, 12 December 2011 (UTC)

It's not really within our right to set on-wiki policy. Can you give me an example of the sort of situation you were thinking of? Okeyes (WMF) (talk) 16:06, 12 December 2011 (UTC)

Scheduling

I'm a bit confused. I got an email a couple of days ago that I thought indicated that Phase 1.0 had begun, and so I went through a series of tests as I described above, and filed bug reports. But now I see that Fabrice has just altered the schedule to state that Phase 1.0 will begin on Dec. 14. Could we please get some clarity on when we should be doing things? Regards, Looie496 (talk) 16:45, 12 December 2011 (UTC)

Yeah, sorry :(. Basically, we're doing pre-testing up until the 14th; users who have been engaged in the process and have actively volunteered are given a link to the prototype designs to play around with (this is what happened with you, and thank you for volunteering to work on it!). The idea is if there are any bugs, we can find them by taking a middle ground between "testing everything ourself" (slow, doesn't allow popular participation) and "releasing the code on to every page (fast, does allow popular participation, also makes every bug more of an issue because more people are being frustrated by it). So, public testing will start on the 14th. You're just in the beta group, as it were :). Okeyes (WMF) (talk) 00:38, 13 December 2011 (UTC)

Okay, I get it now, thanks. Looie496 (talk) 03:42, 13 December 2011 (UTC)

Accessibility redux

I've made some comments about the accessibility of the new AFT prototype at bug 33081. I'll make a note of the current test period at Wikipedia talk:Manual of Style/Accessibility. Apart from the accessibility issues, it sounds great to me! Graham87 06:48, 14 December 2011 (UTC)

Thanks for letting us know! I did bring up accessibility - it's possible it got partly lost in the shuffle, what with all the tweaks and changes. Okeyes (WMF) (talk) 10:39, 14 December 2011 (UTC)

Feedback on stubs

I'm wondering if it's really useful to have the feedback tool put on articles marked as stubs. What feedback could possibly be given that we don't know already? (not enough references; article too short; article doesn't discuss this aspect; etc.) Sasata (talk) 02:11, 15 December 2011 (UTC)

Well, there are some stubs that I might actually be motivated to improve if I knew that substantial numbers of readers actually care about them. Looie496 (talk) 03:08, 15 December 2011 (UTC)

Agreed! Okeyes (WMF) (talk) 13:14, 15 December 2011 (UTC)

Stub template is usually not telling you how to enhance that article, which way to go first (but it could be for example on talk page). You will know what people would like to find there when you get feedbacked stubs.

And not only for stubs: with feedback you can find that article has bad name, is showing only one point of view, there are other meanings of its name so that disambiguation should be made and more and more. --09:49, 17 December 2011 (UTC), Utar (talk)

And why could neither of these functions be performed by posting to the talk page? Is there any objection to me removing the assessment category from low-traffic stubs? Sasata (talk) 14:36, 20 December 2011 (UTC)

Time interval

It would be very useful for editors to be able to find out when were the assessments made - which revision of the article was actually rated. --Eleassar ^{my talk} 13:37, 17 December 2011 (UTC)

Agreed! The plan is currently to include a link to the diff of what the article looked like when the comment was made :). Another user suggested it, actually - editor-submitted ideas like this one have made a real difference as to how the tool is approached. Okeyes (WMF) (talk) 14:27, 17 December 2011 (UTC)

Thanks for the information! Wish you a lot of success. --Eleassar ^{my talk} 15:11, 17 December 2011 (UTC)

Will this appear on our watchlists

Will this appear on my watchlist? If anything can be added that might need the attention of an Administrator, it absolutely must appear on our watchlists (BLP issues alone make this vital). Dougweller (talk) 07:05, 20 December 2011 (UTC)

What do you mean by "appear on my watchlist" - the feedback dashboard, so you get a poke every time a new comment is added? I'll check and find out, although I think the answer may be "no". It's not actually as vital as you might think; we're still discussing the access issues (I.e., for example, who gets access to the "hide" function) and depending on what the community say may open the tool up to say, all rollbackers, or all autoconfirmed users. Okeyes (WMF) (talk) 10:08, 20 December 2011 (UTC)

Can you then assure me that BLP violations will not be possible using this tool? Dougweller (talk) 10:29, 20 December 2011 (UTC)

Err. No, I can't; BLP violations are possible as long as we maintain the existence of an "edit" button. What I can tell you is that such violations will be pretty easy to clear up, with both a "hide" feature and the integration of the full oversight tool. Okeyes (WMF) (talk) 16:03, 20 December 2011 (UTC)

They will be relatively easy to clean up so long as they are on someone's watchlist. They will still require extra work, and if they don't appear on the article's watchlist BLP violations may be difficult to detect. Yes, BLP violations are possible through the edit problem, but this is another and easier way to add BLP violating material to articles and as an easier way needs to be very easy to detect and handle without cutting into editors' time appreciably. Dougweller (talk) 18:45, 20 December 2011 (UTC)

I'm with Doug on this one: if these don't appear on watchlists, how the dickens are we going to spot the vandalisms, the BLP violations, the libels and general destructiveness which is already the bane of our open-source model? "Easy to clear up" is not an adequate excuse for permitting all sorts of vileness to flow from the Magic Firehose of Sewage as it already does on a daily basis. --Orange Mike | Talk 20:09, 20 December 2011 (UTC)

The MoodBar is also not watchlisted; is that similarly deluged? And to flip your own question round at you - if these don't appear on watchlists, aren't easily found by general readers, are NOINDEXed, how is anyone else going to spot the vandalism? Much of the damage from such things comes from public visibility - hence why a defamatory segment of an article is, quite rightly, treated differently from a defamatory OTRS email - and this tool lacks it. Okeyes (WMF) (talk) 22:42, 20 December 2011 (UTC)

This isn't to say that the watchlist idea isn't a good one; I like it, albeit for other reasons - I'm just trying to say that I don't think these concerns are necessarily going to play out :). Obviously our tests should show pretty quickly if it truly does turn into a firehose of sewage, but you have to understand that we're reaching out to a rather different crowd here. I'll bring the watchlist idea up with people, but it may have to wait a couple of weeks; we're pretty focused, what with deployment yesterday, and so everything is really frantic. Okeyes (WMF) (talk) 22:46, 20 December 2011 (UTC)

Thanks. How can we see what articles are involved in this deployment and view the comments? Dougweller (talk) 07:32, 21 December 2011 (UTC)

They should be in Category:Article Feedback 5; the comments are currently not viewable or viewed, unless you want to sign up for half an hour of feedback validation (which is primarily designed to ensure that, right from the get-go, our UX minimises the amount of spam). Okeyes (WMF) (talk) 07:56, 21 December 2011 (UTC)

Ok, I've signed up. I figure if I have concerns/complaints I should also be constructive and make sure that I know enough about it to comment sensibly. I'm not sure where I'll be Tuesday - I may be travelling, so I may not be available for IRC. Thanks for letting me know about this. Dougweller (talk) 10:04, 21 December 2011 (UTC)

No problemo :). Okeyes (WMF) (talk) 13:01, 21 December 2011 (UTC)

What is meant by experienced editors?

" Experienced editors and administrators will have the option to feature posts more prominently, or hide offensive posts." What's the criteria for 'experienced editor'? Thanks. Dougweller (talk) 11:56, 20 December 2011 (UTC)

• As I said in response to your last question, that's about to be determined - at the moment we're focusing on the initial testing of the forms themselves. Okeyes (WMF) (talk) 16:02, 20 December 2011 (UTC)

Vandalism concerns

I am relatively concerned about the potential for mass vandalism using this tool. So I have several questions:

1. As an administrator, will I be able to see the IPs or usernames of the people posting comments? I will need to be able to see them to block any users who are using the AFT as a forum for vandalism.
2. Would being blocked prevent a user or IP from posting comments? If not, then how do we stop a vandal from adding silliness/obscenities/libel to comments?
3. Will we be able to lock pages to prevent them from being commented on? I can tell already that at least a couple BLPs will need to be locked to prevent "OMG _____ is so ghey!!!1". No filter can stop a determined vandal.
4. Who can remove comments?
5. Is removing comments logged? This will be necessary to prevent abuse.
6. Can comments be restored? This might be helpful in the event of accidental or inappropriate deletion.
7. Will a captcha be required? Spambots love these comment fields, and I don't want to spend my day cleaning up ten thousand ads.

Thanks! Reaper Eternal (talk) 14:00, 20 December 2011 (UTC)

• Many of those are still to be determined - we're opening up the questions to the community in a tick. I can tell you that the AbuseFilter and Spam Blacklist are both being weaved into things. Okeyes (WMF) (talk) 16:02, 20 December 2011 (UTC)

Right now, the comments aren't being displayed, just collected for purposes of analysis. The next phase of the project is manual coding of comments in order to determine the signal/noise ratio of different feedback processes and aid in an overall assessment of usefulness. This is an essential step before we sink a lot of additional effort into feedback management. So we've put a fair amount effort into making that process of initial evaluation open and understandable. See Wikipedia:Article Feedback Tool/Version 5/Feedback evaluation.

With that said, you'll find some initial thoughts on feedback management in mw:Article feedback/Version 5/Feature Requirements, including notes on blocking, deletion, etc. Definitely still some gaps, though. --Eloquence* 08:01, 21 December 2011 (UTC)

Suggestion for the asking logged-in users bit

I'm gonna preface this by saying that I'm sorry if this has already been brought up; I didn't see it mentioned as I skimmed the above comments but I could have missed it!

So. There seems to be a lot of support for not asking logged-in users to rate the page. But what about asking non-autoconfirmed users on semi- or fully-protected pages, and asking non-admins on fully protected pages? I know I didn't discover the edit request feature for a long while after I started editing Wikipedia.

Maybe the AFT could include some code that checks the page's protection level and cross-references the user's rights (autoconfirmed/admin). If the user is looking at a page they are unable to edit due to its protection level, the top of the AFT could say something along the lines of "If there's a specific edit you want made to this page, click here to submit an edit request." The "click here" could have a link much like the one you get when trying to edit a protected page; the link opens up a new section on the talk page and already has a section header with the timestamp as well as the proper edit request template and four tildes for the user's signature.

TL;DR - there should be some exceptions to the "don't-ask-logged-in-users" thing.

If you reply here, please put the following thing on my talk page: {{Talkback|Wikipedia talk:Article Feedback Tool/Version 5|ts=~~~~~}} --~~~~~. Otherwise, I can guarantee you won't get clarification or a reply or anything. I'm incredibly spacey and I need that bright yellow "new messages" bar to remind me to check back here. The template's already filled out, just hit "new section" on my talk page and paste it. I even put the signature in for ya. Thanks! — Preceding signed comment added by Cymru.lass (talk • contribs) 20:42, 22 December 2011 (UTC)

Design

The design is very "large and obnoxious"; and although it is very nicely done it needs a UX expert looking into it. Take a look at how it dominates Pretzel Pezzullo for example... I suggest the following - hide the "Send Feedback" button and the textbox initially. Just present the Yes/No question and when either one is clicked then present the further options. Also I would shrink the size of the "Did you find..." font to match article text, and de-bold the header. You shouldn't shove the thing in peoples faces *just* to try and get them to notice it :) Ideally it should be the natural progression when completing reading the article?

On that note it might be worth coding a float-box style that appears to one side once you have scrolled some percentage through the article - just a small sidebox with the Yes/No question that then expands with the feedback form. This is both common/expected behaviour in the modern web, meets good UX standards & also will probably see a much better response ratio (because, UX 101, it "appears" and thus prompts the user, rather than gets shoved at the bottom :)) --Errant ^(chat!) 14:00, 31 December 2011 (UTC)

Different placements are going to be tested, including a "tab" on the side of the screen that unfolds into the feedback form. On the UX front, we had Brandon give it a look; he's happy with the design (except the bits he really, really dislikes, which are corrections to things he really dislikes, which he designed. Some day he'll come up with the perfect shade of blue, I'm sure ;p). Okeyes (WMF) (talk) 17:22, 31 December 2011 (UTC)

I think reducing the size should be given big consideration - it is something of a problem in that it dominates the screen :) --Errant ^(chat!) 17:28, 31 December 2011 (UTC)

We can totally mess around with it; it may be that the "tabbed" version gets far more hits, in which case it'll be less of a priority :). Okeyes (WMF) (talk) 21:02, 31 December 2011 (UTC)

Comparsion to Hebrew Wikipedia article feedback

As I wrote earlier Hebrew Wikipedia has article feedback script which I implemented in July 2011. I have compared the Hebrew Wikipedia article feedback to this:

Article Feedback Tool	Hebrew Wikipedia's script
Tip of "did you know you can edit this page" Nice interface which doesn't take the user to edit page in the process of feedback it is bold and attractive	The box is smaller, and expends and gives additional information only when starting to write the comment The feedback textarea is auto-growing allow any size of comment the feedback is available in the talk page, and can be monitored in the recent changes

In conclusion, although this article feedback tool (V5) is really nice, it is currently unusable as there is NO WAY to read the comments (and if there is, it is hidden somewhere in a special page and isn't integrated to recent changes and to the article talk page) and reader can't write large comments (the box doesn't have scrollbars). Eran (talk) 07:10, 6 January 2012 (UTC)

• Yes; the lack of a page to read the comments is deliberate. It's only been deployed on a few pages so we can test what we need to tweak on the form, and we're still designing the page. The scrolling point is a good one, but when looking at the submitted feedback internally I've definitely seen ridiculously large posts. Okeyes (WMF) (talk) 09:11, 6 January 2012 (UTC)

Which then wrecked FES a bit. --17:10, 6 January 2012 (UTC), Utar (talk)

As I've noted before I believe, I'd oppose readers being able to see the comments. We can catch ordinary vandalism and BLP issues relatively easily with the tools we have now, but they won't work with the tool so far as I know, and the tool makes vandalism and BLP violations easier at it is much simpler to use than editing. Dougweller (talk) 17:15, 6 January 2012 (UTC)

Whether the existing tools will work depends entirely on how the comments page is implemented. WhatamIdoing (talk) 01:29, 7 January 2012 (UTC)