Wikipedia:WikiProject User warnings/Testing/Twinkle

Documentation for testing of user warnings in Twinkle. See more notes and working rewrites for Twinkle deletion notices here.

Background research

Frequency of Twinkle use

Types of messages left on user talk pages through Twinkle between June 1 and September 9th, 2011. Message are categorized according to the auto-generated edit comment text.

We also ran a query to see how many Twinkle user talk edits there had been in recent months. According to this query, there have been 161,934 Twinkle messages posted on user talk pages between the beginning of July and October 19th 2011. Next, we intend to run the same query for Huggle and Igloo edits and to count total user talk page edits during the same time period, in order to find out which of these tools is used most frequently on user talk pages, and what percentage of all user talk page messages are made using these tools.

Most frequently used TW warnings

In order to figure out which templates to focus on in our A/B experiment with Twinkle templates, we first need to know which Twinkle templates are most used most frequently on user talk pages. To gather this sample, we queried the Enwiki database for a random sample of 1000 user talk page revisions made using Twinkle (identified by ([[WP:TW|TW]]) which is populated in Twinkle edit comments by default) since October 2010. Then we grabbed the whole list of user warning templates used by Twinkle and queried the API to get the text of each of these revisions. Then we pulled out all the template names from the HTML comments in the template messages and counted up the ones that appeared on the official list of Twinkle user warning templates.

Below is the list of the most-used Twinkle user warning templates from the last year. Should give us a good start to narrowing down which templates to update when we run the Twinkle study.

twinkle user warning template	number of uses in random sample of 1000 recent Twinkle user talk edits
{{Uw-vandalism1}}	226
{{Uw-vandalism2}}	124
{{Uw-vandalism3}}	115
{{Uw-vandalism4}}	57
{{Uw-block}}	43
{{Uw-unsourced1}}	41
{{Uw-test1}}	40
{{Uw-delete1}}	37
{{Uw-spam1}}	28
{{Uw-delete2}}	25
{{Uw-vandalism4im}}	23
{{Uw-error1}}	17
{{Uw-speedy1}}	17
{{Uw-vblock}}	17
{{Uw-speedy2}}	15
{{Uw-unsourced2}}	14
{{Uw-unsourced3}}	14
{{Uw-editsummary}}	13
{{Uw-spam2}}	13
{{Uw-test2}}	12

The final counts delivered through this method is not completely accurate, because any other, previous messages on the user talk pages these revisions were pulled from which ALSO used one of the listed user warning templates (whether or not it was left using Twinkle), will also be counted. There may not be an easy way around this using the API Query. There is a way to pull up just the text of a single diff by passing non-Query API arguments, but we couldn't make that method work this time. However, this count is probably pretty close. And after all, any user warning template that is in active use is a good candidate for A/B testing, whether or not it's used solely by Twinklers.

Proposed deletion and Articles for deletion notifications

General principles we are shooting for...

Simplicity/clarity: just focus on communicating the one or two actions you want the person to take. It doesn't have to be insanely short, just very clear about what you are encouraging people to actually do. Ultimately there's no way to teach people everything they need to know about deletion policy/process in one warning.
Lack of bite. Which means, because you're giving someone bad news, you need to sugar coat it a little extra to not make it bite-y. And because it's a notification, not a warning, don't use exactly the same language or icons as uw templates.
Letting people know where to ask for help. In the uw templates we've tested, that is usually the talk page of the reverting editor. This time we want to try the Help Desk, because it's already pretty normal for people to ask deletion-related questions there. Plus, I'm not sure if every nominating editor wants to teach the article creator how to navigate AFD etc.
Encouraging people to improve their articles. A lot problems with articles might be fixed if authors kept at it and used deletion discussions as a way to elicit constructive feedback. Let's ask them to do so.

Templates tested

The templates tested were:

PROD: new version, default version
AFD: new version, default version

Analysis

Now that the templates are finished running, things we want to know are...

Does the new version increase or decrease retention of the new editor compared to the default?
Does the new AFD version increase the number of article creators who participate in the relevant AFD?
Does the new PROD version increase the number of article creators who contest the proposed deletion by removing the template or using the talk page?
Do either the old or new templates draw editors who have stopped contributing back into the project? E.g. Let's measure how many people receiving notifications didn't edit at least 30 days prior to getting the message, the age of their accounts (from date of first and last edit) and whether there was a difference between those people and newer accounts.

Feedback

First it was hard for me to track down the item referred to, I had to use "what links here" for {{Z82}} or whichever it was. Secondly that was back in December - you need to ask for feedback much faster than that. As far as the wording goes, I probably didn't read it, I just see the template, and generally wonder why people don't leave a proper message about the page, if it actually needs deleting I will do usually it on the spot, otherwise I can explain why it doesn't and we can avoid an unnecessary deletion debate. The first line is a bit irritating to regular editors "Hello, Rich Farmbrough, and thanks for contributing to Wikipedia!" The phrase "some editors are discussing" implies that something heavy is going down. "I have started a discussion" would be better. (Or "an editor has started...." ideally the nominating editor should be leaving the message.) The rest is pretty good, but targeted at newbies which may or may not irritate old hands. Rich Farmbrough, 00:47, 17 February 2012 (UTC).[reply]

Hey Rich, sorry to make you hunt. When I messaged you I realized that this page didn't have a direct link to the new version. Fixed now. Thanks for the feedback. Steven Walling (WMF) • talk 00:58, 17 February 2012 (UTC)[reply]

I agree with Rich Farmbrough. The AfD I received on November 15th was roughly 1,000 edits ago for me -- I had to go hunt through my talk page archives for it. I really can't remember exactly what I thought when I saw it, but I remember that I went and merged it with the parent article, then agreed that it should be deleted. I agree that the PROD template's statement, "I wanted to let you know that I’m proposing an article that you worked on... (emphasis added)" is better than the AFD template's statement, "I wanted to let you know that some editors are discussing... (emphasis added)" The current AfD statement makes it seem like a group of people independently got together to discuss the article and that the templater is just letting the person know about the discussion, when that's usually not the case. The person posting the talk page notice is usually the person who started the discussion -- they should take credit for their action and say that "I" started the discussion. "Some editors" does, in my opinion, make it seem like something heavy is going on. Banaticus (talk) 23:01, 17 February 2012 (UTC)[reply]

I was about to reply that I have not received any notification. The last AfD notification I received was 29 November 2011. My beef with the deletion processes is that they are quite happy to decide the fate of a wikipedia item and leave the Discussion/Talk page unbannered, red, unassigned to projects etc. (whatever you want to call it). Some wording that items must be tagged with projects on the talk page (to allow article alerts) to notify relevant projects would be nice. Argolin (talk) 01:35, 17 February 2012 (UTC)[reply]
I was called here because I got an AfD notification, created with a use of new template. It was the third AfD notification in my life and I frankly didn't notice any difference as I cared most about the issue, not the notification's text. Still, as I'm already here, I would like to leave a couple of comments regarding these new templates:

They are more personal and friendly, which is definitely good.
New PROD notification lacks the information about consequences of declined PROD (I mean WP:CSD and WP:AFD). I think that this information is needed, as an editor would not be all that happy to learn that after he saved his article it was still deleted or nominated for deletion. The problem here is not with the possibility of deletion, but with the lack of awareness of the receiver.
The case with AFD is even worse: just imagine the motivation to stay civil, which is imposed on Joe, who reads in AfD discussion the statement by Jack that the article Jack encouraged Joe to edit a couple of minutes ago is a blatant advertising which qualifies for speedy deletion. Such a start can damage the consensus building and fuel the uncivil debate by far easier then make Joe feel friendliness in a notification about his work to be deleted soon.

So my summary is: the PROD template should be loaded with the WP:CSD- and WP:AFD-related information and the AfD template should be reduced to something like:

Deletion discussion about %article%

Hello, %username%! I have several concerns regarding the article %article% you invested some effort to. As I am convinced that this article should be deleted, I've started the discussion %discussion%. Your comment would be very appreciated there.

P.S.: I want to note specifically that I don't think the experienced editors should be taken in account here. Anyway, people used to receiving template-based notifications read only keywards ("deletion" in these cases) and links.

Dmitrij D. Czarkoff (talk) 01:47, 17 February 2012 (UTC)[reply]

I agree with this proposal, since I don't agree with the current "new" PROD directions to just remove the PROD template from the article after editing the article. The article likely has more problems than could be fixed with a "simple" edit, such as a complete lack of references, etc. From what I've seen, the creator of the article who just deleted a PROD template would likely get slapped on the hand for removing the template because they probably haven't fixed the reason the article was proposed for deletion in the first place, such as an article on a company that only references the company's website, etc. I think the user should be encouraged to join the deletion discussion. Banaticus (talk) 23:01, 17 February 2012 (UTC)[reply]

I found my PROD message to be friendly and not intimidating at all. It was calm, rational, and was easy to understand. As a person who likes Wikipedia, I hope that more PROD messages are like this in the future so that new editors can easily transition into becoming frequent editors of Wikipedia. GVnayR (talk) 01:49, 17 February 2012 (UTC)[reply]

The proposed versions are, in my opinion, not alarming enough. No matter what, something ugly is happening with your article. "Victims" should be properly warned about this. The wording is indeed better then the original version. Night of the Big Wind _talk 03:08, 17 February 2012 (UTC)[reply]

I agree with Night of the Big Wind, especially concerning the PROD template. The new version is far too sanitised, and in my opinion would encourage the removal of PROD templates without discussion or attempt to address the issues raised. You might as well say, "Someone's tagged your article. You'd better go and remove the tag," and nothing else. The new AfD template is somewhat better, but seems patronising to me; "thanks for contributing," "you haven't done anything wrong," why not slap a picture of a kitten on it as well? In my personal opinion, template messages like this should be somewhat officious and impersonal; we are trying to build an encyclopedia, not a Facebook group. Yunshui 雲‍水 08:08, 17 February 2012 (UTC)[reply]

RE your point that, The new version is far too sanitised, in my opinion would encourage the removal of PROD templates without discussion or attempt to address the issues raised – Well, that's pretty much exactly what the current PROD policy says:

To object to and therefore permanently prevent a proposed deletion, remove the {{proposed deletion}} tag from the article. You are encouraged, but not required, to also:
Explain why you disagree with the proposed deletion, either in the edit summary or on the talk page.

Consider notifying the editors involved in the PROD by placing a {{Deprod}} tag on their user talk page.

Add or modify an {{Old prod full}} tag on the article's talk page, to prevent renomination under Proposed deletion. It will then be listed at Category:Past proposed deletion candidates for easy tracking.

Consider improving the article to address the concerns raised.

In the current deletion system, PROD is meant to be used for non-controversial deletions only; articles that require discussion should be AfDed. If this gets confusing even for veteran editors, imagine how a newbie feels! Maryana (WMF) (talk) 23:20, 17 February 2012 (UTC)[reply]

I do think it's reasonable to encourage more discussion though. The truth is, if a PROD fails, the article will usually get AFD'd next, so we should prepare people for defending their position on an article. Steven Walling (WMF) • talk 23:24, 17 February 2012 (UTC)[reply]

Steve makes a valid point; how many AfD nominations contain the phrase, "PROD removed without explanation by page author"? Saying, "scratch the tag if you want the article kept," is not very helpful, since all that happens is an AfD five minutes later. The "other deletion processes exist" is useful for precisely this reason. Were I a new editor faced with an AfD after I'd removed a PROD tag, my first thought would be, "well, I had to remove the other tag to stop the deletion, so presumably I do that again..." then BLAM! {{subst:uw-afd}} rocks up on my talkpage. Futhermore, there might not even be a need for the rather tedious AfD process if the PROD-remover left an appropriate edit summary or attempted to fix the article, so those suggestions should be stressed in the warning message. Yunshui 雲‍水 23:01, 18 February 2012 (UTC)[reply]

The PROD notification I got in November? I see them so rarely on my own talk page that I didn't pay the form of it much attention, just went to the article to go fix it. On review, it seemed a bit densely jargoned for the new users who should be getting the most benefit out of this template. - Dravecky (talk) 20:25, 17 February 2012 (UTC)[reply]

I like the added friendliness, but it's very important that the existence of Wikipedia's policies and guidelines be mentioned in the user warnings; the vast majority of "keep" arguments made by inexperienced users in AfD discussions make no reference to the policies or guidelines, and such arguments fail to positively contribute to the discussions. I don't think mentioning the policies and guidelines detracts from the friendliness of the user warnings. Neelix (talk) 21:07, 17 February 2012 (UTC)[reply]

I November I have received the old one Template:AfD-notice-rand/default, with the warning triangle. I found it was - adequately so! - quite dramatic with the warning triangle, and quite useful with the links to WP:POLICYLIST and WP:DP. The new one Template:AfD-notice-rand/new is much too harmless. It sounds like someone is giving you a friendly notice, saying you did nothing wrong (soothing you into feeling safe again...) and Thanks for the contrubutions... I think a newbie would be entirely surprised to see that after that his or her article were deleted! So, rather a bit more dramatic rather than all-too-friendly. If you want to word it in a friendly manner, you could choose sth. like: "I encourage you to comment there if you think the article should be kept in the encyclopedia; of course do not hesitate to improve the article in the mean time, in particular if you consider your editing would overcome the specific objections raised against the article." --Chris Howard (talk) 18:26, 18 February 2012 (UTC)[reply]

Feedback from Anna Frodesiak

Love 'em!
Achieves the "General principles we are shooting for..."
The absence of the amber triangle thingy is good. Those make the user from feel like they've done something wrong, or triggered some sort of software warning.
The first person writing conveys a more human tone.
In the AFD one, the softening of "Nomination of X for deletion" into "Deletion discussion about", makes the user feel that a few others are discussing, and not some large committee with some overlord nominating. So, that's less embarrassing and more private.
In the PROD one, I like the clear bullets. The user will be freaking out, and will want clear step-by-step instructions on what to do.
In the PROD one, only 2 blue links is better than the current, and overwhelming, 7.
Friendlier, so better user retention. Anna Frodesiak (talk) 01:40, 17 February 2012 (UTC)[reply]

from Slowking

the new templates are nicer.
however, the critical thing is how they are used. a personal note, indicating that the nominator actually read the article might go a long way.
offering to move to incubator or subpage would be nice as well.
you might have a go at "speedy deletion" and "Replaceable fair use" also, i'm seeing misuse of speedy, "appearance" to avoid prod, afd.
would like to see more training in wp:before
driving by may solve the problem, but dosn't make the system of article creation better. that would require Wikipedia:Article wizard, or Wikipedia:Teahouse.
the move to coach and librarian, away from traffic cop or drill sargeant is good. Slowking4⇔ †@1₭ 21:01, 17 February 2012 (UTC)[reply]

from John Vandenberg

I was asked to comment with regards to my experience with the messages received in November, which you can see here, which were combined with these edits to the article, and the resulting AFD was a definite keep. As you can imagine, this use of Twinkle doesnt reflect well on Twinkle. The use of CSD->PROD->AFD without discussing the matter is rude. I didn't care what the messages said; I knew I was being templated so I zone out and dont read the message (sorry!;-)). Being templated doesnt bother me much, but I was annoyed that the templater didnt need to specify a reason to CSD and PROD an article which had been in Wikipedia since May 2007 - over four years. It was obviously not an uncontroversial deletion, so it should have gone from CSD to AFD. There was an obvious merge target, so there was no need for an AFD either. A one-to-one discussion, maybe followed by a third opinion, should have resolved the matter to everyones satisfaction. John Vandenberg ^(chat) 00:01, 18 February 2012 (UTC)[reply]

I realise that my comment here is probably complaining to the wrong department, as you are focusing on improving the messages, and im not in the demographic you're trying to help with the message improvements. Still, these templates are being used instead of discussion, and that is something that the message writers need to consider. It's not the tools fault, but at the same time our systems arnt good at preventing a Twinkler misusing it enmass. Imagine the above sequence happening to a new user; we'd have lost them for sure. Maybe, just maybe, the messages can help keep that new user engaged in the system by giving them support. For e.g., if the Twinkler doesnt provide a reason for PROD (e.g.), the message could put the users talk page in a {{help me}} state, so our more experienced volunteers can assist, and the prodded page can say "this is a stupid prod without a reason provided; please provide a reason or remove this template". John Vandenberg ^(chat) 00:55, 18 February 2012 (UTC)[reply]

from Bob the Wikipedian

Um...the message I got happened to be for an article I created a long time ago on a non-notable topic, so I didn't exactly read the message. Rather, I saw the note, saw "deletion", thought "haha! I remember that! I can't believe I actually made an article on that! Non-notable indeed!" and decided to just let it decay and get deleted by someone else, although I could easily have deleted it myself. So, that said, I'm glad I got a message, but I didn't exactly read it! Bob the WikipediaN ^{(talk • contribs)} 03:37, 18 February 2012 (UTC)[reply]

from Dirk van der Made

I was asked to comment on something that has something to do with communication. Well, here's a communication issue. What is it about? Well, templates, and how they look, it seems. But what do they look like then? And what alternatives are there? Did I ever receive one? If so, when? And the templates are AfD and PROD. I guess the first stands for Article for Deletion. PROD I haven't a clue. Of course I could look that up, but if you want more feedback it would be a good idea to be a bit more helpful in that respect. I have other things to do, but I'd like to help. If I don't have to do all the figuring out. Oh, and it appears to be related to Twinkle, which is something one has to install to use. But afaik I haven't installed it, so how can I comment on it? DirkvdM (talk) 07:42, 18 February 2012 (UTC)[reply]

I agree, this is very abstract as far as communication goes. What you've been asked for feedback on is a warning message, either for AfD or PRoD (I haven't any idea what that is), that someone else used Twinkle to add to your user talk page. They're wondering what you thought about that message you would have received in November-ish. Personally, I think at least a link to the notification in question would have been helpful at the least. Bob the WikipediaN ^{(talk • contribs)} 20:37, 18 February 2012 (UTC)[reply]

This is about the notices you received regarding "Heerlen photo gallery" (A gallery created December 2006). The messages are a AFD then prod. The AFD is Wikipedia:Articles for deletion/Heerlen photo gallery. John Vandenberg ^(chat) 11:31, 19 February 2012 (UTC)[reply]

Ah, so I assume prod means 'proposed deletion'. But I still don't know what the difference is between afd and prod.

Do I get this right? I am asked to comment on the presentation of a message that was placed in an article I started, and which has since, as a result of the message, been deleted. So the message is gone. So how can I comment on it? That would be silly, so I assume I misunderstand. DirkvdM (talk) 17:16, 19 February 2012 (UTC)[reply]

Hi Dirk. Yes, prod means proposed deletion. wp:proposed deletion is a quick deletion process; wp:AFD is usually a longer process. You are being asked to comment on this AFD message and this prod message. Do those two messages make sense? Did they warm your heart? ;-) Would you like a scotch?! ;-) John Vandenberg ^(chat) 10:08, 20 February 2012 (UTC)[reply]

Well, now you've done it, John. One of these Aprils firsts, I'm going to propose that survey to Rcom :D Maryana (WMF) (talk) 23:01, 21 February 2012 (UTC)[reply]

From User:bashereyre

I very much like the automatic notification if I inadvertently forget to check a possible disambiguation (ie Master/Hackney Bashereyre (talk) 08:21, 18 February 2012 (UTC)[reply]

Those are pretty friendly, actually! I like the part that says you can remove them from your talk page if you want. We should've tested text like that...

Anyway, sorry for the confusion: what we were asking about for the purposes of this experiment was AfD and PROD templates, both the current ones and the test versions we made (see above). Any feedback on that would be most appreciated :) Maryana (WMF) (talk) 23:06, 21 February 2012 (UTC)[reply]

From George J. Dorner

As memory serves, I have only been involved in one AfD/PROD during my entire WP career, so I have no standard of comparison. Incidentally, that occurred in October 2011, not November. I PRODded Confirmation and overclaiming of aerial victories during World War II as being non-notable. The community decided to keep it, although it is a pig's breakfast that no one will bother to work on.

As for whatever template I used for the purpose...I didn't pay any mind to it. To me, it was just another case of WP bureaucracy to be lumbered through.

Georgejdorner (talk) 15:26, 20 February 2012 (UTC)[reply]

From Stevertigo

I have not used these templates, nor have much had articles which have had these used on them. I agree with those above who like the idea of less bite, but I also agree that these templates should prominently feature the process and policy pages as an introduction to newbies. Its possible in an attempt to oversimplify something that one winds up just watering things down too much. -Stevertigo (t | c) 20:38, 20 February 2012 (UTC)[reply]

From jd2718

I was templated three times a few months ago after creating a substandard stub (which was quickly improved by other editors). I recall distinctly being annoyed that the first to propose deletion had not bothered even to google the subject - but have no recollection of how I perceived the template per se. I guess the content outweighs the form by that much... I know that's not what you are looking for, but how about a script that posts back on the nominator's talk: "you have just proposed that someone else's work should be obliterated. Have you bothered to check if this is the right thing to do?" And since you are worried about the tone of the message - "Thank you and have a nice day" Jd2718 (talk) 03:14, 21 February 2012 (UTC)[reply]

From ottawahitech

On Feb 17 I received a message on my talk page which started with these words: Hi Ottawa. In December you got either an AfD or PROD notification...

FYI I get an AFD or a PROD notification just about every other day (or so it seems) - it is getting to the point where I find it very difficult, if not impossible, to make long-lasting contributions to Wikipedia. I seem to be running around trying to put out fires constantly.

So sorry, I cannot help with whatever change happened to the format of the AFD/PROD message back in December. Ottawahitech (talk) 15:13, 23 February 2012 (UTC)[reply]

From DGG

As a number of people have said above, the real problem is of course the deletion process--which desperately needs fixing so the whole thing works in a friendlier direction with more emphasis on keeping content and contributors, while at the same time removing what shouldn't be articles. However, this means fixing the general attitude of many experienced Wikipedians to deletion in general. That isn't going to be quick. Changing the templates can be quick, and can be seen as a first place to start,

The new templates are much better, but as has also been remarked above, they don't give a sufficient sense of urgency. It's very difficult to give a sense of urgency without being unfriendly. I'll give it a try in a little while. DGG ( talk ) 20:03, 26 February 2012 (UTC)[reply]