CMUResearcher

Please feel free to leave comments or suggestions!

Our talk page

Hi, I notice you describe this as "our talk page" please can you confirm that that was a slip of the tongue and that only one individual has access to the password of this account. ϢereSpielChequers 15:07, 16 July 2009 (UTC)

Hi, not to worry. Glad you plan to contribute. Exploding Boy (talk) 15:14, 16 July 2009 (UTC)

My apologies Exploding Boy for the misunderstanding, I didn't realize creating a username for 4 people would be a violation of policy. Truthfully, the account was only ever used by myself, but we thought longer term it would be better to have others be able to edit with it but now I see that is not possible. I tried very hard to not violate any policies with the survey, I contacted both the Chief Research Officer, and the the Wikipedia Research Network for advice and followed it. This survey is not for the promotion of any organization or product, it is an attempt to help the Wikipedia community, much like the creators of suggestbot or any other technical contribution, but to help the community we need input from the community. Also, personally I would like to get more involved in editing articles, I've been looking through lately on places I can make a contribution, so far issues with Organizational Behavior seem good for me to update. CMUResearcher (talk) 18:54, 16 July 2009 (UTC)

Who are you?

So who, exactly, is conducting this study? I see no names whatsoever, whether grad student, doctoral candidate, principal investigator, or faculty advisor. And the outside website used for gathering data -- I would have thought Carnegie-Mellon, of all places, might have the ability to pull together a functional webpage -- gives me enormous pause. I still remember Marty Rimm, after all. --Calton | Talk 17:23, 16 July 2009 (UTC)

Good to hear from your Calton! About your comment on using an outside website for the survey, did you have any trouble with the functionality of the webpage? I'm just curious if there is a reason you suggested we should have created it in-house. As for who is conducting the study, the principle investigator for this study is Robert Kraut, he is full professor in Human Computer Interaction. Other team members include professor Niki Kittur who has published several papers on Wikipedia, and a graduate student Moira Burke who has also published research on Wikipedia and is a graduate student in HCI. Lastly, myself, Ben Collier, a PhD student. -CMUResearcher (talk) 18:55, 16 July 2009 (UTC)

Survey difficulty

Just leaving a note to inform you folks that I started, but did not complete a survey. The format at present, where I have to categorize every single characteristic (including many that I don't bother checking up on for RfA candidates) is a strong disincentive for completion. After about 10 I felt as though I was dragging phrases from left to right just to finish. I know that the alternative (allowing participants to sort as little as they like) will make it difficult to weed out aborted attempts and result in drastic under-representation of traits, it may be worth considering if your response is lackluster. For what it's worth I wish you luck but I probably won't go back to reattempt it. Protonk (talk) 05:42, 5 August 2009 (UTC)

Ditto. It also wasn't clear to me the reason for the format and categories. What are you actually trying to find out? My answers might change depending on what the question is. Additionally, many of the left-hand items don't seem to fit with the right-hand items at all in any meaningful way. Also, it's not clear enough that you don't have to actually sign up. This doesn't strike me as a very well designed study. Exploding Boy (talk) 06:38, 5 August 2009 (UTC)

Same here. This really feels weird. I can't see any value in this. In addition to the concerns raised by others above, I am concerned about the high number of leading questions. Example: When you name an item "Edit summary length", it just begs to be dragged to the box "Edit summary usage". The list also seems to contain some fundamental misunderstandings of how Wikipedia works. For instance, "appearing on watch lists" is so vague that it's hard to know what you mean. Every user appears on watchlists. Maybe you meant the frequency of such appearances; but that is nothing but their "edit count". I also wonder how "Demonstrating NPOV" can be seen as a low-level measurement. NPOV is defined by the interaction of different viewpoints, and is about as hard to define for an individual as the equality of a pig. (That latter point may not be shared by everyone at Wikipedia, but I'd still like to know how you propose to measure NPOV of an individual!) — Sebastian 17:52, 5 August 2009 (UTC)

I started the survey and then abandoned it. The phrases and categories don't lend themselves to natural matching, so the exercise seems a bit pointless. It's quite baffling - and I say that as a research professional with 19 years' experience in social science research and experimental design. Majoreditor (talk) 00:31, 6 August 2009 (UTC)

Likewise. Even though I'm inclined to believe it is an opinion survey, it fails even at that since every single unsorted thing has to be tacked, and many of the items are the kinds of things I'd need to be clairvoyant to tack. Also, like SebastianHelm, I have issues with what seems to me like working backwards from an answer to get a solution and very vague responses (i.e. "appearing on watchlists" - ALL editors, good editors and vandals alike, tend to show up on watchlists, so a more narrow definitition is required here). -Jeremy ^{(v^_^v Tear him for his bad verses!)} 06:13, 6 August 2009 (UTC)

Yes, but there are certain pages to edit that will make you show up on more peoples' watchlists. Also, the more different articles you edit and the more frequently you do so, the more you show up, as well. Then, people start recognizing your name. hmwitht 14:03, 6 August 2009 (UTC)

Greetings...

Hello, CMUResearcher, and welcome to Wikipedia!

To get started, click on the link that says "welcome".

I (and the rest of us here, too) hope you like it here and decide to stay!

Happy editing! DThomsen8 (talk) 12:05, 5 August 2009 (UTC)

Always sign your entries on Village Pump

Please, always sign your entries on Village Pump or elsewhere. The edit bar has an icon for doing it. Beyond that, I am put off by remarks above about how difficult it is to finish your survey. Do you learn something from those who start but don't finish? --DThomsen8 (talk) 12:36, 5 August 2009 (UTC)

Remarks moved here from other pages

In an effort to keep conversations on topic, I moved the following here from other pages: — Sebastian 22:46, 5 August 2009 (UTC)

Oh, so I decided to take it - content work and the like were dumped into trust worthiness, and everything else was in "varied experience". I wonder if they will understand. Ottava Rima (talk) 22:26, 5 August 2009 (UTC)

Interesting ... again, but the term WP:SPAM is starting to come to mind. — Ched :  ?  22:42, 5 August 2009 (UTC)

I kind of agree. Are you planning on actually editing Wikipedia, or is your whole purpose here to promote this ... study (or whatever it is)? Exploding Boy (talk) 00:40, 6 August 2009 (UTC)

• I don't mind if this account never edits the encyclopedia. Researchers and other curious social scientists are going to pay more attention to wikipedia and a natural path for communication may be the user/user talk pages. We should remember that they are new here and try not to demand that they render unto Caesar, as it were. I'm probably injecting my own bias here--if this were a company selling widgets I would have blocked the account on sight for spamming, but I don't feel too bad giving these folks a pass based on the context. Protonk (talk) 02:40, 6 August 2009 (UTC)

I guess for me it's partly that the survey itself just seems so amateurish and badly designed. Maybe it would be worth thinking up some kind of academic wikiproject that could be used for people who are interested in recruiting research subjects. Exploding Boy (talk) 02:48, 6 August 2009 (UTC)

I dunno. I suspect the researchers are both learning how to do surveys and learning how wikipedia works at the same time. We (by that I mean those of us who have plowed considerable amounts of time and effort into the project) know almost intuitively what about wikipedia is important or interesting. We can spot the salient policies, boards and norms from a mile away. We can articulate the fundamental frictions easily. A new user can do none of these things. Perhaps this team is experimenting with a novel methods (asking respondents to categorize behaviors), subject matter, and tools (maybe the web based survey bit is new). I feel that we can do little wrong by cutting them some slack. Protonk (talk) 03:05, 6 August 2009 (UTC)

FWIW, I completed the survey. I'm not sure how many other people did, though, and I'd be curious to find out. hmwitht 14:07, 6 August 2009 (UTC)

These people are legit

I know personally Kraut, Kittur, and Burke (though not Collier himself). They are well-known and highly qualified scientists. What they (and I, as a researcher myself) are trying to do is learn how Wikipedia works in a scientifically rigorous way, to glean lessons from this unique and successful community/information resource -- which can then be applied to other areas, leading to better communication and better decision-making in a variety of venues.

The claimed affiliation could be falsified, of course, but it seems very unlikely to me that an evildoer would choose this rather obscure fictitious identity.

You can look me up in the real world as Reid Priedhorsky. --R27182818 (talk) 14:56, 6 August 2009 (UTC)

I can also vouch for these individuals. The work they are doing here is to understand and improve Wikipedia. We should not be pushing so strongly against their efforts. Like Wikipedians, they are working to make Wikipedia better. It is just that they are trying to understand/improve Wikipedia as a system rather than on an article-by-article basis. --EpochFail ^{(talk|contribs)} 15:24, 6 August 2009 (UTC)

Well, if you both know them, would you have an explanation why they are so acting so amateurish - both in in their lack of preparation to understand Wikipedia, and in the way they conduct the survey? Science grows by dwarfs standing on the shoulders of giants, and there's plenty to stand on, both for understanding Wikipedia (e.g. the book "Wikipedia: The Missing Manual by John Broughton" is said to be good) and for how to do surveys. Does CMU not have better standards for their research? — Sebastian 16:58, 6 August 2009 (UTC)

I'm not sure anyone's really "pushing against them strongly." I think people are interested, but have some fairly legitimate concerns. To begin with they violated our username policy and got their first account blocked--it's easy to do and happens often, but doesn't bode well for a well-prepared study. Then their study just seems, well, odd. There isn't sufficient information provided on the survey page about who is conducting the research and why, and what question(s) they're seeking to answer; the study doesn't appear well-designed or capable of producing meaningful results. There are no open-ended questions. Finally, it seems strange that they failed to take advantage of one aspect of Wikipedia: the talk page. CMUReasercher could have created a survey discussion subpage where people could actually discuss... whatever it is they're trying to ascertain. Anyway, best of luck to them and all that; perhaps we'll all turn out to have been utterly wrong. Exploding Boy (talk) 17:16, 6 August 2009 (UTC)

Actually, Wikiversity would probably be more useful to them, since it has a few more bells and whistles (like DynamicPageList) to hel organize and analyze the data (see my comment below). I'm generally not one to jump to attention when someone says "hi! please take our survey!", but I am interested in this sort of research. WV has the added advantage of letting others also take part in the research phase from all sides of the table. --SB_Johnny | ^talk 23:23, 16 September 2009 (UTC)

Response to Survey Concerns

Thank you for your feedback! As with any research there are always tradeoffs to be made when trying to collect data. In this case, we had the option of including less than the true number of measures or more than the true number of measures. I'm not sure what your background in research is, but imagine that there are 15 good measures of an RfA candidate. If we came up with 10 and had the community rank them, we would be missing 5. If came up with 20 and had the community rank them, when we do the cluster analysis over a large sample, it becomes statistically evident that 5 of them are not good measures, and we throw them out. In your examples above, any one person cannot affect the results so do not worry about skewing anything. If you found 4 or 5 items you thought were poor, place them anywhere randomly, and when we analyze data over 100+ users we will see that people are not consistently placing items in the same bin, and thus they are a poor measure. In addition, when naming measures and categories it was a challenge to make them short enough so you aren't required to read a paragraph every time you made a decision, but long enough to be meaningful. While we spent substantial time improving the survey and piloting it, I'm sure as with any piece of research we make errors in clarity vs. brevity.

To perhaps say the same thing more clearly, in both lists there are good and poor measures, and good and poor categories. Rather than have the research team simply decide what is good and bad, we have asked the community. If a measure is poor, we will find that out because the community cannot agree what category it is for across a large number of people. We're sorry if that has created in frustration in taking the survey because you feel some measures are poor, but if you would complete the survey we would have the data to empirically confirm the measures are poor from community consensus.

Thank you again for all the feedback, I hope I've addressed the issues you've raised and feel free to continue to comment.

CMUResearcher (talk) 17:55, 6 August 2009 (UTC)

If you're hitting difficulties...

Hi CMUResearcher. Assuming these aren't deeply personal questions that require anonymous answers, you might have a bit better luck just asking people to take the survey on Wikiversity, which is more or less "in house". I'm sure you would find plenty of people willing to help set it up on the Colloquium if you're not familiar with wiki-markup or data processing. --SB_Johnny | ^talk 23:17, 16 September 2009 (UTC)

Research and RFA

Hi CMU, Based on your earlier survey I wondered if you would be interested in a problem area on the Wiki that would benefit from research - see Wikipedia talk:Requests for adminship#Admin retention. Alternatively perhaps your existing research can cast light on our problems with RFA and adminship. ϢereSpielChequers 16:22, 17 September 2009 (UTC)

WikiProject Proposal --> Wikipedia:WikiProject_Research

I saw that you added your name to the list of supporters in the project proposal for WikiProject Research. We started the the actual project a couple of days ago. You might want to add your name there. --EPOCHFAIL^(talk|work) 16:22, 17 November 2009 (UTC)

WP:Research last call for cleanup before an RFC

I'm messaging you about this because you are listed as a member of Wikipedia:WikiProject_Research. PiperNigrum and myself are about to start the (poorly documented) process of submitting WP:Research for review by the community and I'm making one last call for cleanup and input. Please give the article a careful read if you have a chance. Unless major flaws are discovered, we'll be adding the {{rfc}} template to the talk page to start the process on March 2nd. That's one week from this posting. --EpochFail^(talk|work) 22:32, 23 February 2010 (UTC)

Access to deleted article

Hi Aaron,

An article I wrote was recently deleted by an admin but I need to access the article (or at least the text of the article).

Is there any easy way for me to do this? Can you (or another admin) send me the text of my article?

I greatly appreciate any assistance you can provide,

Kbattick (talk) 15:56, 27 July 2010 (UTC)kbattick