Wikipedia talk:India Education Program/Analysis/Independent Report from Tory Read
Greetings. I’ve just posted the Pune Pilot Project report. Please let me know if you have questions or factual corrections. Foundation personnel (Barry Newstead, Frank Schulenburg, Hisham Mundol and LiAnna Davis) have reviewed the report for factual errors. Ayush Khanna, data analyst for global development at the Foundation, reviewed the data I’ve cited. I had the final decision about content.
You’ll notice that I’ve identified Foundation personnel by name and that I’ve identified the rest of the people involved by their role only. I decided to take this route because a number of people I interviewed requested that I keep their identities anonymous. Had I named the other people quoted, it would have been too easy for readers to deduce the identities of the people who wish to remain anonymous.
Please accept my apology for missing the January 15 deadline. The assignment was more complex than I anticipated, and I had other client commitments in the mix.
My thanks to all of you who gave your valuable time and expertise to interviews for this report.Toryread (talk) 23:08, 20 January 2012 (UTC)
- Thanks, Tory. I haven't had the time to read your report in depth to comment and ask questions, but I'd like to note outreach:Expanding Wikipedia's Education Program: Key learnings from the pilot programs. MER-C 10:04, 21 January 2012 (UTC)
Comments
editNow that I've read the report in depth:
Some Wikipedians criticized the CAs (see an example here), and CAs said they felt unfairly judged. "The Wikipedia community in India didn't help us, didn't speak up for us, didn't defend us," said one CA. "We're part of the community now, and it hurt."
Isn't this a direct consequence of the local community being sidelined? — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- Your point is fair, but it's more complicated than that. A number of Wikipedians I interviewed in India for India Chronicles and this report said that there is a tendency for some in the India community to hang back and wait for failure, then say, I told you so.Toryread (talk) 22:53, 24 January 2012 (UTC)
- Unrelated Comment : Part of the problem why some hang back is because of the response they get initially. Read PS of this mail. I will post my replies on the topic sometime soon Srikanth (Logic) 11:31, 25 January 2012 (UTC)
- "Local factor" (as in Pune) is overrated IMO. On the mail thread I had asked about how strong they are to help, since I knew only 2 English wikipedians who were in some position to help IMHO, one already took fellowship, the other wrote that mail. Even for the next phase, if planned, situation will be no different, India has fewer English Wikipedians who identify themselves as one living in India and are free to help. Srikanth (Logic) 14:02, 25 January 2012 (UTC)
- Logic has misunderstood the local factor which did not pertain to online community support as provided by online Wikipedians and global advisors. Pune community from beginning emphasised its value as advisors and not as online mentors (for which there were not enough wikipedians). The Pune community geared itself to provide input pertaining to knowledge about not just Wikipedia but also local educational conditions. Shortly after starting, the project and community drifted apart. The IEP project faulted in assumptions about these two areas - local environmental realities and Wikipedia environment. If the local conditions were gauged more correctly, the later copyvio, cleanup issues may not have manifested or may have occurred in diminished form. Anyway, we are not saying that if the Pune community had been involved throughout as stakeholders the problems would not have manifest, only that there was no chance for us to give corrective advice. Our point of our whole representation was :
- Make local communities true stakeholders throughout the process.
- Listen carefully to them as they bring a set of ideas, experiences which would not probably come from other stakeholders.
- They can provide opportunities, point out threats and stand behind the project locally all of which may be necessary.
- You may succeed without local community support but your chances of going astray or not succeeding are also greater.
- AshLin (talk) 13:29, 26 January 2012 (UTC)
- Logic has misunderstood the local factor which did not pertain to online community support as provided by online Wikipedians and global advisors. Pune community from beginning emphasised its value as advisors and not as online mentors (for which there were not enough wikipedians). The Pune community geared itself to provide input pertaining to knowledge about not just Wikipedia but also local educational conditions. Shortly after starting, the project and community drifted apart. The IEP project faulted in assumptions about these two areas - local environmental realities and Wikipedia environment. If the local conditions were gauged more correctly, the later copyvio, cleanup issues may not have manifested or may have occurred in diminished form. Anyway, we are not saying that if the Pune community had been involved throughout as stakeholders the problems would not have manifest, only that there was no chance for us to give corrective advice. Our point of our whole representation was :
Although two global Wikipedians interviewed for this report called for Foundation staff and consultants to lose their jobs over mistakes they made planning and managing the Pune Pilot Project, the evidence suggests that this action would be premature.
Editors responsible for this many copyvios and poor quality content tend to be permabanned (search WP:BANNED for "copyright"). Who's going to be held accountable for this mess? (see my comment about moral hazard below) Who will be held accountable in the future? — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- This is a fair question, best answered by someone at WMF.Toryread (talk) 22:53, 24 January 2012 (UTC)
- The silence from the WMF in response is deafening. MER-C 07:17, 31 January 2012 (UTC)
- Surprised? It's a common trick not to respond when you're not in the right forum to force something through against dissent; far easier to only do it somewhere when their opinions are the only ones that matter, not here. The Blade of the Northern Lights (話して下さい) 07:44, 31 January 2012 (UTC)
- What would be gained by sacking people? If they've learned from the mistakes made here, they should keep their jobs. This appears to have been an institutional-level stuff-up anyway. Nick-D (talk) 07:13, 1 February 2012 (UTC)
- I'm more interested in formal warnings so that if they stuff up in the future, they're out of there. As it stands, the burden of the IEP failure falls disproportionally on the general editing community and not the paid WMF consultants who caused the mess. This is an unacceptable and unsustainable moral hazard. MER-C 13:41, 1 February 2012 (UTC)
- The idea that there has been no costs for the WMF staff involved is unjustified by the facts. Take a moment to reread all of the public discussion on the IEP that refers to people by name and pretend that the mistakes you made at work were similarly discussed in public. The kind of criticism we are getting here is something we accept and appreciate when we come to work at WMF, but it is highly unusual for people to have their work scrutinized publicly in this manner (including having people saying that you should be fired). Would you feel that there were no costs to you, if this was your job? The costs are there and will not be easily forgotten, which is good as no one wants to repeat this. Many programs fail in many environments including ours. The people whose jobs should be on the line are those that don't learn from mistakes and improve. Not those who make mistakes. --Barry Newstead (WMF) 20:17, 1 February 2012 (UTC)--20:12, 1 February 2012 (UTC) — Preceding unsigned comment added by Bnewstead (talk • contribs)
- Just a note: On wiki discussions such as these are likely to be archived upon conclusion. — Nearly Headless Nick {C} 21:44, 1 February 2012 (UTC)
- That is a fair point, but remember the WMF planned and executed this program without any community consultation whatsoever. Therefore, the only fair outcome free of moral hazard is if the WMF bore all of the burden, including the cleanup. (Yes, I am aware that WMF staff, unless under certain circumstances, cannot touch mainspace with their WMF accounts.) You're forgetting that we are volunteers who contribute our free time and/or the money that goes into your salary. We are not obliged to clean up after messes you cause, and if we decline to do so then you will have serious problems. Furthermore, you would be doomed to repeat the same mistakes if we didn't spend the time to scrutinize your actions and offer criticism on the understanding that you will take it on board and actually fix the problems. Stop taking us for granted.
- The idea that there has been no costs for the WMF staff involved is unjustified by the facts. Take a moment to reread all of the public discussion on the IEP that refers to people by name and pretend that the mistakes you made at work were similarly discussed in public. The kind of criticism we are getting here is something we accept and appreciate when we come to work at WMF, but it is highly unusual for people to have their work scrutinized publicly in this manner (including having people saying that you should be fired). Would you feel that there were no costs to you, if this was your job? The costs are there and will not be easily forgotten, which is good as no one wants to repeat this. Many programs fail in many environments including ours. The people whose jobs should be on the line are those that don't learn from mistakes and improve. Not those who make mistakes. --Barry Newstead (WMF) 20:17, 1 February 2012 (UTC)--20:12, 1 February 2012 (UTC) — Preceding unsigned comment added by Bnewstead (talk • contribs)
- I'm more interested in formal warnings so that if they stuff up in the future, they're out of there. As it stands, the burden of the IEP failure falls disproportionally on the general editing community and not the paid WMF consultants who caused the mess. This is an unacceptable and unsustainable moral hazard. MER-C 13:41, 1 February 2012 (UTC)
- What would be gained by sacking people? If they've learned from the mistakes made here, they should keep their jobs. This appears to have been an institutional-level stuff-up anyway. Nick-D (talk) 07:13, 1 February 2012 (UTC)
- Surprised? It's a common trick not to respond when you're not in the right forum to force something through against dissent; far easier to only do it somewhere when their opinions are the only ones that matter, not here. The Blade of the Northern Lights (話して下さい) 07:44, 31 January 2012 (UTC)
- The silence from the WMF in response is deafening. MER-C 07:17, 31 January 2012 (UTC)
- You have a volunteer account, do you not? The best way you can restore community confidence is help in the cleanup. Read WP:Cv101, then convince us that you are able to clean up after your own mess by clearing a couple hundred items at WP:SCV. Ask MRG if you need help. Chop, chop! MER-C 09:12, 3 February 2012 (UTC)
- I wish it were that easy. MER-C, I'm sorry, but they don't have that option. The Foundation heavily restricts what they can edit - even on their volunteer account. Due to legal reasons, they can't edit (as a volunteer) anything that they found as a result of their work. It's a die-hard rule, set by Geoff shortly after he arrived, well in advance of this program. Please - don't be upset with them for not doing that, they've asked time and again if they could be allowed to, and I'm sorry, but the answer is simply no. :( Philippe Beaudette, Wikimedia Foundation (talk) 14:38, 9 February 2012 (UTC)
- I understand. This gets back to my original question, to which there still hasn't been a satisfactory response: how is the Foundation going to hold their consultants accountable? MER-C 08:49, 10 February 2012 (UTC)
- I wish it were that easy. MER-C, I'm sorry, but they don't have that option. The Foundation heavily restricts what they can edit - even on their volunteer account. Due to legal reasons, they can't edit (as a volunteer) anything that they found as a result of their work. It's a die-hard rule, set by Geoff shortly after he arrived, well in advance of this program. Please - don't be upset with them for not doing that, they've asked time and again if they could be allowed to, and I'm sorry, but the answer is simply no. :( Philippe Beaudette, Wikimedia Foundation (talk) 14:38, 9 February 2012 (UTC)
Mundol was actively searching for a communications person for the team, but there was not yet a team member with dedicated responsibility for overseeing talk page conversations with the global Wikipedia community.
It goes without saying the best person for this role is already an established Wikipedian. You're not hiring a PR flack, are you? — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- Hisham, can you answer this?Toryread (talk) 22:53, 24 January 2012 (UTC)
- I have delayed the selection of the communications person primarily because I was unable to find someone from the community or who is intimately familiar with the community and projects. I am hoping that I can conclude the process reasonably shortly with the right candidate. Hisham (talk) 03:46, 25 January 2012 (UTC)
To get help with cleanup, the San Francisco team reached out to OAs from the U.S. program, and 20 OAs volunteered to assist on the Pune Pilot Project. These OAs went right to work and made a major contribution to the cleanup effort, but CAs said they wished the work had been more collaborative. "The OAs did great work, but they sidelined us," said one CA. "They didn't even include us in the cleanup tables. Communication between the CAs and the OAs wasn't good. There needs to be direct communication."
Are those cleanup tables the ones at WP:IEPS? Given the general lack of mainspace edits from the CAs (see the table at WT:IEP#Going it alone) and the copyvios in their contributions the sidelining was inevitable. Furthermore, the report fails to mention that the OA cleanup was, according to one American(?) OA, "useless" and had to be superseded by a community cleanup effort. — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- I was unaware of this. Thanks for adding it.Toryread (talk) 22:53, 24 January 2012 (UTC)
Keep the phase-two pilot small.
Numbers? — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- I recommend a maximum of 200 students, with 20 active CAs. If there are fewer CAs, then include fewer students. Hisham and Nitika, what are your thoughts?Toryread (talk) 22:53, 24 January 2012 (UTC)
- My current thinking is fewer than that, Tory. I'd recommend a number of less than 100, maybe even 50 students - and determined by the calibre of faculty and students - as well as the calibre and number of Campus Ambassadors. Hisham (talk) 03:46, 25 January 2012 (UTC)
Have students compose new articles in sandboxes.
Sandboxes also must be checked for copyvios. Who's going to do this? — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- Professors and well-trained CAs would be my recommendation. Hisham and Nitika, what are your thoughts?Toryread (talk) 22:53, 24 January 2012 (UTC)
- Agreed. I'd add that professors also need to be selected based on their teaching ethos with regard to plagiarism so that we have professors on board who get the issue and are familiar with how to prevent it. I'd also suggest software solutions (CorenBot?) to augment this. Hisham (talk) 03:53, 25 January 2012 (UTC)
- Regarding software solutions, I'd like to add that one admin specifically requested that WMF support new tools for New Page Patrol and the cleanup system.Toryread (talk) 19:01, 25 January 2012 (UTC)
- Provide a dashboard where the pending articles and tasks can be listed for normal wikimedians to participate too. AshLin (talk) 14:13, 26 January 2012 (UTC)
Other comments:
- The en.wp community will not tolerate any more IEP failures. To quote a certain administrator: this report "fails on one important point: the availability of the community and their enthusiasm (or future lack of it) to clean up a mess made by the WMF and to make any recommendations for solutions for it" (private communication). I have stated multiple times that I will apply for sanctions to limit the extent of another cleanup round. — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- I recommend that WMF design the next IEP pilot such that there is no mess to clean up. What happened to the English WP community during the Pune Pilot should not happen again.Toryread (talk) 22:53, 24 January 2012 (UTC)
- The userspace to mainspace transition can be used to control this. Have two models:
- Articles on new topics can be developed in userspace and passed before moving.
- Specific information added to existing articles be placed on talk pages, asking normal Wikipedians to approve the edits before merging. :::AshLin (talk) 14:13, 26 January 2012 (UTC)
- The userspace to mainspace transition can be used to control this. Have two models:
- I recommend that WMF design the next IEP pilot such that there is no mess to clean up. What happened to the English WP community during the Pune Pilot should not happen again.Toryread (talk) 22:53, 24 January 2012 (UTC)
- The report does not mention the of lack of Wikipedia experience -- to the best of my knowledge, both WMF employees have zero mainspace edits under any account. — MER-C (07:22, 24 January 2012 (UTC)), — (continues after insertion below.)
- On the SF team, Schulenburg is an experienced Wikipedian. The India team did not have comparable Wikipedia experience. The India team has since added Shiju Alex, who is an experienced Wikipedian, with emphasis on Indic language Wikipedias.Toryread (talk) 22:53, 24 January 2012 (UTC)
- I'm referring to LiAnna Davis and Annie Lin. MER-C 15:43, 25 January 2012 (UTC)
- My perspective: A person does not necessarily have to be a hard-core Wikipedian to do a good job as a WMF employee or consultant. That being said, I do think that staff working on projects to bring in new editors should roll up their sleeves and do some work in the article namespace on the encyclopedia, so they understand how it works and what the experience is like, as well as the vital role that native Wikipedia community members play in ensuring quality content. However, reducing the issues that came up in the Pune Pilot Project to, "All of this could have been avoided if only Wikipedians were in all the paid roles," is a mistake. Wikipedia experience may be necessary, but it is not sufficient. WMF roles require many skills that have nothing to do with being a competent Wikipedia editor.Toryread (talk) 19:01, 25 January 2012 (UTC)
- I'm referring to LiAnna Davis and Annie Lin. MER-C 15:43, 25 January 2012 (UTC)
- On the SF team, Schulenburg is an experienced Wikipedian. The India team did not have comparable Wikipedia experience. The India team has since added Shiju Alex, who is an experienced Wikipedian, with emphasis on Indic language Wikipedias.Toryread (talk) 22:53, 24 January 2012 (UTC)
- There's a sort of moral hazard going on here: it is not the organizers but the en.wp community that have to bear the negative consequences of any failure. What personal (e.g. monetary) incentives do WMF employees and consultants get for increasing the size/growth of the India/Global Education Program? — MER-C 07:22, 24 January 2012 (UTC) — continues after insertion below
- They don't get monetary incentives for hitting growth targets, to my knowledge. I believe that they do have targets set forth in the strategic plan, and I assume their job performance is evaluated in part against these targets. Barry, please correct me here if I'm wrong.Toryread (talk) 22:53, 24 January 2012 (UTC)
- No one gets monetary performance incentives at WMF. The main quantitative metric WMF looks at is active editors, so in actual fact there isn't really moral hazard on this one. If we piss off editors in the community by forcing them (you) to "bear the negative consequences", we risk losing them (you) which would defeat the purpose of any program aimed to increase active editors. Our program work aims to bring new editors into the community, help them to become productive contributors and do so without creating a negative impact on the existing community. We clearly failed with the IEP pilot, but we can and expect to do better. Barry Newstead (WMF) 00:51, 25 January 2012 (UTC)
- I think MER-C is referring to monetary incentives, generally, not performance based incentives. — Nearly Headless Nick {C} 20:50, 25 January 2012 (UTC)
- No one gets monetary performance incentives at WMF. The main quantitative metric WMF looks at is active editors, so in actual fact there isn't really moral hazard on this one. If we piss off editors in the community by forcing them (you) to "bear the negative consequences", we risk losing them (you) which would defeat the purpose of any program aimed to increase active editors. Our program work aims to bring new editors into the community, help them to become productive contributors and do so without creating a negative impact on the existing community. We clearly failed with the IEP pilot, but we can and expect to do better. Barry Newstead (WMF) 00:51, 25 January 2012 (UTC)
- They don't get monetary incentives for hitting growth targets, to my knowledge. I believe that they do have targets set forth in the strategic plan, and I assume their job performance is evaluated in part against these targets. Barry, please correct me here if I'm wrong.Toryread (talk) 22:53, 24 January 2012 (UTC)
The recommendations are sensible and we expect their implementation. MER-C 07:22, 24 January 2012 (UTC)
- This project, and all the reports and analyses so far, have left me with just one over-riding impression, "We're the WMF. We're doing this anyway. Screw you."
- The first communication I had from the WMF about this mess wasn't any question about IEP, or even an apology for its mis-management, but the first question was whether I supported "the goal of attracting new, quality contributions to Wikipedia?" That's the sort of question any "PR flack" would put up - invert the responsibility and instead make it the en.wp community's need to explain why they're being so obstructive against an obvious benefit (the distance of the actuality from this theoretical benefit being conveniently ignored).
- Holy cow. I made a mistake. I was trying to get a sense of whether the goals set forth in the strategic plan were truly shared by Wikipedians I interviewed. Bad question. Mea culpa.Toryread (talk) 22:53, 24 January 2012 (UTC)
- "The en.wp community will not tolerate any more IEP failures." My concern here is instead whether the WMF will tolerate any further bellyaching from the en.wp community. There is no indication so far that the WMF sees an en.wp community as anything other than a free resource to be used to support their expansionist plans. Expansion is all, not quality. There is no distinction made between an editor with a few thousand edits and a fresh CA or OA with a dozen. Editors, and the content produced so far, just aren't seen as having any ongoing value - they are both only there to be squandered and thrown away, as part of adding yet more edits, no matter what their quality. This included throwing students with zero previous edits straight into FAs. So en.wp editors are not only ignored, but I wonder what measures WMF will use to keep them out of the way in the future? Andy Dingley (talk) 10:23, 24 January 2012 (UTC)
- I was also not at all happy to be greeted with a push poll-esque, leading question at the start of the survey. While Tory did apologize and say she would adjust the wording after I more or less tore a strip off her, I have the strong impression that that question was representative of the goals of this study: to find some loopholes through which the Foundation could continue the IEP. I'm sure there are ways it could be carried on productively, mostly by tearing down and rebuilding, but the survey and these noncommittal results don't give me confidence that that's going to end up happening. The upshot here seems to be "everyone was mean, and well, no one in IEP knew what they were doing, so it's fine to do again a little smaller and with more cheerleading, right?" A fluffernutter is a sandwich! (talk) 15:36, 24 January 2012 (UTC)
- I think that what I'm saying is more nuanced than that. See below.Toryread (talk) 22:53, 24 January 2012 (UTC)
- I was also not at all happy to be greeted with a push poll-esque, leading question at the start of the survey. While Tory did apologize and say she would adjust the wording after I more or less tore a strip off her, I have the strong impression that that question was representative of the goals of this study: to find some loopholes through which the Foundation could continue the IEP. I'm sure there are ways it could be carried on productively, mostly by tearing down and rebuilding, but the survey and these noncommittal results don't give me confidence that that's going to end up happening. The upshot here seems to be "everyone was mean, and well, no one in IEP knew what they were doing, so it's fine to do again a little smaller and with more cheerleading, right?" A fluffernutter is a sandwich! (talk) 15:36, 24 January 2012 (UTC)
- "The en.wp community will not tolerate any more IEP failures." My concern here is instead whether the WMF will tolerate any further bellyaching from the en.wp community. There is no indication so far that the WMF sees an en.wp community as anything other than a free resource to be used to support their expansionist plans. Expansion is all, not quality. There is no distinction made between an editor with a few thousand edits and a fresh CA or OA with a dozen. Editors, and the content produced so far, just aren't seen as having any ongoing value - they are both only there to be squandered and thrown away, as part of adding yet more edits, no matter what their quality. This included throwing students with zero previous edits straight into FAs. So en.wp editors are not only ignored, but I wonder what measures WMF will use to keep them out of the way in the future? Andy Dingley (talk) 10:23, 24 January 2012 (UTC)
- I did one of the email interviews but the first question was a 2-parter: a) attracting new, quality contributions to Wikipedia? and b) attracting demographically diverse contributors to Wikipedia?. The basic gist of my anwers was that the goals were not really the problem. The problem was the way the WMF set about trying to achieve them. I find it odd that there was no mention of the results of that question in the report, given its prominence. I assume most answers were similar to mine, so perhaps it wasn't thought particularly illuminating to state the obvious? Nevertheless, it would have been interesting to know what the general response was "for the record". I imagine only those being interviewed in their capacity as WP editors/admins were asked that, and I find it hard to believe that it was because the WMF seriously thought that we didn't support
motherhood and apple piethose goals. So yes, why was it asked? Voceditenore (talk) 16:38, 24 January 2012 (UTC)- Most people agreed with you, that the goal is not the problem. The problem was in how this particular effort was designed.Toryread (talk) 22:53, 24 January 2012 (UTC)
- I did one of the email interviews but the first question was a 2-parter: a) attracting new, quality contributions to Wikipedia? and b) attracting demographically diverse contributors to Wikipedia?. The basic gist of my anwers was that the goals were not really the problem. The problem was the way the WMF set about trying to achieve them. I find it odd that there was no mention of the results of that question in the report, given its prominence. I assume most answers were similar to mine, so perhaps it wasn't thought particularly illuminating to state the obvious? Nevertheless, it would have been interesting to know what the general response was "for the record". I imagine only those being interviewed in their capacity as WP editors/admins were asked that, and I find it hard to believe that it was because the WMF seriously thought that we didn't support
- I asked the questions. WMF had nothing to do with the question set. Next time I'll focus-group test the questions. A written survey was not part of my contract. I was hired to do talking interviews with you all. Some of you did not want to do that, so to accommodate you, I drew up a question set on the fly. I made a mistake with the first question. Can we let this go now?Toryread (talk) 22:53, 24 January 2012 (UTC)
- PS I would tend to agree with Fluffernutter's characterisation of the "upshot" of the report. Having said that, the evidence at WMF Global Development Midyear report 2011-12 (13 January 2012) and Comparing the Pune and the Cairo Pilot – what we are doing differently (16 January 2012) give some hope that IEP Version 2 might be much better than Version 1. I hope so, because it appears that the round 2 students are going to start on July 4th. Somewhat less confidence-inspiring is India Program/Education Program/Program Design (at least in its present state, last updated today). Voceditenore (talk) 17:26, 24 January 2012 (UTC)
- Yes, we are really focused on the lessons from the IEP and taking action to avoid a repeat. First step is to be more transparent and communicative in the planning stages. We will make more mistakes, I'm sure, but they will be better ones. We really haven't started significant discussions yet on what the next version of the IEP might look like. We wanted to see this report. The redesign will happen in an open process here on en:wp. Barry Newstead(WMF) 00:57, 25 January 2012 (UTC)
- PS I would tend to agree with Fluffernutter's characterisation of the "upshot" of the report. Having said that, the evidence at WMF Global Development Midyear report 2011-12 (13 January 2012) and Comparing the Pune and the Cairo Pilot – what we are doing differently (16 January 2012) give some hope that IEP Version 2 might be much better than Version 1. I hope so, because it appears that the round 2 students are going to start on July 4th. Somewhat less confidence-inspiring is India Program/Education Program/Program Design (at least in its present state, last updated today). Voceditenore (talk) 17:26, 24 January 2012 (UTC)
- Now that you highlight it, MER-C, that "overseeing talk page conversations" is particularly disturbing. It seems that a fundamental that is being missed by the non-editor staff is that talk pages are not optional. Volunteer Wikipedians are responsible for monitoring and responding to comments on talk pages, especially our own. If you're being paid? Well, then you can definitely keep up with 15 minutes worth of reading a day. If the staff do not understand that they are all supposed to be reading all the relevant talk pages, what are they passing on to students? Well, they've passed on to students that talk pages are unnecessary. I should have hammered this point more in the interview, but there was a lot to cover. I had 18 students in my class, as an OA, and not one of them responded to my welcome message or any warnings on their own talk pages, let alone contacted me for help. And unless there's a way to turn off the big yellow message bar, they must have seen those messages, but didn't feel like they were worth responding to, or able to respond to, or something. And the CAs weren't much more responsive either, so I can only assume that this was a flaw in the training module itself. Danger High voltage! 20:25, 24 January 2012 (UTC)
- I agree, talk pages are not optional.Toryread (talk) 22:53, 24 January 2012 (UTC)
- As a very limited or casual editor in this project, I had a similar experience as Danger regarding unresponsiveness of CAs/students. AshLin (talk) 14:13, 26 January 2012 (UTC)
- I agree, talk pages are not optional.Toryread (talk) 22:53, 24 January 2012 (UTC)
- I have to agree with Fluffernutter here: this report does seem to be an attempt to find a way for the program to continue rather than to honestly evaluate whether it should. The report indicates that copyright violation and plagiarism are pervasive in Indian society and the Indian education system, and I am extremely skeptical that a revised program will be able to bring the number of copyvios down to acceptable levels. Even if you ignore the copyright issues the quality of the content submitted was still very poor and many students can't write adequately or have no grasp of any policies or guidelines. If there is to be another program then the obvious question to ask is who will clean up the mess it creates. The program isn't really capable of doing this itself since CAs don't have much Wikipedia experience (several were caught adding copyvios themselves) and OAs are uninterested in doing cleanup work. The result is that the normal editing community gets landed with the job, which is not a sustainable or fair solution. I am puzzled by the recommendation that all students should compose articles in sandboxes as a way of heading off copyvios. As MER-C notes above sandboxes still have to be checked for copyvio and a review of IEP participants' contributions shows that almost all students who contributed to the encyclopedia did use a sandbox for something.
There is one suggestion I keep meaning to post somewhere. Normally if a user is caught violating copyright they get a warning which tells them what they're doing wrong, if they persist after this they get blocked. In the case of the IEP students should have been warned about copyright prior to starting editing, so I think we ought to just dispense with warnings and block students caught adding copyvios. The report does say that students took notice of blocks and modified their behaviour as a result. Hut 8.5 21:17, 24 January 2012 (UTC)
- I also support the block without warning approach. Unlike ordinary newbies, they've already been told not to add copyvios. MER-C 15:43, 25 January 2012 (UTC)
- I don't particularly like the idea of blocking without warning. I get where the suggestion is coming from - in our standard process of mistake-warning-block, they're starting at step two - but I don't think immediate blocking will actually have much effect on the quality of the class's production. What strikes me as more workable would be a specialised "uw-IEPcopyvio" template, which includes information not only on general copyvio policies, but also a reminder that plagiarism will make them fail the class, or about how copyright violation puts Wikipedia in actual legal jeopardy - something to remind them that there are real-world consequences to their actions. A common theme among the students spoken to after the program seems to be that they didn't think of Wikipedia as "real", that they didn't understand that real people were looking at their production, and that their mindset was that copyvios didn't actually matter much. I think we'd get more benefit from offering an onwiki warning highlighting the truth of these facts than we would from just immediately putting the students into a block-based no man's land on their first infraction. A fluffernutter is a sandwich! (talk) 15:54, 25 January 2012 (UTC)
- Immediate blocking probably would have some impact on the quality of contributions if they were made aware of the existence of the rule before they started editing. Not only would it have the obvious deterrent effect but it would convey the message that copyvio is taken seriously, that there are consequences to their actions and that there are people who will punish those who behave badly. Hut 8.5 22:47, 25 January 2012 (UTC)
- I wouldn't support immediate blocking, but next time ( 8-( ) I will certainly blank bad content on sight and warn, whereas first time round I was very deferential to students on the basis that this was assessed work. I no longer care about what mark the students might receive. I will no longer take on any role in educating students as to better working practices - that's the CA's role, and I still see nothing to convince me they'll be any more effective next time round. If the students are detrimental, they'll be treated like the vandals accordingly. Andy Dingley (talk) 01:20, 26 January 2012 (UTC)
- Please also keep in mind that this is a fresh batch of students going through. They are not the previous batches being recycled as per my understanding. So assumptions about their previous experience would not be correct and normal leeway and courtesy which we extend to any new editor should be forthcoming here too. AshLin (talk) 14:13, 26 January 2012 (UTC)
- Yes, normal leeway, same as other editors - so no instant blocking. However the first batch were given far more leeway than any other editor would receive, and I won't repeat that. Andy Dingley (talk) 14:17, 26 January 2012 (UTC)
- I'm aware that these will not be the same students. The point is that according to the plan students will have been educated about copyright violations before they start editing articles. With normal newbies it would be acceptable to block if they persisted after {{uw-copyright}} was placed on their talk page, and a real-life presentation should be more effective than this. Hut 8.5 16:24, 26 January 2012 (UTC)
- Please also keep in mind that this is a fresh batch of students going through. They are not the previous batches being recycled as per my understanding. So assumptions about their previous experience would not be correct and normal leeway and courtesy which we extend to any new editor should be forthcoming here too. AshLin (talk) 14:13, 26 January 2012 (UTC)
- I wouldn't support immediate blocking, but next time ( 8-( ) I will certainly blank bad content on sight and warn, whereas first time round I was very deferential to students on the basis that this was assessed work. I no longer care about what mark the students might receive. I will no longer take on any role in educating students as to better working practices - that's the CA's role, and I still see nothing to convince me they'll be any more effective next time round. If the students are detrimental, they'll be treated like the vandals accordingly. Andy Dingley (talk) 01:20, 26 January 2012 (UTC)
- Immediate blocking probably would have some impact on the quality of contributions if they were made aware of the existence of the rule before they started editing. Not only would it have the obvious deterrent effect but it would convey the message that copyvio is taken seriously, that there are consequences to their actions and that there are people who will punish those who behave badly. Hut 8.5 22:47, 25 January 2012 (UTC)
- I don't particularly like the idea of blocking without warning. I get where the suggestion is coming from - in our standard process of mistake-warning-block, they're starting at step two - but I don't think immediate blocking will actually have much effect on the quality of the class's production. What strikes me as more workable would be a specialised "uw-IEPcopyvio" template, which includes information not only on general copyvio policies, but also a reminder that plagiarism will make them fail the class, or about how copyright violation puts Wikipedia in actual legal jeopardy - something to remind them that there are real-world consequences to their actions. A common theme among the students spoken to after the program seems to be that they didn't think of Wikipedia as "real", that they didn't understand that real people were looking at their production, and that their mindset was that copyvios didn't actually matter much. I think we'd get more benefit from offering an onwiki warning highlighting the truth of these facts than we would from just immediately putting the students into a block-based no man's land on their first infraction. A fluffernutter is a sandwich! (talk) 15:54, 25 January 2012 (UTC)
- I also support the block without warning approach. Unlike ordinary newbies, they've already been told not to add copyvios. MER-C 15:43, 25 January 2012 (UTC)
My main point is, the program has potential, and the first pilot was poorly enough designed that we can't tell yet if the program can work in India. On the ground in Pune, in spite of everything, there is new capacity and a lot of positive energy, and students made some solid contributions to the encyclopedia. The Foundation and the community have both made a big investment in the IEP at this point, and it's not smart to walk away until you do a well-designed, well-regulated phase two pilot. In order to really know the answer, you have to run another pilot. A critically important dimension of any redesign must be: No negative impact on the native editing community.Toryread (talk) 22:53, 24 January 2012 (UTC)
- How many respondents among the en.wp community stated that the IEP should be axed? MER-C 15:51, 25 January 2012 (UTC)
- Out of 10 people interviewed from the en.wp community, two said to cancel the IEP. All 10 said to put the IEP on hold until a solid plan is in place. Four volunteered to play a role in the Phase Two Pilot.Toryread (talk) 19:01, 25 January 2012 (UTC)
My engagement with WMF is over now, so I need to wrap up my role here on the talk page. Thanks again to all who participated.Toryread (talk) 19:01, 25 January 2012 (UTC)
- Thank you Tory. I hope you don't take any of the comments above personally. I hope you understand that this is a passionate project. As you said, your engagement with WMF has come to an end, I wish you the best. Regards. Theo10011 (talk) 04:49, 26 January 2012 (UTC)
- I am finally keeping my thought here. Nobody asked though. Like Tory said "WMF should design the next IEP pilot such that there is no mess to clean up." +1 on this. In the first place How WMF had shown green flag to start the program I don't understand. It was WMF responsibility to see whether IEP was all set to roll or not ?? (correct me If I am wrong). And even I agree we have not enough experienced Wikipedians from India. But my question is "Had IEP tried to get help from all of them ??" This was one of the biggest disadvantage, I don't know if anyone has pointed out already. This should not be happen again. Thanks -- ɑηsuмaη ʈ ᶏ ɭ Ϟ 06:18, 26 January 2012 (UTC)
Question to Barry
editAlthough two global Wikipedians interviewed for this report called for Foundation staff and consultants to lose their jobs over mistakes they made planning and managing the Pune Pilot Project, the evidence suggests that this action would be premature.
Hi Tory, Who are you referring to there, if I may ask. if they have chosen to remain anonymous - I would like to ask if they were indian community members? Regards.
From the message above, I see Tory ending her engagement on the topic, 4 days after beginning, with barely two days of discussion about this above. I understand Tory's position in this, and wish her well. What I would like to ask again now to Barry directly - How valuable was this process? This is the reason I asked why aren't you getting a local, preferably a community member for the post-mortem? You can't expect anyone besides an existing community member to be involved in this process beyond their contract, or paid-term of engagement. This should have been a longer, on-going engaging process. I mentioned this exact point to you months ago, when the first mention of this was made. Theo10011 (talk) 00:37, 26 January 2012 (UTC)
- The engagement with Tory had definitely met my expectations and was a good investment. It is really important that we do effective evaluation of our work and invest in thorough reviews when we have major lessons to capture from pilot projects. Both Tory's work process (wide range of interviews with all stakeholders) and the product speaks for itself. She did a job that would have been difficult for a community member to do. She brought expertise to the process from many years of experience in this sort of work, has some background on our community from the India Chronicles work, and brings a professional distance that helped to provide for some level of NPOV. She also helped us to capture learning quickly while the experience was still fresh in people's mind. The fact that she is departing isn't a problem. The discussion will continue without her, as we (the community and WMF staff) are still here (as you see above) and the next step is to take the lessons and recommendations from her report and integrate it into our future activities. Something that we are already doing as mentioned above. Barry Newstead(WMF) —Preceding undated comment added 17:29, 26 January 2012 (UTC).
- Of course, I read similar statements from you in the report itself, I was wondering if you had a change of heart after seeing the results. The problem I referred to you last year and elsewhere was that you repeatedly kept bringing in individuals from outside the community, outside the geography, to tell you what is wrong. Tory is not a Wikipedian, she is not an Indian - that is the ideology that brought you here; India chronicles was a project she undertook for Wikimedia, which you are using now as her qualification.
- Tory's report is detailed and provides a report for what happened, like an eyewitness - not what went wrong or how to address them; to that extent I would say Kudpung's analysis below goes further than Tory's. I was wondering if Tory's late report and resignation from discussion a day after her submission made you re-evaluate what is apparent to others.
- NPOV is en.wp policy that applies to articles, not reality. It is needed when you write an encyclopedia, not when you hire experts. If you expect to work with farmers in Africa, you don't go and hire a lawyer from Australia to remain objective; he would not know anything about farmers or Africa, that's not NPOV just common sense. One that I have been trying to point to you for a few months. Theo10011 (talk) 00:43, 30 January 2012 (UTC)
More comments
editThank you, Tory - a solid and well-researched report, and sensible recommendations. Kudos to the WMF for commissioning it. Much of it, obviously, was already known to those involved, in fact I was surprised how few surprises there were. Points I think worth picking up:
It's unclear why the number grew so large... When (Mundol) discussed the number with Schulenburg and Newstead, he assumed they would tell him if they thought it was too large. Schulenburg said he had questions about the total number, but he didn't want to step in and take control of key decisions away from the India team... Newstead said he, too, had some misgivings, and he raised them with Schulenburg and Mundol, but they appeared to be aligned and confident, and he let the plan go forward.
Without wishing to rub it in too much, the whole genesis of the disaster (and I do not think that is too strong a word) is here. Tact and giving people their head are all very well, but this really suggests a need for clearer communication and firmer management control.
- I agree that this was the genesis (of course, it is easy to see in hindsight). I would add that it is the single biggest lesson (among many) I've taken from the IEP for myself. I want to build a global team that can have very candid discussions without being concerned about hurting the feelings or stepping on the toes of colleagues, so that we get to the best quality answers. I take responsibility myself for not forcing a harder reckoning on the front end. Barry Newstead (WMFF) 23:12, 30 January 2012 (UTC)
"For most of us, Wikipedia was a black box," said one student. "Our contributions were magically reverted, magically blocked, with no explanation. We were left trying to guess what we did wrong."
It was very noticeable that there was almost no communication from students on their talk pages. I suspect that Wikipedia was presented to them as a website for them to edit, and perhaps was even thought of in those terms by WMF staff. So the students were taken aback, and did not know what to do, when "the website" seemed to fight back, deleting their contributions and lighting up orange bars across their screens. Instead, Wikipedia should be presented as a community which they are invited to join and work with, a complex community of thousands of people with a ten-year history and its own ways of doing things. They should be able to understand that view, because they all have recent experience of joining another community - their college - and having to find out how to get along with its ways of doing things.
"Some simply didn't have the English language skills to write their own content, and copy-paste was their only practical recourse."
I had suspected that was one reason for the copyvios: this confirms it is essential that students who are to contribute to en:wp should have reasonably fluent written English.
...most classes that did the Wikipedia assignment in the U.S. program had fewer than 20 students... The professor quoted above had 180 students in his class.
This large disparity of class sizes means that an Indian professor will need much more support. (It's notable that two of the most successful education programs are those of User:Jbmurray and User:JimmyButler, two extremely hands-on teachers with smallish classes).
The next phase of the pilot should be designed to expressly investigate which parameters and interventions can result in high-quality, copyvio-free content contributions by Indian university students.
Absolutely, and we need to know in advance how that will be done. Unless the proposed measures are extraordinarily convincing, every single edit from the Phase 2 pilot will have to be checked for copyvio - another reason for keeping it small. The hope must be that Phase 2 will provide a road map to an eventual IEP where it is not necessary to check every contribution.
useful report
editI'm glad to read this report and find it useful. Thanks. Sumanah (talk) 21:44, 27 January 2012 (UTC)
Another user's overview
editAn excellent report, although it may have failed in just a few aspects that were not directly part of Tory's brief, and hence some of the background issues were not adequately addressed. I concur entirely with JohnCD's compliments - and lack of surprises.
User retention: If we piss off editors in the community by forcing them (you) to "bear the negative consequences", we risk losing them (you) which would defeat the purpose of any program aimed to increase active editors. Our program work aims to bring new editors into the community, help them to become productive contributors and do so without creating a negative impact on the existing community. (Barry)
For the 3,000 (is it really as few as that?) most active contributors to Wikipedia, either as regular content contributors and/or quality control experts, the volunteer commitment is an altruistic job. The unfortunate outcome is that some regular, experienced editors have now already been pissed off, rather than to retain the free gold the community already has. Moral hazard (MER-C) may not be a totally inaccurate analogy to the way in which the IEP appears to have been designed and planned.
Community support: It has neither been examined nor understood why the WikiProject India with over 500 active members, were possibly involved so little with IEP, and why perhaps their numbers were so few among the handful of experienced editors who volunteered for the clean-up. Many of the editors who were most active on the clean up came from their other preferred subject or maintenance areas to help out.
Tools: Provide a dashboard where the pending articles and tasks can be listed for normal wikimedians to participate too. (AshLin)
A project was started in late September by the WMF in response to a community consensus requirement for far more control over new pages (See mw:New Page Triage, and mw:Article creation workflow) - perhaps the developers could comment on what progress has been made, and if these could be ready in time for new GEP projects and/or phases. Current facts are that backlogs at NPP (New page patrol) have gone over their breaking point, and unfortunately, concentrated efforts often result in a high rate of error. AfC (Articles for creation) are reaching unprecedented levels (over 400 pending pages) that are beyond the control of the few experienced users who work there. Additional patrolling of sandbox drafts would fall into these categories - perhaps CorenSearchBot (Hisham) could be extended to check user pages and AfC for copyvios. Admins appear to be drifting drifting away from their traditional areas of work such as deletion, AfD closure, blocking, admin noticeboards (issues being archived before an admin has taken a look at them). etc) with backlogs beginning to become noticeable, and the number of new sysop candidates has reached an all-time low.
Any influx of new articles from expansion programmes anywhere will add to these problems.
- Indeed, there are broader challenges EN:WP is facing that the IEP failure highlighted. I'd say the grand challenge for us all is to reverse the editor decline by finding ways to accelerate the time it takes for a new editor to become sufficiently proficient that the burden placed on the experienced users is reduced. Combine this with tools development that you mention and we make the whole experience more enjoyable for all. It is ambitious, but it is a meta-challenge to figure out how to help a new editor learn the ropes quickly and figure out where they can get help, so they a) make fewer mistakes that need unusual levels of intervention from the broader community; and b) have the opportunity to really enjoy the editing experience. Note: This is by no means an easy thing to achieve. In the IEP context, v2.0 needs to tackle required student (and ambassador) training as a core design question. Barry Newstead (WMF) 23:35, 30 January 2012 (UTC)
- Do you see the influx as IEP editors as being a solution to editor decline?
- Or would you, as many editors dealing with IEP#1 have expressed the concern, that IEP is a cause of editor decline? Andy Dingley (talk) 00:09, 31 January 2012 (UTC)
- I've been in some off-wiki discussions about this with Kudpung, so I'm definitely biased towards his perspective of things (though I did my own research to confirm my thoughts); I don't know how much patience the Foundation thinks we have for this, but I will definitely say that it's wearing thin. The only thing that prevents me from leaving this morass alone and exclusively sticking to my main content areas (Burmese and Ainu articles) is that with the number of terrible pages coming in, and the very few non-children who are willing to wade into it, people will be overrun and far too many copyvios will sneak in. Given how we've reacted to this, I don't think any second try at this will be at all effective until you've given us a considerable amount of time to flesh everything out and let us calm down.
- Also, I don't necessarily think the best solution here is to make it seem easier to edit here; it's a difficult place to edit, and we should acknowledge that. I only came along in 2010, and I made it just fine because I didn't try to run head-on into everything; I started with something small (trying to remove the word "perished" from articles, as it's unencyclopedic in all but a very few contexts and is extremely annoying to me personally), and gradually worked my way into the community. If I had tried to do everything from the first day, I would have ended up like the narrator and subject of No One at the Bridge (I know it's an obscure reference, but it's the best I can do at almost 1 in the morning). Just on Saturday, at Wikipedia Day in New York, I was trying to show 5 new users how to start on Wikipedia, and I made it a point to say that it takes time to establish oneself as an editor here. Because of that, they're coming in knowing it'll take a while to learn the basics, and they'll make sure they know what they're doing before trying to undertake a large rewriting of an article or creating a new article. That is what we need to do; show people the basics, and then let them slowly learn how to apply them. The people who are introduced into Wikipedia that way and/or independently approach it that way are the people who are sticking around, so it follows that we should be trying to introduce more people that way. The Blade of the Northern Lights (話して下さい) 05:57, 31 January 2012 (UTC)
- Barry Newstead (WMF)- New comments usually go at the bottom. This isn't exactly a mailing list, inline replies break up someone else's comments(read WP:TPO), as well as make it harder to follow for others. New comments should go below someone else's reply for better organization. Theo10011 (talk) 22:16, 31 January 2012 (UTC)
Communications: I have delayed the selection of the communications person primarily because I was unable to find someone from the community or who is intimately familiar with the community and projects. I am hoping that I can conclude the process reasonably shortly with the right candidate. (Hisham)
The IEP needs the help of Wikipedians who are experienced in policy, editing, and the East-West cultural dichotomy. It is possible that such people may not necessarily be available from within the local Indian community.
Loosing jobs: The suggested redistribution of salaried roles may not be as premature as is assumed. Each member of the WMF probably has a defined but finite personal skill set, and it would be a huge benefit to get them together - each experts in their fields of culture, local knowledge, education, software development, Wikipedia editing, legal, and economics - and draw on the experience of regular editors and admins, to listen to what they say rather than relying on statistics, and pool these individual qualities and resources. The numerical evidence is already available in the amount of community comment that is telling us the same things. Certainly it would be most advisable for those designing automobiles to have driven a car in heavy traffic, to know if the roads have been built yet, that the traffic lights are in place, and to listen to the input from those who know about engines, traffic flow, and the availability of drivers.
Cost: The final unasked questions from the volunteer community are: how much did this operation and it's clean up impact on the WMF funds, including the need for an independent assessment, and to what extent has this drain on finances and human resources set back other pressing developments?
Summary: Global education programmes should be set to a clear road map and time frame that take into account all the available and still needed peripheral elements of financial, electronic, and human resources. A cognitive approach to the GEP needs hands-on experience in dealing with today's new pages. These are all points that can be addressed, but we are all aware that messages on talk pages such as this one and the many others that discuss the IEP and USEP here at Wikipedia and WikiMedia, and on instant relay chat, are often misinterpreted rather rather than objective; forward thinking suffers, and the process is too long. Maybe one solution would be to hold a real-life symposium of equal numbers of staff/consultants and editors who were concerned with the IEP planning and its aftermath. Compared to the overall cost of the pilot project, the additional expense might be a net benefit for all future education programmes.
Kudpung กุดผึ้ง (talk) 07:16, 28 January 2012 (UTC)
- I'd like to second Kudpung's question regarding cost. I personally opted not to donate monetarily to Wikipedia this year because of my concerns about the funds sunk into this program and have no intention of donating in the future unless the cost of this program has been explained and any waste be actively eliminated from future iterations. Danger High voltage! 08:52, 28 January 2012 (UTC)
- I'd third kudpung's review of this. It is more insightful than the actual report, which seems more like an eyewitness account about what happened, not an analysis. Danger, I doubt you'd ever get the actual monetary value for individual projects, similar requests have been made on Meta and elsewhere for other projects. Theo10011 (talk) 00:30, 30 January 2012 (UTC)
- I don't like how Barry is characterizing problems with IEP as some larger shortcoming of en.wp. IEP is the indian version of an education program, designed and implemented by WMF, a very small part of en.wp in the larger scheme of things. Shortcomings should rest with the individuals - consultants, staff who designed and ran this. It is only just, you take responsibility for this, and not blame it on some boogeyman problem that has apparently been around longer than the consultants or the program. I'm not sure what blaming your own shortcoming on editor decline accomplishes? figuring out the ambitious "meta-challenge" aside, I am glad to read expert opinions on how to fix the English Wikipedia. Theo10011 (talk) 00:26, 1 February 2012 (UTC)
- I'd third kudpung's review of this. It is more insightful than the actual report, which seems more like an eyewitness account about what happened, not an analysis. Danger, I doubt you'd ever get the actual monetary value for individual projects, similar requests have been made on Meta and elsewhere for other projects. Theo10011 (talk) 00:30, 30 January 2012 (UTC)
- I would like to second Kudpung's position on the cost of the program as well. The analysis is incomplete without allowing for a review of the costs of the program. This is essential from the perspective of understanding the cost-effectiveness and sustainability of paid consultant-led programs, and for determining opportunity cost in terms of resource allocation and volunteer time utilized (and lost). Additionally, the second phase of the pilot program is slated to start soon and from the looks of things the implementation of the IEP pilot and research for why things didn't work out as they should have will consume the entire first year of WMF operations in India. I have previously queried Hisham Mundol on the costs of the program, but did not get a satisfactory response. If the WMF wants to spend donor money raised through projects which this community works so hard for, they need to be made as accountable as volunteers who work across the globe on outreach. — Nearly Headless Nick {C} 10:00, 1 February 2012 (UTC)
I don't necessarily think the best solution here is to make it seem easier to edit here; it's a difficult place to edit, and we should acknowledge that
This is key. When references to WP:COMPETENCE were bought out on mailing list, they were perceived as attacks. Yes the page itself says it might not always be civil to point out WP:COMPETENCE, but then things have to be discussed. Srikanth (Logic) 15:52, 1 February 2012 (UTC)
- I don't quite get why this is so hard for people to grasp; writing an encyclopedia is, by its very nature, hard, and not everyone is suited for it. My extensive NPP experience has taught me the percentage of people who could even potentially become decent editors here is rather low; it's better to admit that and tailor our image so it fits the perception of those who could be decent editors. This isn't a fucking government organization, where we're required to meet standards of diversity; it's a private website, and websites by their very nature are going to attract a certain populace. Instead of fighting it and blasting through veins of gold to reach pig iron, how about we work with our strengths here? But of course, that would require far fewer flights and associated expenses. The Blade of the Northern Lights (話して下さい) 03:16, 2 February 2012 (UTC)
New version of IEP already on Meta
editThe work on the new incarnation of this program seems to have already started here - m:IEP/Welcome
- "Based on learnings from Public Policy Initiative, qualitative analysis of IEP version 1.0, quantitative analysis of IEP version 1.0, we've created a new program design for the India Education Program Pilot version 2.0." - that was awfully quick.
Does this mean the redesign is complete, and we're ready for another round? Theo10011 (talk) 12:14, 31 January 2012 (UTC)
- Absolutely not to your query regarding if the redesign is complete. The page is a draft - as it says at the head of the page. I had asked for the page to be created only so that we capture all the learnings immediately and hence don't lose anything. Hisham (talk) 14:25, 31 January 2012 (UTC)
- It really wasn't a query, especially not directed at you. I think you are jumping the gun again a bit, there still needs to be more discussion; it has been a week since Tory's report. I am less than optimistic about the same people who designed and ran the last IEP, create the new version so soon. Theo10011 (talk) 00:13, 1 February 2012 (UTC)
Flawed
editI've been in close proximity to the India staff, and the individuals running this program. With the sole exception of Frank, there isn't a single person who designed and implemented this program, that I would be confident about editing themselves. They barely know how to edit themselves, who to ask for help, or how, what is the right and wrong practice. This includes, Barry, Hisham, especially Nitika - they are not versed in the en.wp policies and general editing culture, they were hired as consultants by WMF. Nitika, the person directly in-charge of the education program, is still yet to acclimate with general editing practices and learn any guidelines and policies. This is a really important point, when the students and CAs they are overseeing, have the same or more experience than them, that is likely to be a bad start.
I honestly believe, they need more experience and time to learn the ropes first, before they design and run any future iteration of the education program here. I mean this in all sincerity and not as a personal slight against anyone individually. What I did suggest last year elsewhere to Hisham and others, that he needs experienced Wikipedians to design this, who should have the final say. From the report it clearly seems, that the entire chain of command - Nitika -> Hisham -> Barry consists of individuals who don't have the required know-how about either Wikipedia or India, either one or the other(or neither), they shouldn't be making the final decision. I would be strongly opposed to this program continuing this soon in the current form, without the new process being vetted properly first. Theo10011 (talk) 00:58, 1 February 2012 (UTC)
- At the risk of bringing up the WMF's worst nightmare, Kudpung is an extremely valuable resource at your fingertips. He's familiar with academics in Asia and is a very experienced editor here. This community takes months to acclimate to, regardless of whether you've been hired by the WMF or not; there's no possible way anyone could run an effective student program here without being very familiar with the way things tick here. The Blade of the Northern Lights (話して下さい) 01:06, 1 February 2012 (UTC)
India Programs' Communication Consultant
editMundol was actively searching for a communications person for the team, but there was not yet a team member with dedicated responsibility for overseeing talk page conversations with the global Wikipedia community.
Update : User:Noopur28 is announced as wmf:RFP/Consultant, Communications - India Programs.Read the announcement on India list Srikanth (Logic) 16:27, 5 March 2012 (UTC)
- Good to hear that there's a communications person now. Could someone tell us what Noopur's qualifications are? S/he doesn't appear to have done a lot of editing on Wikipedia (mostly automated-style edits, going back only to December), which surprises me a little given how it's been acknowledged that lack of Wikipedia experience among IEP staff has caused problems in the past. Is Noopur experienced with social media and PR? Have they worked in communications before? Do they have a presence in Wikimedia's India chapter(s) or on an Indian language wiki that's deeper than their involvement on en.wp? I'd very much like to be assured that not only is there now a communications person in place, but that the communications person is actually willing and able to communicate with the active Wikipedia community in a way that is informative and comprehensible. A fluffernutter is a sandwich! (talk) 17:37, 5 March 2012 (UTC)
- Noopur is graduating with a Master's in Arts & Aesthetics and has a a triple honors Bachelor's in Media Studies, Literature and Psychology. She has interned with a radio channel and a newspaper. She is a member of the India Chapter. She's involved in en-wp and Commons.
- Noopur's tasks will be to support community building in India through media/PR, social media, digital outreach and some elements of community communications. The responsibility for communicating on program work rests individually with each of us in the India Program team. --Hisham (talk) 11:27, 6 March 2012 (UTC)
- Hi Hisham, this is not quite the job description that had been mentioned before. From Tory's report:
- "The importance of communicating with the global Wikipedia community on talk pages was not emphasized," she [Tandon] said. One factor was that Mundol had been actively recruiting a communications person, but he had not yet found the right candidate, so no one was expressly tasked with talk-page communication.
- Further up this page, you wrote:
- "I have delayed the selection of the communications person primarily because I was unable to find someone from the community or who is intimately familiar with the community and projects."
- Correct me if I'm wrong, but it now appears that Noopur is not going to be tasked with ensuring that IEP communicates adequately with the rest of English Wikipedia, and indeed has very little experience here and no significant talk page contributions at all. Instead, it's still up to each of the IEP team to do so individually. The question is, who takes the ultimate responsibility if that doesn't happen? Voceditenore (talk) 13:57, 6 March 2012 (UTC)
- Hi Hisham, this is not quite the job description that had been mentioned before. From Tory's report:
- Noopur's tasks will be to support community building in India through media/PR, social media, digital outreach and some elements of community communications. The responsibility for communicating on program work rests individually with each of us in the India Program team. --Hisham (talk) 11:27, 6 March 2012 (UTC)
- You mean to say that no one from the India community applied except a newbie with two months of editing experience on Wikipedia? Perhaps you should see that the issue is exactly with your team communicating to prospective users when they themselves do not have any serious editing experience on the English Wikipedia. So where are we going with this after months of deliberations? You are still hiring individuals who have close to zero editing experience. The editing experience of your team has a direct bearing on the performance of the India Education Program, edition two of which is in the pipeline. This is not a judgment on Noopur28, who is a good faith contributor and has done some off-wiki work over the past two months. Please explain why did you think it was a good idea to hire her over more experienced candidates?
- To highlight some contradictions in your decision-making process, I will quote from the page announcing vacancy for a Communications consultant:
- "Ideally, you should have at least 3 years of experience in social media, marketing, communications and PR. You must be familiar with online community building & dynamics as well as outreach."
- To highlight some contradictions in your decision-making process, I will quote from the page announcing vacancy for a Communications consultant:
- Quoting you:
- "I have delayed the selection of the communications person primarily because I was unable to find someone from the community or who is intimately familiar with the community and projects. I am hoping that I can conclude the process reasonably shortly with the right candidate. Hisham" – [1] in response to MER-C's comment – [2]
- Quoting you:
- As a response to MER-C's question:
- "It goes without saying the best person for this role is already an established Wikipedian. You're not hiring a PR flack, are you? — MER-C"
- Clearly, Noopur28 is not "PR flack", but she is not an established Wikipedian by a long shot. If required, I can produce evidence which proves that she is unfamiliar with Wikipedia policies and guidelines, and does not know her way around the project from her 678 edits over the past three months.
- — Nearly Headless Nick {C} 13:57, 6 March 2012 (UTC)
- As a response to MER-C's question:
- Under a thousand edits over a few months isn't "an established Wikipedian", but it does indicate someone with an interest in becoming one - as do their article creations. This is a good start. Someone with an interest in editing, even if not much mileage as yet, is what the role needs. Andy Dingley (talk) 14:29, 6 March 2012 (UTC)
- Well, I'd split the difference. Someone with an interest in editing is better than someone who's never touched Wikipedia, yes, but both are inferior to someone who's experienced with Wikipedia and knows how it and its community work. I sort of expected that, what with the dramatic foul-ups last year, we'd be getting an experienced Wikipedian in this position, but instead we seem to have gotten someone who could learn to use Wikipedia, eventually, instead of someone who knows how to use Wikipedia. Which means she's starting from behind and is likely to be making newbie-mistakes while learning, in a position where she ought to be well past those mistakes and able to facilitate authoritative communication between IEP and the community from the start.
Or, the tl;dr version: they've hired a PR person to handle outreach to people who aren't Wikipedians. They've yet to hire a communications person who will be able to interface with the existing Wikipedia community in a useful way. A fluffernutter is a sandwich! (talk) 14:48, 6 March 2012 (UTC)
- Well, I'd split the difference. Someone with an interest in editing is better than someone who's never touched Wikipedia, yes, but both are inferior to someone who's experienced with Wikipedia and knows how it and its community work. I sort of expected that, what with the dramatic foul-ups last year, we'd be getting an experienced Wikipedian in this position, but instead we seem to have gotten someone who could learn to use Wikipedia, eventually, instead of someone who knows how to use Wikipedia. Which means she's starting from behind and is likely to be making newbie-mistakes while learning, in a position where she ought to be well past those mistakes and able to facilitate authoritative communication between IEP and the community from the start.
- Under a thousand edits over a few months isn't "an established Wikipedian", but it does indicate someone with an interest in becoming one - as do their article creations. This is a good start. Someone with an interest in editing, even if not much mileage as yet, is what the role needs. Andy Dingley (talk) 14:29, 6 March 2012 (UTC)
- There is more to come here. A Global Education Program Online Communications Contractor has been identified and Frank will be announcing the candidate next week when he returns from vacation. --Barry Newstead (WMF)17:11, 6 March 2012 (UTC)
- ...the Global Education team has just announced that Rob Schnautz has been engaged as the Online Communications Contractor. He has been a Wikipedian since 2006 as User:Bob the Wikipedian. He has also been involved in the US Education Program as an Regional Ambassador. He will play an important role in helping us work more effectively with the Wikipedia community. Welcome Rob! Barry Newstead (WMF)19:30, 6 March 2012 (UTC)
Ok this is all very interesting. I can't help but laugh at characterization of Noopur as PR person, or PR flack. She is neither. Theo10011 (talk) 22:00, 6 March 2012 (UTC)
- A social media marketer was exactly what I meant when I wrote my "PR flack" comment. I agree that Noopur isn't a "PR flack" by background, but the job description clearly calls for one. I am deeply concerned that the role of communications contractors will degenerate into merely flak-catching, allowing Schulenburg, Mundol et al to avoid dealing with the community and the resulting scrutiny and accountability:
[Bob] joins the team now to serve as a liaison between the existing English Wikipedia editing community and the Education Program team. This means I'll be less active on talk pages and IRC...
— [3]
- I am also concerned that the poor communication between the WMF in San Francisco and Wikimedia India acknowledged in Tory's report will hamper the effectiveness of Bob's ability to fulfill his role regarding the IEP.
- In the meantime, it's March already and there hasn't been anything substantial posted about the second IEP on en.wp (that I know of). I'd say you have about a month before we get impatient... MER-C 06:03, 7 March 2012 (UTC)
Wikipedia Education Program Extension
editMight be off-topic here, but since many folks who work(ed) on *EP are here, posting it. Invitation to help beta-test the MediaWiki 1.19 extension for the Wikipedia Education Program Srikanth (Logic) 08:40, 6 March 2012 (UTC)
Simple english wiki
editWould it be worth considering to use simple english wiki for these initiatives first? It needs a lot of work, and might be something worth incorporating in the larger design. Theo10011 (talk) 16:15, 15 March 2012 (UTC)
- One of the problems with the IEP was the students' poor command of written English. When they tried to write in their own words instead of copy-pasting, there were not only serious grammatical errors (sometimes to the point of rendering the text incomprehensible), but also very poor knowledge of English punctuation and capitalisation conventions. Simple English isn't "bad" English, and in fact writing an article in simple English which still conveys the facts accurately and clearly requires a high degree of competency in "normal" written English and discourse skills that even many native speakers lack. So no, I don't think that's a solution. A better solution is to ensure that only those students who participate here are proficient in standard written English. Voceditenore (talk) 17:00, 15 March 2012 (UTC)
- There's also the problem that the Simple English Wikipedia is far smaller than the English Wikipedia - 144,472 users have done something here in the last 30 days versus only 779 on the Simple English Wikipedia, we have 1,504 administrators versus 26, and so on. If a large number of students started contributing poor quality content to the Simple English Wikipedia then their cleanup resources would be completely overwhelmed. Hut 8.5 18:31, 15 March 2012 (UTC)
- I don't believe there's any evidence that the IEP student group with poor English skills would be able to contribute more appropriately to simple: any more than they could to en:. Writing "simple English" isn't simply a relaxation of English grammar or an excuse for sloppiness, it requires just as high a standard of English and also an editorial control over how much of its richness and complexity is used. Andy Dingley (talk) 10:49, 16 March 2012 (UTC)
Update on the India Education Pilot
editJust wanted to inform you that we have put up a post about the India Education Pilot here. Please fell free to initiate, advance or follow the conversation on the same page. Thanks Nitika.t (talk) 10:20, 19 June 2012 (UTC)