Article display preview: | This is a draft of a potential Signpost article, and should not be interpreted as a finished piece. Its content is subject to review by the editorial team and ultimately by JPxG, the editor in chief. Please do not link to this draft as it is unfinished and the URL will change upon publication. If you would like to contribute and are familiar with the requirements of a Signpost article, feel free to be bold in making improvements!
|
The complex, fraught, exciting history of public outreach in Wikimedia
Anyone that has been watching the Wikimedia Foundation for the past few years will have noted the prominence that the Education program has achieved in the organization. This article aims to comprehensively introduce our readers to what this education program entails, past, present, and future.
Early efforts at public outreach
In December of 2004 Angela Beesley, Florence Nibart-Devouard, and Jimbo Wales, then all board members in the still-nascent Wikimedia Foundation, met with representatives from the Open Society Institute to discuss possible monetary contribution to the Wikimedia movement. The OSI (renamed in 2010 to the Open Society Foundations) then as now acted as a grantmaking body for civil society groups worldwide, with a particular focus on the developing world, and so an idea developed out of the meeting to use the grant funds to develop an "outreach program", perhaps focused on the Arabic Wikipedia, which "aim to encourage people to work on their local language versions of the projects, as well as fleshing out their areas of expertise in the larger Wikipedias...through workshops, training sessions, and other events." The suggestion came at an extremely early moment in the movement's history when the Foundation was more concerned with procuring the funds and technical equipment it needed to sustain the project, and so the idea was never acted upon at the time (though the OSF has gone on to fund several other movement initiatives since then, including part-sponsoring the PediaPress printable Wikipedia initiative in 2009 and funding Daniel Mietchen as Wikimedian in Residence on Open Science in 2011) .
Writing for the Signpost in 2007 future board of trustees member Michael Snow stated that:
“ | Given its visibility in academic circles, Wikipedia remains a focus of both praise and criticism from educators [...] the most widely publicized academic response to Wikipedia has been the Middlebury College history department's resolution, passed in January [of 2007], that prohibited students from citing Wikipedia as a source. This type of approach has sometimes been questioned as too draconian, or simply elicited puzzlement at the notion that college students would think of citing any encyclopedia in their papers at all. Covering the story, the New York Times also pointed out how some professors are using Wikipedia in their classrooms, sometimes incorporating Wikipedia editing into assignments. | ” |
"Meanwhile, for its part Wikipedia has to develop its approach toward those coming to the project from the world of academia. Thus last week Wikipedia editors launched a project to assist with the use of Wikipedia in an academic setting — it will be known as WikiProject Classroom coordination. This effort will try to provide guidance and assistance for the various school and university projects that involve Wikipedia." <INT Michael Snow>
This effort was relaunched in late 2008 by Frank Schulenburg, the Foundation's then-head of public outreach <VERIFY TITLE>, including an early reference to the gender gap. <MORE INFORMATION ABOUT THIS ERA AND ITS TRANSITION INTO THE NEXT IS NEEDED> <INT Frank Schulenburg>
Public policy initiative
However the project really took off in May 2010, when the Wikimedia Foundation announced that the Foundation had successfully secured a $1.2 million grant from the Stanton Foundation to fund a public outreach pilot project. The Stanton Foundation, a trust in the stead of former television executive Frank Stanton, had previously funded in the Foundation's Multimedia Usability Project, and was now willing to fund a Foundation outreach experiment on a grand scale, a project which Schulengurg stated that "will enable us to experiment and document best practices, so that academics and educational institutions worldwide can partner with us in helping Wikipedia to continually improve in quality and content". <HOW DID THE EARLY EFFORT LEAD INTO THIS? WHAT WERE THE NEGOTIATIONS?> In announcing the project Sue Gardner stated that:
“ | Wikipedia is a key informational resource for hundreds of millions of people...the Stanton Foundation wants to increase people's understanding of public policy-related issues, and supporting quality on Wikipedia is a great way to accomplish that goal. Meanwhile, the Wikimedia Foundation is keen to experiment with techniques for encouraging subject-matter experts to work alongside our volunteers to improve quality. This funding will enable us to do that, and I am — as always — very grateful to the Stanton Foundation for its support. | ” |
The new initiative's project outline presented an initial 17-month timeline, in which the first task was to be the recruitment of a public policy expert advisory board to help the Foundation draft quality measures, assess baseline quality of then-extant public policy article coverage, and assist in the development and delivery of project educational and training materials. The board's composition, consisting of eight variously-positioned academic policy experts, was announced in early August. At the same time the Foundation began hiring community volunteers in supporting roles, including then-recently retired Signpost editor-in-chief Sageross as "online facilitator", who penned a column in the Signpost elaborating on the initiative's early moves to the community:
“ | We've seen many very successful Wikipedia classroom assignments in recent years, but the common denominator seems to be that the instructor must be an experienced Wikipedian to make it work. Through a combination of in-person instruction, systematic online assistance and mentoring during courses, and high-quality instructional materials, we hope to change that; even a teacher unfamiliar with Wikipedia's culture and practices ought to be able to run a successful assignment. | ” |
Once the team was assembled the first major hurdle was quality metrics: there had to be a way to quantify the extent and impact of the changes generated in articles of United States public policy over the course of the project, and to do that Foundation staff and volunteers needed to comprehensively tag and assess all articles in the domain, working under the umbrella of the newly created WikiProject United States Public Policy. Concurrently Amy Roth, hired as initiative research analyst, was given responsibility for analyzing the effectiveness of the metrics that the WikiProject put into place: a specially composed variation on the 1.0 assessment scale. As Sagesoss explained, this conventional approach—assessment scale tweaks are common amongst large community WikiProjects—was married to a much more radical, publicly accessible article feedback tool placed at the bottom of all articles in the WikiProject's domain (subject to a certain traffic maximum established for technical reasons). <WHAT BECAME OF THIS RESEARCH? [1] Article quality assessment team>
Concurrently, Annie Lin, the initiative’s Campus Team Coordinator, performed the advertising and recruiting activities necessary to put together the pilot project's first campus ambassador candidates, advertising both on Wikimedia and on the campuses themselves. These campus ambassadors would be volunteering two to three hours of their time per week to assist student in their writing and article development on-site, responsible for "face-to-face technical and informational support for new contributors". The project began training its campus ambassadors in August, with the kickoff event taking the form of a three-day workshop at George Washington University with 20 Wikipedians, students, teaching assistants, librarians and professors attending. Assessing the program at this early stage Schulenburg stated that contact with schools and professors at the five schools that the Foundation hoped could pilot the course in the fall and winter of 2010 had been grossly positive, mentioning writing, evaluative, technical, and collaborative skills that the students would be developing alongside the educational materials, online ambassadors, and campus ambassadors that the Foundation was developing to tackle project challenges.
The thirteen courses taught at these five schools were selected and finalized. The last element of the initiative team was the recruitment of online ambassadors, experienced Wikipedians who will "train students on the technical and cultural basics of editing Wikipedia...[and] support newcomers through their first 100 edits"—in community terms, ensure volunteers remain unbitten <ASK A VOLUNTEER FOR INPUT HERE>.
All in all the pilot program claimed work on 190 unique articles, the efforts of the students notarized at the community leaderboard. Speaking of the breadth of the impact that the program had on Wikipedia in their November 2010 outcome assessment the public policy team had to say that:
“ | The students in the program have produced high quality content. In some cases, they have earned accolades from the Wikipedia community (most notably in the form of links from Wikipedia’s front page). Some classes have produced a great deal more content than others, reflecting the different approaches taken by the various professors. Also, fewer students than expected have focused on articles on U.S. public policy; rather, they have tended to focus on an interdisciplinary and broad scope of articles related to public policy.
Given the experimental nature of this project, we are encouraged to see a strong model for collaboration emerging in the current work, even though the differences in approach from one class to another has yielded inconsistent results. Students have responded well to the "Welcome to Wikipedia" brochures, and to the Campus Ambassadors and Online Ambassadors. Pairing an enthusiastic, outgoing Wikipedia newcomer with an experienced Wikipedian proved to be the ideal combination for Campus Ambassadors in each classroom. |
” |
Looking back on the pilot project today the general theme of the learning outcomes was one of focus. Though the promise of "dozens, possibly more than 100, public policy students (mostly graduate students) and their professors" that Sageross had outlined in earlier Signpost articles had been met, easily satisfying the Stanton Foundation's terms for continuation of the program, despite the report's assessment of contributions being broadly good the pilot had nonetheless produced an unexpected impact range. The amount of material and the quality of material varied widely course-to-course, with some courses mobilizing students significantly better than others; furthermore it was found that professors tended to encourage and students tended to work on broader topical areas than the program initially expected, with only 81 of the 190 articles improved falling into what the assessment team had assessed as the domain of United States public policy articles (of which 61 were improved and 20 were completely new). Learning outcome recommendations included making mandatory expectations in the assignment clearer, setting assignments that last at least half the academic term, new guidelines for course design and sample syllabi, a "Wikipedia Teaching Fellowship" awarded to professors who agree to follow guidelines and who "share their assignment with us [the Foundation] prior to the beginning of the term".
The Stanton Foundation grant followed the policy program through a second semester significantly wider in scope and impact than the one before it. The original topic, public policy, remained broadly the same, but contrary to initial report recommendations topicality requirements were relaxed, and the 33 courses hosted on the English Wikipedia under the auspices of WikiProject United States Public Policy in the 2010–2011 spring semester included such topics as Arab Media, Professional and Technical Editing, and Business Data Communications. There was even a course taught at Syracuse University explicitly titled Wikipedia and Public Policy, the topic of which was "the history and economic impact of Wikipedia and social media...[and] how Wikipedia has changed how we learn, work, and communicate."
Aude covered covered the program wrap-up for the Wikipedia in Higher Education Summit, held July 8–9, 2011 at Simmons College in Boston, for the Signpost. The program was attended by approximately 120 professors, students, ambassadors, and staff attached to or with an interest in the Public Policy Initiative, and it was here that a full review of the program's first year, as well as the discussion of the program's future, was presented. After an opening keynote by national archivist David Ferriero community staff members presented the final tally: 24 universities, 47 courses, and 800 students participated in the initiative, approximately 200 students in the Fall and 600 in the Spring. Sue Gardner presented the dead-tree equivalent of the initiative's work—11 packs of printer paper—and in her proceeding keynote speech spoke of the place that the public policy initiative held in the Foundation's strategic plan; as Aude explained:
“ | She [Sue Gardner] explained that the important goal for Wikimedia now and in the foreseeable future is increased participation, along with improved quality which in turn attracts more readers—a small portion of whom become editors who, in turn, help strengthen quality of the content and create a virtuous circle. In addition, increased diversity is a key goal of the Foundation, including closing the gender gap and better geographic representation. Already, the initiative has had much higher female participation rates than routine editing, and, by expanding the initiative to India, Brazil and elsewhere, the Foundation hopes to stimulate better geographic diversity. | ” |
The subject of gender ratio and geographic diversity were of critical importance. 46% of campus ambassadors were female, bellying the wider trend in the Wikimedia community—as the Signpost first broke in February of 2011, major media coverage of the community's yawning gender gap, long a cause for private concern with first Jimbo Wales and then Sue Gardner, had by this time become a leading organization concern at the Foundation. Furthermore the vast majority of edits to Wikimedia projects come from Anglophone countries, the majority from the United States in particular, and the Foundation hoped that by unveiling its global education program the Foundation could purposefully increase the movement's impact worldwide.
The final public policy pilot analysis was released in November 2011. The report stated that the program turned out to be "one of the most successful outreach projects ever undertaken by the Wikimedia Foundation", and, taking cues from outcomes presented at the summit, went on to say that:
“ | All told, more than 800 students contributed 8.8 million characters of content to the Wikipedia article namespace – the equivalent of 5,800 printed pages. And even better, the content students added was of high quality: Public policy articles that students worked on improved an average of 64 percent. Most important, because of the Stanton Foundation's commitment to establishing a support system for professors who want to use Wikipedia assignments in their classroom, professors in public policy and other disciplines around the world will continue to assign students to edit Wikipedia articles, improving article quality for years to come.
Up until this point, many of our projects to recruit subject-matter experts like academics produced only short bursts of activity on Wikipedia. The Public Policy Initiative has changed that. Not only are we improving the quality of Wikipedia every term, we are addressing some of the big questions facing the Wikimedia movement: recruiting new editors, closing the gender gap, and welcoming new users are all key facets of our strategic plan for the next five years. The Public Policy Initiative started making inroads on these movement-wide problems. |
” |
Next steps
This last point was a particularly important one, as even with the pilot program still ongoing work had already begun to extend the work being done to the next semester. Lin remained active in soliciting further volunteers for the project throughout the academic semester and an important topic of discussion in almost every meeting conducted under the auspices of the program was scalability—where do we go from here? The decision was made that the initiative needed to advance from a Wikimedia staff to a volunteer-driven program, and a steering committee <INT steering committee> was created to investigate how to do so. Even as the Foundation published their midterm update the topic of Wikipedia in education again recurred in public discussion (see concurrent Signpost coverage), but the relative success of the pilot seemed to signal bright things ahead.
At the same time the program went international. Five courses were organized and taught in Brazil, five in Canada, and twenty-four in India. These programs did not hold any allegiance to the original policy focus that the United States program was beholden to, and instead covered a wide range of subjects, everything from the history of the Roman Empire to the psychology of language to discrete signal processing (a topic in electrical engineering).
Community feedback
So far in this article we've focused on the Foundation's narrative of the events, but .
- Introduce negative community feedback.
- Why were contributions being counted in bytes in the first place? Were the quality metrics really accurate? (see comments in Sage's articles)
- Opinion essay on the program: Wikipedia:Wikipedia Signpost/2011-12-12/Opinion essay
- Sven's comment in the opinions is worth highlighting.
- Doc Jame's stats on value of student edits, also in the comments to the op-ed.
- The Pune disaster. Summarize out extensive special report: Wikipedia:Wikipedia Signpost/2011-11-07/Special report.
Interviews...
- Ragesoss - early communications and background
- Drmies and Kudpung were involved in the early efforts and could say more about what was required of online volunteers.
- jbmurray (always had this view) and Mike Christie, who advocated in 2011 a shift toward academics instead of students as the focus, not students.
- LiAnna
- Research:Quality_of_PPI_editor_work