Grants:IEG/WikiProject X/Renewal/Final

Add links
From Meta, a Wikimedia project coordination wiki



Welcome to this project's final report! This report shares the outcomes, impact and learnings from the Individual Engagement Grantee's 6-month project.

Part 1: The Project[edit]

Summary[edit]

In a few short sentences, give the main highlights of what happened with your project. Please include a few key outcomes or learnings from your project in bullet points, for readers who may not make it all the way through your report.

  • The CollaborationKit extension completed its first phase of development. Following a security review it was deployed to Test Wikipedia, which is considered to be a production wiki.
  • At the 2017 Wikimedia Developer Summit, James Hare worked with session attendees on designing a system to integrate automatically generated task recommendations with manually submitted ones.
  • Wikipedia Requests grew in adoption, with over 1,400 article-related tasks submitted, including a large donation of a worklist related to Dalit history developed as part of the Whose Knowledge project.
  • As a huge boon to the Women in Red project's productivity, our Python contractor developed a new automated metrics reporting system. This drastically reduced the amount of work the project participants needed to embark, making their efforts more efficient.
  • However, Women in Red faced challenges with the WPX UI template system, owing to its tremendous growth and unique scaling difficulties. In 2018 they partially discontinued the use of the UI while still using the membership features and automated metrics reports. This is an experience worth learning from.

Methods and activities[edit]

What did you do in project?

Please list and describe the activities you've undertaken during this grant. Since you already told us about the setup and first 3 months of activities in your midpoint report, feel free to link back to those sections to give your readers the background, rather than repeating yourself here, and mostly focus on what's happened since your midpoint report in this section.

The midpoint report covers activities from January 2016 to May 2016. This report will cover activities since then.

The burn down chart for CollaborationKit, from May 2016 to May 2017.
  • CollaborationKit
    • After we made the decision to build out CollaborationKit, we did the initial work to build the extension, but were affected by staffing shortages. To mitigate this, we brought on Brian Wolff as a consultant to help with development. We also gave Isarra additional engineering duties in addition to her design work. With full staffing, we were able to continue with project implementation.
    • The three of us (James Hare, Isarra, Brian Wolff) met in Washington, DC in July 2016 for a three-day team meeting to discuss the product roadmap. At the conclusion of the meeting, the three of us were aligned on how to best implement CollaborationKit, and we were able to move forward. Our demo wiki is here: https://wpx.wmflabs.org/w/index.php/Main_Page
    • CollaborationKit is based on the earlier WPX UI template system for WikiProjects on the English Wikipedia, but with closer integration into the MediaWiki software. The extension gives people the ability to create project hub pages as well as easy-to-edit worklists. The link above is a good demonstration of the software.
  • Wikipedia Requests
    • Most Wikipedia Requests work post-initial launch focused on some large projects, including coordinating with Whose Knowledge on a contribution of tasks related to Dalit history and a project on facilitating edits related to conflicts of interests (COI).
    • We are very happy with the Dalit history project; more in the Outcomes section. As for the COI edit request project, we were never able to give it the prioritization it needed, but we still believe the demand is there.
  • Reports Bot
    • We brought on Ben Kurtovic, The Earwig, as a contractor to support reports bot development. He focused on rewriting the backend for Reports Bot to make it work better and support more languages in the long term.
  • WikiProject Women in Red
    • We worked closely with WikiProject Women in Red to improve their workflow, given their tremendous rate of growth both in terms of participation and number of tasks to keep track of.
    • As a project interested in adding more articles about women and their works to Wikipedia, they kept track of their work. They did this manually and it took a lot of time. Our contractor Ben Kurtovic developed an automated metrics report to track new articles per month. This dramatically reduced the amount of manual reporting work by project members, allowing them to focus their energy on other things.
    • Ahead of their Women's History Month campaign in March 2017, we assisted with a re-design of their landing page to make it more concise. Resulting from this is the new Worklists page. While this was a one-off effort that affected one project, It's worth considering how this can be adapted to other projects as they scale up in activity. The guiding question is: how do you present project details without overwhelming people?

Outcomes and impact[edit]

Outcomes[edit]

What are the results of your project?

Please discuss the outcomes of your experiments or pilot, telling us what you created or changed (organized, built, grew, etc) as a result of your project.

  • CollaborationKit
    • Working with the security team we passed security review in 2017 and deployed the code to the beta cluster and subsequently to Test Wikipedia. Test Wikipedia is a production wiki, so we successfully deployed CollaborationKit to production, meaning it can now be enabled on any Wikimedia wiki. However...
    • There are still some outstanding issues, including UI-related particulars that were brought up during a design review at the Wikimedia Hackathon in Vienna. Unfortunately we have not had time to figure out how to get CollaborationKit ready for real-world use.
  • Wikipedia Requests
    • Wikipedia Requests started as an experiment to help support WikiProject Women in Red and related efforts. However, as of writing, over 1,400 tasks have been submitted into the system, suggesting significant interest in the system. 114 unique users have interacted with Wikipedia Requests.
    • One of the significant users of Wikipedia Requests was by WikiProject Video Games, which independently adopted the system as part of their workflow. We made it possible to embed Wikipedia Requests lists onto pages on English Wikipedia, and this template has been transcluded onto 24 pages.
    • There was a large donation of tasks related to Dalit history, courtesy of the Whose Knowledge project. This allows the Dalit History Month project to keep track of work, including automatic reporting of complete and outstanding tasks. The requirements of this project also meant an overhauled system for mass imports – it works a lot better now!
    • When I worked for NIOSH I submitted many requests related to occupational safety and health. Rather than keep this to-do list siloed within our WikiProject, we wanted to include it as part of this broader index so that others can find these tasks and help us work on them.
  • Reports Bot
  • WikiProject Women in Red
    • WikiProject Women in Red, supported by WikiProject X, is a thriving and successful project with 264 active members as of April 2018. They've succeeded in moving the needle: while 15% of English Wikipedia biographies were about women, as of April 2018 that number is now 17.54% – a significant improvement.
    • While the project is successful, the WPX UI has had trouble keeping up. In 2018 the decision was made to only keep some of the WPX UI templates that were originally installed in 2015. The membership feature and the automated metrics report remain, but we are interested in knowing more about how WPX UI did not meet Women in Red's needs. Does it not work for highly active projects, or when there is a lot of content to be organized?

Progress towards stated goals[edit]

Please use the below table to:

  1. List each of your original measures of success (your targets) from your project plan.
  2. List the actual outcome that was achieved.
  3. Explain how your outcome compares with the original target. Did you reach your targets? Why or why not?

Core[edit]

Planned measure of success
(include numeric target, if applicable) 		Actual result Explanation
Back-end Infrastructure: Re-engineered backend that works reliably (less than one script failure per week) Done Thanks to the work of Ben we have a robust and reliable Reports Bot system.
Back-end Infrastructure: Wikimedia project-agnostic architecture Done
Back-end Infrastructure: Implementation of error logging Done
Back-end Infrastructure: Implementation of tracking mechanisms that can measure work done via on-wiki tools Not done We instead decided to focus on other projects, such as providing article metrics for WikiProject Women in Red.
Front-End Interface: System for creating and registering standardized modules that can be deployed on demand on a WikiProject using WPX UI Not done We instead focused on building CollaborationKit.
Front-End Interface: Deployment of DIY tools for implementing WPX UI Not done While the code is written, it has not been deployed to any non-test Wikimedia projects.
Front-End Interface: User-friendly documentation of WPX UI template/module stack for end users Not done With the work on CollaborationKit ongoing, support for the existing WPX UI templates was dropped.
Front-End Interface: Comprehensive documentation of WPX UI template/module stack for technical audiences Not done With the work on CollaborationKit ongoing, support for the existing WPX UI templates was dropped.
English Wikipedia: Use of WPX UI design and tools on at least 20 WikiProjects, including specifically to coordinate the Art+Feminism campaign 15 on English Wikipedia + 1 on Bangla Wikipedia + 0 on Farsi Wikipedia + 1 on Thai Wikipedia = 17 total. Almost! There was no active marketing of WPX UI because of the ongoing work on CollaborationKit. That said, we are pleased that others have taken to porting the code to other wikis, including Bangla, Farsi, and Thai Wikipedias.
English Wikipedia: Creation or improvement of at least 100 articles through tools At least 88 We never developed the capability of tracking these kinds of metrics. However, we know that 88 article-related tasks have been marked "complete" in Wikipedia Requests.
English Wikipedia: Support the development of reports and worklists based on arbitrary lists of pages (as opposed to current system which is tied heavily into WikiProject categorization) Not done More focus was on improving reports bot's core functionality.
English Wikipedia: Fully automated article development workflows on the Women in Red project Not done We needed to prioritize work elsewhere
Wikidata: Identification of needs of Wikidata community Not done Some initial work was done, but nothing actionable.
Wikimedia Commons: Identification of needs of Wikimedia Commons community Not done Some initial work was done, but nothing actionable.


Stretch[edit]

Planned measure of success
(include numeric target, if applicable) 		Actual result Explanation
Back-end Infrastructure: Interactive worklists rendered on-wiki through a JavaScript client to a service on Tool Labs, eliminating the need to store lists within the revision table Not done
Back-end Infrastructure: Implementation of "to-do list" functionality where tasks are tracked on-wiki, can be marked as "done," etc. Done Wikipedia Requests was launched and has been used to submit over 1,400 article requests.
Front-end Infrastructure: Development of portable WPX UI kit that can be readily deployed to other wikis Done The CollaborationKit extension is designed to be easier to use and usable on more than just English.
Front-end Infrastructure: Adoption of system to facilitate volunteer translation Not done
English Wikipedia: Integration of Content Translation extension into worklists Not done
English Wikipedia: Implementation of reference recommendation system tied into Wikipedia Library and/or other discovery services Not done
Wikidata: Development of a proof of concept for a WikiProject on Wikidata Not done
Wikimedia Commons: Implementation of image identification / structured data system for media holdings on Wikimedia Commons Not done
Wikimedia Commons: Development of a proof of concept for a WikiProject on Wikimedia Commons Not done


Think back to your overall project goals. Do you feel you achieved your goals? Why or why not?

Global Metrics[edit]

We are trying to understand the overall outcomes of the work being funded across all grantees. In addition to the measures of success for your specific program (in above section), please use the table below to let us know how your project contributed to the "Global Metrics." We know that not all projects will have results for each type of metric, so feel free to put "0" as often as necessary.

  1. Next to each metric, list the actual numerical outcome achieved through this project.
  2. Where necessary, explain the context behind your outcome. For example, if you were funded for a research project which resulted in 0 new images, your explanation might be "This project focused solely on participation and articles written/improved, the goal was not to collect images."

For more information and a sample, see Global Metrics.

Metric Achieved outcome Explanation
1. Number of active editors involved 0 We are measuring over such a long time period that I am not sure "active editor" is a meaningful measure here.
2. Number of new editors 0 This project was not designed to recruit new editors.
3. Number of individuals involved 815 Posts to WikiProject X talk page, people signed up for WikiProject X newsletter, members (active and inactive) on WikiProjects using WPX UI, and users of Wikipedia Requests, all de-duplicated.
4. Number of new images/media added to Wikimedia articles/pages 0 We do not have the capability to measure this outcome.
5. Number of articles added or improved on Wikimedia projects 88 This is the number of completed requests in Wikipedia Requests. This doesn't include any other WPX tools.
6. Absolute value of bytes added to or deleted from Wikimedia projects 0 We do not have the capability to measure this outcome.


Learning question
Did your work increase the motivation of contributors, and how do you know?

Yes. Our work focused on identifying how Wikipedia's user interface was making it harder for editors to do their work. By automating work such as metrics reporting, we free up volunteers' time and make it easier to show contributors the impact they are having.

Indicators of impact[edit]

Do you see any indication that your project has had impact towards Wikimedia's strategic priorities? We've provided 3 options below for the strategic priorities that IEG projects are mostly likely to impact. Select one or more that you think are relevant and share any measures of success you have that point to this impact. You might also consider any other kinds of impact you had not anticipated when you planned this project.

Option A: How did you increase participation in one or more Wikimedia projects?

Option B: How did you improve quality on one or more Wikimedia projects?

Option C: How did you increase the reach (readership) of one or more Wikimedia projects?

This work helped to increased participation on Wikimedia projects by creating a novel form of participation through Wikipedia Requests. Entries in Wikipedia Request are like proto-articles; proto-articles might not have a home on Wikipedia, but they can be submitted to Wikipedia Requests, and a volunteer can expand on that initial effort to build an article. Prior to Wikipedia Requests, the only way to submit article ideas would be through various scattered wiki pages, with little hope for discovery. Centralizing these requests helps improve Wikipedian productivity, which in turn allows them to help improve the quality of Wikipedia.

Project resources[edit]

Please provide links to all public, online documents and other artifacts that you created during the course of this project. Examples include: meeting notes, participant lists, photos or graphics uploaded to Wikimedia Commons, template messages sent to participants, wiki pages, social media (Facebook groups, Twitter accounts), datasets, surveys, questionnaires, code repositories... If possible, include a brief summary with each link.

Code:

Demos:

Newsletter issues

Learning[edit]

The best thing about trying something new is that you learn from it. We want to follow in your footsteps and learn along with you, and we want to know that you took enough risks in your project to have learned something really interesting! Think about what recommendations you have for others who may follow in your footsteps, and use the below sections to describe what worked and what didn’t.

What worked well[edit]

What did you try that was successful and you'd recommend others do? To help spread successful strategies so that they can be of use to others in the movement, rather than writing lots of text here, we'd like you to share your finding in the form of a link to a learning pattern.

What didn’t work[edit]

What did you try that you learned didn't work? What would you think about doing differently in the future? Please list these as short bullet points.

I don't think projects that rely entirely on part-time staffing work. I think for large ambitious projects like these, the staff need to be able to work full time to maintain concentration and get things accomplished on schedule. It is worth thinking about grantees as professionals and supporting them as such.

Other recommendations[edit]

If you have additional recommendations or reflections that don’t fit into the above sections, please list them here.

There's this tension in grantmaking to spell out a compelling vision while submitting a proposal that is practical and useful. One idea I have is to separate these into two separate exercises, with one document that describes a long-term vision for a project and then a grant proposal that borrows a slice of it. That way, people are encouraged to think their ideas all the way through while focusing on a subset of that to work on.

Next steps and opportunities[edit]

Are there opportunities for future growth of this project, or new areas you have uncovered in the course of this grant that could be fruitful for more exploration (either by yourself, or others)? What ideas or suggestions do you have for future projects based on the work you’ve completed? Please list these as short bullet points.

  • From Reports Bot and SuggestBot we have automated recommendations for tasks. From Wikipedia Requests we have ideas submitted from humans based on their knowledge and experience. These are separate workflows. In the future, how do we bring them together into one interface?
  • Movement organizers are not well served by Wikimedia's technical infrastructure, and people trying to organize work on-wiki (such as the Women in Red project) have limited tools to work with. We need to take this group of people seriously and invest in tools to let them do their work more efficiently.

Part 2: The Grant[edit]

Finances[edit]

Actual spending[edit]

Please copy and paste the completed table from your project finances page. Check that you’ve listed the actual expenditures compared with what was originally planned. If there are differences between the planned and actual use of funds, please use the column provided to explain them.


Expense Approved amount Actual funds spent Difference
Product Management $10,000 $10,000 $0
Design $12,500 $12,500 $0
Software Development (for contractors) $7,500 $7,500 $0
Total $30,000 $30,000 $0


Remaining funds[edit]

Do you have any unspent funds from the grant?

Please answer yes or no. If yes, list the amount you did not use and explain why.

If you have unspent funds, they must be returned to WMF. Please see the instructions for returning unspent funds and indicate here if this is still in progress, or if this is already completed:

  • There are no unspent funds.

Documentation[edit]

Did you send documentation of all expenses paid with grant funds to grantsadmin(_AT_)wikimedia.org, according to the guidelines here?

Please answer yes or no. If no, include an explanation.

  • Yes.

Confirmation of project status[edit]

Did you comply with the requirements specified by WMF in the grant agreement?

Please answer yes or no.

Yes.

Is your project completed?

Please answer yes or no.

I think it should be considered complete for the purposes of this grant, but I would argue the work is far from over.

Grantee reflection[edit]

We’d love to hear any thoughts you have on what this project has meant to you, or how the experience of being an IEGrantee has gone overall. Is there something that surprised you, or that you particularly enjoyed, or that you’ll do differently going forward as a result of the IEG experience? Please share it here!

  • It was a very interesting experience. It challenged me to develop skills I didn't have, and I had to constantly challenge my assumptions. I wish I could have had more time to work on it, but I think with what (relatively) little time I had, I was able to accomplish something useful. I am very eager to see other people take on this work, and I ready to help advise. harej (talk) 08:47, 14 April 2018 (UTC)
  • I would definitely second 'interesting' with regards to this. For my own part, I would say I learned a lot about the development process with regards to leveraging new technologies - working with content models and creating interfaces using that proved to be highly illuminating in general - but we all also went in with so little foreknowledge of what what such technical projects truly entailed. Far more than the previous round, this one has taught me the importance of staging, why with these projects, minimum and stretch goals are so important. After the learnings outlined above, I'd say the biggest issue we had here was not knowing where to end the project as part of this grant, because though our budget basically only covered work through part of 2016, we simply weren't done, and did not have deliverable results, even well into 2017. This proved to be a major stress point for us and put a serious damper on the entire thing to the point where, while we still felt obligated to see it through to meet what felt like a more reasonable subset of our goals, we simply did not have the resources in any sense to continue. Thus, it's clear that we must actually end this in order to move forward with the project, and accept the goals we did meet. For me personally, this is probably the most important realisation to have come out of it, which I fully intend to bring forward to future projects. -— Isarra ༆ 16:00, 14 April 2018 (UTC)
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Grants:IEG/WikiProject_X/Renewal/Final&oldid=17939719"