Wikimedia Technical Engagement/Practices to support maintainers and high-impact tools

Issue tracker: #tool_maintainers
From mediawiki.org


This page concerns a pilot project in 2022/2023 to develop practices to support maintainers and high-impact tools. This project will be prototyped in the tools ecosystem. Practices/programs should apply to other parts of the technology ecosystem. The proposal follows up on the two experimental team projects: Tool Impact Metrics; and Newcomer Support.

The situation[edit]

Data insights

Today we have data on Gerrit contributors through the community metrics dashboard (Bitergia). We know which parts of MediaWiki and extensions have not been touched for 5-10 years and which projects don’t have active code maintainers. We lack similar insights and data on tool developers and maintainers, their areas of focus, the impact of their tools, which resources they use, or if their projects use up-to-date programming languages, frameworks, libraries, etc. This makes it difficult to decide how/where to best support these developers and maintainers or to advocate for their needs.

Tool maintainers and high-impact tools at risk

A key challenge is that often a single maintainer is responsible for a project. When a tool is widely used, this presents a double challenge: It can be a challenging decision for maintainers to pause or stop their engagement. Once they leave the project or move on to other work, communities would lose a critical tool for their work and projects. At the same time, it can be difficult for maintainers to find someone to whom they can hand over a project. This puts communities' workflows or projects at high risk.

The community norms for critical bots and tools provide guidance on key best practices so that a tool can be supported by a wider community. The abandoned tool and right to fork policies ensure that taking over a project is generally possible. However, Wikimedia lacks people and practices that help maintainers share the work with others or to hand over a tool to others. (Ad-hoc) support exists for some tools: For example, other people than the maintainer might respond to questions from tool users on a talk page or improve documentation, tool users might give presentations or design tutorials for a tool they want to recommend to others, or other developers might occasionally contribute.

Project scope[edit]

The focus of the pilot project is scoped around Toolforge tools that perform edits on a wiki. In alignment with the WMF's annual plan commitment, the selected tool for the pilot should be critical for Wikimedia Commons or Wikidata. The path to narrow this down could look as follows:

  • Tools that run on Toolforge -> shared data source (includes tools and bots)
  • Tools that run on Toolforge and perform edits/augment editing on the wikis -> metrics: "wikis a tool is active on + number of edits" (includes tools and bots)
  • Tools that run on Toolforge, perform edits/augment editing on either Wikidata or Wikimedia Commons (includes tools and bots)
  • A specific tool that runs on Toolforge, performs edits/augments editing on Wikimedia Commons, has a high number of edits (either a tool or a bot)
  • A specific tool that runs on Toolforge, augments editing on Wikimedia Commons, has a high number of edits, and a single maintainer (or is otherwise underserved)
  • A specific tool that runs on Toolforge, augments editing on Wikimedia Commons, has a high number of edits, a single maintainer (or is otherwise underserved) and additional evidence that this tool is critical for the Commons community
  • A specific tool that runs on Toolforge, augments editing on Wikimedia Commons, has a high number of edits, a single maintainer (or is otherwise underserved), additional evidence that this tool is critical for the Commons community, and the maintainer would like to work with us.

Note: "high number of edits" isn't the sole indicator for high impact - i.e., tools might be critical for curation workflows that don't necessarily translate into a high number of edits.

Practices to support tool maintainers and high-impact tools at risk[edit]

There are multiple layers of how we can think about addressing this problem:

  • Provide data insights: How can we leverage data about tool maintainers and their work to inform Wikimedia’s programs, project focus, advocacy work, and decisions on the support that the movement could give to these tools and their maintainers? How could we inform developers about risk factors (i.e., outdated library)? As part of the Tool Impact Metrics experiment week project, we looked into the metric wiki project(s) the tool is acting on as well as number of edits. As part of the pilot project, the computing of two additional metrics is planned.
  • Avoid single maintainer burnout: There might be better ways to address this challenge than having one busy tool maintainer take over an abandoned tool or supporting the work from another busy tool maintainer (though it's up to people how much and where they like to contribute).
  • Focus on newcomers: If the ones who are already doing a lot of work on other tools shouldn’t become co-maintainers of even more tools, the target audience for building more sustainable structures around tools is newcomers (i.e. tech-interested tool users).
  • Diversify our thinking on technical contributions: Developing and maintaining a tool requires more than just code and programming skills. However, we tend to primarily think about code and the challenge of finding developers to co-maintain a tool in question. Today a tool maintainer often takes care of all aspects of the tools’ lifecycle: Design and product management, developing and maintaining code, writing admin and end-user documentation, gathering and addressing bug reports, communication, etc. Shifting our thinking from a one-does-it-all model and primary focus on code to an approach where people with diverse interests and skill sets are empowered to contribute to a tool would unlock our potential as a community and open the path for shared responsibilities and increased sustainability.
  • Effective temporary and casual contributions: Supporting a tool doesn’t have to be a tool-life-cycle-long commitment. What type of contributions could be attractive and suitable for newcomers/casual contributors to support tools and maintainers looking for contributors (code, docs, translation, etc.)? What would need to be in place to enable that? What type of initiatives can empower people to contribute effectively temporarily? What type of contribution would make a good fit for the pilot project?
  • Build support networks: The ultimate goal might be to help build support networks around a critical and high-impact tool: A network of people who care about a tool, contribute to it in various ways, and with a set of practices that enable others to contribute effectively occasionally and temporarily. What (existing) initiatives could we support or bootstrap to help distribute the work required to maintain and evolve a tool on more shoulders?

The role of casual/temporary contributions[edit]

Casual/temporary contributions are one way of contributing. Thinking about this might also help us explore how best to break down tasks and ways of providing support. Casual contributions play a significant role in open-source projects. At Wikimedia, some of these come from editors with technical skills; some come from other open communities who join through specific initiatives (i.e., GLAM datathon, Ionian Hackathon in 2022, Google-Code-In, etc.). How can we enable and amplify casual/drive-by or temporary contributions so that they add value to the ecosystem/for a specific tool as part of the pilot project?

  • [In scope for the pilot] What might enable casual/temporary contributions that are helpful? How can a tool maintainer identify and support such contributions?  
  • [Out of scope for the pilot] What type of project enables one-time participants of a hack event/initiative to contribute immediately? How do regular/temporary contributors intersect?
  • [Out of scope for the pilot] What type of projects should we not recommend for casual contributions? (i.e., build a tool, deploy it to Toolforge, and leave ;))
  • [Out of scope for the pilot] What does a path from casual to regular contributor look like? What hooks people in?

Note: For the pilot project, it's recommended to narrow the focus down to 1-2 types of casual/temporary contributions that might make a good fit to support tools and maintainers.

Milestones & timeline[edit]

  • Improved tool data insights (engineering)
    • Q2 & Q3 (Oct 2022–Mar 2023): Metric for up-to-dateness or resources used is computed.
    • Q3 (Jan–Mar 2023): Complete the metrics that will be used in the dashboard.
    • Q4 (Apr–Jun 2023): Up to 2 additional metrics needed by the community are available.
  • Increased tool ecosystem insights (research)
    • Q3 (Jan–Mar 2023): Complete the Cloud Services survey analysis in order to retrieve information useful for the project (tool maintenance).
      • Questions for research:
        • What support mechanisms already exist for a tool and its developers (e.g., for monitoring a tool's activity and health, bot approval practices)?
        • What are things that tool developers would like to have support on (including, but not limited to, code)?
        • What support could tool users imagine providing to a tool they frequently use?
        • How - and in which fields - could casual/temporary contributions be effective and support a tool? What might enable such contributions?
        • Stretch: What kind of metrics tool developers or users would like to have about tools? (correlated to the stretch goal above.
    • Q3 (Jan–Mar 2023): Create a list of prioritized tools based on qualitative & quantitative data retrieved from the dashboard, research, and interviews.
  • Initiate & strengthen connections (outreach)
    • Q2 (Oct–Dec 2022): Connect with others who work in this field or who might be interested - explore if setting up a shared project page is something they'd be interested in.
    • Q3 (Jan–Mar 2023): Explore how to identify a tool/maintainer to run the pilot with - which Commons or Wikidata-related tool might be a good fit? Where do communities or maintainers see a critical need?
    • Q3 (Jan–Mar 2023): Start reaching out to maintainers.
    • Q4 (Apr–Jun 2023): Connect their needs/ideas to the research/work done in the team to prepare this project.
    • Q4 (Apr–Jun 2023): Continue to work with identified partners.
    • Q4 (Apr–Jun 2023): Develop and conduct the pilot together.
  • Path from pilot to a set of practices.
    • Q4 (Apr–Jun 2023): Evaluation, including distilling practices that are agnostic to specific technical areas.
    • Q4 (Apr–Jun 2023): Define next steps.

Intended outcomes[edit]

  • Recommendations on three key practices that might help support contributors and tools at risk and a plan on how to implement these as part of the pilot
  • Metrics for up-to-dateness and resources used are computed
  • One pilot initiative completed jointly with maintainers/supporters of a high-impact/at-risk tool
  • Stretch: Up to 2 additional metrics needed by the community are available
  • Evaluation

Example implementation[edit]

  • The project team has identified the following:
    • user-facing docs
    • someone willing to serve as a backup for restarting the web service in case it’s down,
    • a group of people monitoring and triaging bug reports would help the tool maintainer.
  • Project team develops an outreach plan and supporting guidelines jointly with the maintainer and helps recruit the support group. This could include:
    • Landing page and call to action
    • Communication across multiple platforms and channels
    • Meet-the-maintainer-meeting with brainstorming & mix-match of interests, skills, and needs
    • a sprint with small tasks around this tool
    • Setting up a workboard
    • Tutorials for certain aspects
    • Sharing lessons learned so that others can replicate the parts that worked well, and iterate on the parts that didn't work well

Material (wip)[edit]

Retrieved from "https://www.mediawiki.org/w/index.php?title=Wikimedia_Technical_Engagement/Practices_to_support_maintainers_and_high-impact_tools&oldid=6076890"