Wikimedia Technical Engagement/Goals

Technical Engagement's main focus in FY 2022/23 lies on making our existing services and programs stronger in order to increase sustainability of the services that we provide to the Wikimedia movement.

The majority of the work that Technical Engagement does is foundational work. This includes maintaining and evolving Wikimedia Cloud Services including support for Striker, Paws and Quarry (Cloud Services). It includes maintaining and evolving Toolhub and the Developer Portal, coordinating the Techblog, and managing service requests around the tech community metrics dashboards (Production or third party services). It encompasses evolving Wikimedia's technical documentation and providing support for users of Phabricator, Toolhub and Cloud Services (Technical Documentation and Support). Finally, it includes coordinating initiatives and programs like Small Wiki Toolkits, Hackathons, Coolest Tool Award, Outreach- and internship programs, contributions to other open source projects, or advocacy for best FOSS practices (Outreach, Community and Capacity Building).

Planning framework

 * Strengthen foundational work to make our existing services and programs stronger (80%).
 * Focus/Experiment: Space for experiments to explore ideas and cross-functional focus projects to move the needle in a specific field (max 20%).
 * Increase synergy effects and reduce SPOFs through cross-training and shared responsibilities.
 * Broad annual direction for the different services and initiatives. Multi-quarter timelines for multi-quarter initiatives. Detailed planning and goal setting in quarterly planning cycles.

Cloud Services


Release Toolforge Build Service - T194332 (multi-quarter project)
 * Release community beta T267374
 * Multiple language stacks T325799

Find a sustainable local storage solution for cloud-vps T324729

Move off from icinga and introduce alertmanager T328502 (multi-quarter project)

Support 'unmanaged' projects in cloud-vps T326818

Modernize openstack rbac T330759

Move WMCS dashboards to grafana.wmcloud.org T333568

Create CloudLB PoC in dev T324992 (multi-quarter project)

Upgrade Ceph
 * Upgrade Ceph and Host OS T306820 T309789

Cloud Services
Implement Kubernetes as a Service (Magnum) - T280792 ✅
 * Migrate PAWS to Magnum T308873

Ensure alerting and on-call systems are sustainable and scalable - T313444 (multi-quarter project)
 * Enable alerts and pages from cloudvps/toolforge/paws/quarry - T313031

Improve integration with WMCS of SRE cookbook automation - T319401 ✅
 * Collaboration with Infrastructure Foundations team
 * Finalize cloud-cumin setup and utilize for WMCS cookbook runs

Release Toolforge Build Service - T194332 (multi-quarter project)
 * Release community beta T267374

Upgrade ToolsDB - T301949 ✅
 * Finish migration to new ceph backend

Continue GridEngine deprecation (multi-quarter project)
 * Deprecate GridEngine Docs T326756 ✅
 * Planned Toolforge jobs improvements T327254 ✅

Upgrade OpenStack ✅
 * Upgrade cloud vps openstack to Openstack version Zed - T323086

Upgrade Ceph
 * Upgrade Ceph and Host OS T306820 T309789

Shut down final remaining bare-metal NFS servers T301280 ✅
 * Update maintain-dbusers.py to operate in virtualized environment T303663

Create CloudLB PoC in dev T324992 (multi-quarter project)
 * Collaboration with Infrastructure Foundations team
 * Test implementing cloud load balancers in development ✅

Clarify database as a service offerings ✅
 * Update Documentation to clarify when to use Trove and when to use Toolsdb T326754
 * Audit Toolsdb users for potential migrations to Trove T291782

Research and decide on new Openstack deployment paradigm T326758 ✅
 * In support of adding new features, and creating a sustainable, maintainable and repeatable cloud deployment

Research supporting unmanaged Cloud-VPS projects T326818 ✅
 * Identify first users, develop support for non-puppet managed VMs

Analyze and share results from Cloud Services Annual Survey - T322500 ✅
 * Transform and massage the data
 * Analyze the survey responses
 * Publish and present the findings

Production or third party services
Improve Toolhub record content coverage and consistency - T327558
 * Collaborate with the local community to improve documentation of tools which support the workflows of Wikimedia Commons - T327559
 * Implement a data feed for Toolhub based on descriptions of tools on commonswiki - T327560
 * Mentor two Outreachy interns in the project “create a web application for editing Toolhub records” (workboard)
 * Increase the number of tools that adhere to our published best practices [Tool Sweep; review at least 30 tools each month]. - T327398

Technical Support and Documentation
Create on-wiki content based on Small Wiki Toolkits workshops conducted in 2022 - T327282 Improve Cloud Services documentation - T321582 Improve Pywikibot documentation - T320625 Add features to documentation (collection) metrics tool - T320626
 * Prepare and migrate content of the first workshop: How to host a bot on Toolforge - T327283
 * Prepare and migrate content of the second workshop: How to run a basic script via Pywikibot - T327285 (stretch goal)
 * Finish and publish Trove user guide - T326854
 * Design new IA for Cloud docs and get feedback - T327319
 * Pywikibot Toolforge tutorial - T134495
 * Cleanup and improvements as per T308063
 * Define requirements for this tool

Provide bugwrangler support in Phabricator

Outreach, Community and Capacity Building
Organise and run Wikimedia Hackathon 2023
 * Manage logistics for the event
 * Coordinate scholarship application review and selection
 * Design program framework

Plan and run GSOC and Outreachy (summer round) - T326991
 * Recruit projects and mentors for upcoming GSOC/Outreachy rounds

Support 2023 Wikimania Hackathon program planning

Research and plan for integrating the Tech Blog with Diff - T327323
 * Complete stakeholder conversations and create a needs assessment and recommendation for the team
 * Create a project plan for integrating the blogs

Launch tech swag program - T320768
 * Finalize the design
 * Submit the purchase order

Improve our understanding of the tool ecosystem and tool maintainers' needs - T323302
 * Complete 4 interviews with community members and stakeholders - T323317
 * Investigate how to aggregate information about tools that is captured by manual review during Tool Sweep - T327397

Cloud Services
Upgrade OpenStack
 * Upgrade cloud vps openstack to Openstack version Yoga - T305828 ✅

Implement Kubernetes as a Service (Magnum) - T280792 (multi-quarter project)
 * Migrate PAWS/Quarry to Magnum T308873, T301469 (Done in dev)

Ensure alerting and on-call systems are sustainable and scalable - T313444 (multi-quarter project)
 * Enable alerts and pages from cloudvps/toolforge/paws/quarry - T313031

Move cloud hosts to single NIC implementation - T319184

Improve integration with WMCS of SRE cookbook automation - T319401 ✅
 * Collaboration with Infrastructure Foundations team

Finalize and Publish cross-realm guidelines for case 4 - T296411 ✅

Release Toolforge Build Service - T194332 (multi-quarter project)
 * Implement Harbor (status: implemented in toolsbeta)
 * Plan Beta Features and Delivery

Upgrade ToolsDB - T301949

Allow self-service tool deletion - T285403 ✅

Continue GridEngine deprecation (multi-quarter project)
 * Provide support for Toolforge jobs bugs/feature requests ✅

Coordinate Cloud Services User Survey
 * Setup Cloud Services User Survey in Lime Survey ✅
 * Run Cloud Services User Survey ✅

Production or third party services
Finalize taxonomy model for Toolhub - T308030 ✅
 * Implement controlled vocabulary tags in Toolhub

Technical Support and Documentation
Provide continuous support in Phabricator
 * See also: Bugwrangler

Building community, social connection, and technical expertise around documentation (multi-quarter project)
 * Improve documentation and update Phabricator tasks around documentation in two collections (Pywikibot - T320625; Cloud Services - T321582)
 * Publish "Intro to the Wikimedia Technical ecosystem" draft as a wiki page and as a guided experience for newcomers. ✅
 * Improve mw:Documentation landing page and content - T277128 ✅

Outreach, Community and Capacity Building
Coordinate the Coolest Tool Award - T313443 (multi-quarter project)
 * Conduct the 2022 edition of the Coolest Tool Award ✅

Coordinate Wikimedia's participation in Outreachy Round 25 and provide mentorship in one project (multi-quarter project)
 * Prepare and support mentors and projects for Outreachy Round 25 - T313361
 * Support Outreachy applicants and mentor an intern in the project "Create a Web Application for Editing Toolhub Records"

Conclude the 2022 Small Wiki Toolkits workshops
 * Organize two small wiki toolkit workshops ✅
 * Reflect and summarize the past season of Small Wiki Toolkits and propose a plan for next year ✅

Reward volunteer and mentor contributions through the yearly swag program (multi-quarter project)
 * Create project plan for swag program

Clarify responsibilities around Gerrit privilege handout workflow
 * Clarify who needs to be involved/consulted on making the decision - T273164
 * Update the Gerrit Privilege Policy pages - T237618

Focus project
Follow up of the experiment week projects in Q1 (planned start date: November, more info tba).

Cloud Services
Deprecate Gridengine (continues, multi-quarter project) Ensure alerting and on-call systems are sustainable and scalable - T313444 (continues, multi-quarter project)
 * Migrate tools off GridEngine - T313405 (continues, multi-quarter project)
 * Migrate at least one tool to k8s ✅
 * Figure out if migrations of “like” tools can be templated and migrated more easily ✅
 * Consolidate Alerting to a single platform (alertmanager) ✅
 * Audit alerts to ensure any pages are required, as opposed to automatically filing a ticket, or simply removing ✅

Release Toolforge Build Service - T194332 (continues, multi-quarter project)
 * Implement Harbor (status: implemented in toolsbeta)
 * Plan Beta Features and Delivery

Update Striker to work with GitLab for Toolforge tool git hosting automation
 * Replace the existing Diffusion (Phabricator) git integration - T296893 ✅
 * Migrate ~450 existing tool owned git repositories to GitLab - T315706 ✅

Implement Kubernetes as a Service (Magnum) - T280792 (continues, multi-quarter project)
 * Finish bringing service online
 * Evaluate usage for Quarry/PAWS - T308873, T301469 ✅
 * Evaluate usage for community use ✅

Production or third party services
Finalize taxonomy model for Toolhub - T308030
 * Conduct community feedback round on taxonomy model ✅
 * Implement controlled vocabulary tags in Toolhub (postponed to Q2)

Get more people to understand tech community metrics on wikimedia.biterg.io and increase redundancy across the team
 * Create onboarding plan for cross-training a second person - T294870 ✅
 * Organise a consultancy hour with Bitergia - T306787 ✅

Increase the number of blog posts on the Tech Blog - T313072
 * Draft & publish 4 blog posts from the Technical Engagement team
 * Help 4 WMF staff members to draft and publish blog posts
 * Help 4 technical contributors to draft and publish blog posts

Technical Support and Documentation
Make it easier to assess the state of documentation and contribute meaningful improvements (project board)
 * Identify documentation collections and how to assess the state of the content they contain. ✅
 * Make the results of this assessment publicly available and easy to navigate. (tracking: T313041)

Complete basic content review and improvements for all on-wiki collection landing pages that are linked from the Developer Portal (Tracking: T312997)
 * Review landing pages for Data dumps and Research:Data. ✅
 * Review Quarry landing page. ✅
 * Review Pywikibot landing page. ✅

Provide continuous support in Phabricator
 * See also: Bugwrangler

Outreach, Community and Capacity Building
Coordinate Wikimedia's participation in Outreachy Round 25 (continues, multi-quarter project)
 * Recruit mentors and projects for Outreachy Round 25 - T313361 ✅

Continue with 2022 Small Wiki Toolkits workshops
 * Organize three small wiki toolkit workshops on the topic of bots and scripts - T312612 ✅

Clarify Technical Engagement support on Rapid Grants applications - T304380 (postponed)
 * Setup criteria for what makes a successful grant/project
 * Decide on team capacity and workflow process to assess and review grant applications

Clarify responsibilities around Gerrit privilege handout workflow (continues)
 * Clarify who needs to be involved/consulted on making the decision - T273164
 * Update the Gerrit Privilege policy pages - T237618

Coordinate the Coolest Tool Award- T313443 (continues, multi-quarter project)
 * Plan the 2022 edition of the Coolest Tool Award

Coordinate the 2022 Wikimania Hackathon - T313554 ✅

Define timeline and project plan Implement the event
 * Clarify timeline and integration in Wikimania program and platform
 * Define project plan including main deliverables, support needs, and weekly tasks
 * Technical contributors have opportunities to share knowledge and connect socially
 * Present clear resources for individual newcomers (and newish-comers) so they can find the information they need to build what they want. - T313629
 * Coordinate community-run sessions and spaces for people to share technical knowledge. - T313627
 * Coordinate community-run social sessions and spaces with fun activities. - T313627
 * Growing communities of technical contributors can find support and feel welcome at the Wikimania Hackathon
 * Engage with local meetup and affinity meetup organizers from the Wikimedia Hackathon. - T313625
 * Ensure that the accessible platform and accommodations are advertised widely. - T313627
 * Technical contributors have opportunities to work together and to share outcome with Wikimania attendees
 * Organize the pre-event showcase to help people connect with projects and people. This will include explicit instructions to use Phabricator and come back for the final showcase. - T313631
 * Design an accessible and useful platform space for people to collaborate in. - T313630
 * Provide Wikimania attendees an opportunity to learn about the hackathon projects at the final showcase. - T313631

Experiment Week
We've planned in a few days this quarter to explore two topics:

✅ Research how other open source communities support newcomers - T313543
 * Create comparison matrix and summarize key findings
 * Provide recommendation on what Wikimedia could try out or explore further

✅ Identify metrics to measure the impact of a tool - T313546
 * Research and summarize available data sources and gaps to define tool impact metrics
 * Create visualization to display raw data