Wikimedia Technology/Goals/2019-20 Q4

Technology Department Team Goals and Status for Q4 FY19/20 in support of the  Medium Term Plan (MTP) Priorities and Annual Plan for FY19/20



Analytics
Team Manager: Nuria Ruiz
 * MTP-Y1: Platform Evolution Build a reliable, scalable, and comprehensive platform for creating services, tools and user facing features that produce and consume event data
 * 5% of production and 100% analytics events have been migrated to the new event platform.
 * By June 2020 all production and consumption of new event data originated in our websites is flowing through this new platform
 * ✅ Client Error Logging is deployed to 1 wiki and error stats are displayed on our operation dashboards.


 * Modern Event Platform Build a reliable, scalable, and comprehensive platform for creating services, tools and user facing features that produce and consume event data
 * Deploy a new event stream for analytics using the new Event Platform infrastructure: Session length
 * Vertical MEP from web to backend: Migrate SearchSatisfaction EventLogging event stream to Event Platform


 * Smart Tools for Better Data Make easier to understand the history of all Wikimedia projects
 * ✅ Wikistats UI is localized for languages and number formatting
 * Wikistats UI is more flexible when it comes to explore metrics. Allow spliting and filtering simultaneously
 * Design (together with core platform team) an alternative architecture for historic data endpoints used by iOS application
 * Implement foundations for newpyter (hadoop hosted distributed jupyter notebook setup)


 * Smart Tools for Better Data Increase Data Quality, Privacy and Security
 * ✅ Bots: Label high volume bot spikes in pageview data as automated traffic
 * ✅ Deploy first iteration of bot spikes detection model on the pageview pipeline by June 2020


 * Core Operational Excellence. Increase Resilience of Systems
 * Create a MySQL replica for backups for all MySQL instances we use MySQL on, like Oozie or Superset task T234826
 * ✅ Airflow as an easier job scheduling alternative, PoC for refine workflow
 * ✅ Unify stats and notebook cluster. Decomision notebook hosts and make puppet role of stat1007 just like the other stats boxes



Fundraising Tech
Team Manager: Erika Bjune
 * Core Work
 * Support Advancement in testing and planned Q4 campaigns
 * Finish employer field work on matching gifts V2
 * Scalability: decouple front end and back end of payments by splitting out functionality of contribution tracking table



Engineering Productivity Team
Team Manager: Greg Grossmeier
 * MTP-Y1: Platform Evolution Maintain and evolve developer tooling, testing infrastructure, validation environments, deployment infrastructure, and supporting processes
 * Release Engineering and SRE jointly create a plan of action to implement a Deployment Pipeline compliant...
 * Developers have a consistent and dependable deployment service.
 * Improve developer productivity by automating manual steps out of the model development and deployment pipeline
 * Build and support a fully automated and continuous Code Health and Deployment Infrastructure
 * Maintain and improve the Continuous Integration and Testing services
 * Reduce infrastructure gaps in the areas of backups & disaster preparedness, observability, infrastructure...
 * Service owners, deployers and other stakeholders are able to develop, test, deploy, observe and maintain services...
 * The organization is able to make data-driven decisions about tests, testing infrastructure, and deployments.
 * ✅ Ensure WDQS Performance stability over time
 * MTP-Y1: Platform Evolution We will improve developer efficiency for all developers, new and experienced, internal and external
 * Create a cohesive documentation portal to onboard new developers to our API
 * Services are able to intercommunicate in a reliable, secure and standardised way in our infrastructure
 * Improve all baseline developer efficiency metrics by 10% by the end of the year.
 * Production-like containers
 * Create an easy to use REST API with the basic functionality needed to interact with our platform
 * ✅ Determine a baseline set of metrics to assess internal developer efficiency, including time to first merge (new...
 * We will improve Cycle Time for Internal Experienced Developers by 10% year over year.
 * Local development container system
 * ✅ We will plan and execute this year’s Technical Conference and produce a prioritized list of future work outcomes as...
 * Successfully run Wikimedia’s technical internship and outreach programs
 * ✅ Find committed owners for all prioritized outcomes of the 2019 Technical Conference in time for annual planning.
 * Increase visibility & knowledge of technical contributions, services and consumers across the Wikimedia ecosystem
 * Develop, test and evaluate different formats to build technical capacity in smaller wikis
 * Align developer services with SRE best practices
 * Improve Resilience of Wikimedia's Gerrit Install
 * ✅ Deploy developer services with Scap3
 * ✅ Improve availability and redundancy of Wikimedia's Phabricator
 * Build and support a fully automated and continuous Code Health and Deployment Infrastructure
 * ✅ Work across teams to ensure that 5% of projects are moving through the continuous delivery pipeline
 * Automate MediaWiki Train
 * Provide a new CI/CD platform to Wikimedia Technology and Product teams by the end of FY19-20
 * Continuously deploy at least one project by the end of fiscal year 2020
 * Static variant configuration for production
 * Improve ability for developers to share changes
 * Improve all baseline developer efficiency metrics by 10% by the end of the year
 * Develop the 2020 Developer Satisfaction Survey, gather, and summarize results, giving us another yearly data point...
 * Extend and improve Phabricator to enhance engineering productivity.
 * The organization is able to make data-driven decisions about tests, testing infrastructure, and deployments
 * Provide infrastructure to store data and metrics in support of decision making with regards to tests, testing...

Performance
Team Manager: Gilles Dubuc
 * MTP-Y1: Platform Evolution Create a culture of performance across all Wikimedia engineering teams by the end of the fiscal year, to reduce the frequency of performance regressions
 * ✅ By early November, smoothly and successfully give managerial duties of the Performance team to Gilles
 * Foster a culture of performance
 * Expand the coverage of performance monitoring
 * Improve performance
 * Collaborate with other teams via performance reviews
 * Performance review of the client-side-only version of the Graph extension
 * Performance review of the DiscussionTools extension
 * Performance review of Wikidata Bridge
 * Performance review of Improved Commons search
 * ✅ Performance review of the KaiOS app
 * Performance review of the GrowthExperiments extension
 * Performance review of Push Notifications infrastructure
 * Expand the coverage of performance monitoring
 * ArcLamp flame graphs stored in Swift, retained at least 2 years
 * Expand the coverage of synthetic performance monitoring
 * ✅ Collect and graph First Input Delay
 * Implement alerts for synthetic search
 * Add operational monitoring for 100% of the performance-team services
 * ✅ Expand the coverage of backend performance testing
 * Organise and oversee implementation of First Paint on Safari
 * Test out small variance (2%) in latency for our test
 * Remove XHGui dependency on MongoDB
 * Be able to run WPT & WPR on our own Kubernetes cluster.
 * Simulate slow connections on real devices.
 * Foster a culture of performance
 * ✅ Provide performance expertise to FAWG outcome (Sept '19-June '20)
 * ✅ Make it easier for engineers to find performance issues during development (Fresnel)
 * ✅ Organise and run the Web Performance devroom at FOSDEM 2020
 * Publish 8 blog posts about performance
 * ✅ Publish blog post about Wikipedia JS init improvement
 * Publish blog post about ResourceLoader feature test
 * ✅ Publish blog post about organising and running a FOSDEM devroom
 * ✅ Publish blog post on web performance calendar about Long tasks and FID
 * ✅ Publish blog post about WikimediaDebug v2
 * Blog post (with video?) about the WebPageTest and WebPageReplay setup
 * ✅ Publish blog post on web performance calendar about RUM insights
 * Publish blog post about CPU microbenchmark
 * ✅ Document and evangelize synthetic testing, User Timing and Element Timing
 * Collaborate with other teams via performance reviews
 * Organise and oversee org-wide frontend web performance training
 * Improve performance
 * Improve software consistency (speed/success) in handling contributor actions
 * ✅ Improve Wikipedia save-edit performance
 * Improve MediaWiki PHP startup time and recover from PHP7 regression
 * Audit default JS payload and lead efforts to reduce its cost
 * Make logged-in MediaWiki end-user use the closest datacenter
 * Migrate prod localisation cache to faster static-array distribution
 * Parallel MediaWiki phpunit test prototype

Quality and Test Engineering
Team Manager: JR Branaa
 * MTP-Y1: Platform Evolution Enable engineering-wide quality and testing strategy, tooling, education, and personnel.
 * An explicit set of unit, integration, and system testing tools is available for all supported engineering languages
 * Quality and Testing Engineering - Team formation and migration
 * ✅ Introduce TDD as a way of getting a better quality code throughout the Foundation
 * ✅ Evaluate alternative system level testing tooling options and provide a single recommendation by the end of Q2.
 * Work closely with Product teams to build and establish working relationships for the Quality and Test Engineering...
 * Organise regular TDD workshops

Release Engineering
Team Manager: Tyler Cipriani
 * Improve Resilience of Wikimedia's Gerrit Install
 * Gerrit is migrated to 2.16
 * Gerrit's backup restore is proven to work correctly



Machine Learning / Scoring Platform
Team Manager: Aaron Halfaker
 * MTP-Y1: Platform Evolution Expand quality control AIs and tooling to underserved communities in order to make fundamental services available for consumption by tools
 * ✅ We will deploy 8 new quality control AIs to Wikimedia Projects to increase the availability of AIs for tooling
 * ✅ We will improve 3 AIs in a statistically significant way based on community feedback to ensure the utility of AIs for...
 * ✅ We will recruit 4 new campaign coordinators to advertise the availability of AIs to increase the rate of consumption in...
 * Support interest-based routing in Wikipedia
 * ✅ Improve the topic taxonomy used in current topic models
 * ✅ Deploy 5 new topic models to production targeting Growth's homepage experiments
 * ✅ Document the topic modeling pipeline
 * ✅ Implement a language independent topic modeling pipeline.
 * ✅ Extend and improve ORES topic models to support new languages.
 * Support model auditing behavior and coordination between patrollers
 * Deploy Jade to production on at least one wiki by the end of FY
 * Submit a paper about Jade for publication by the end of Q4
 * Increase the rate of community-based false-positive reporting in damage detection models by 100X
 * Write a report about user behavior in Jade in Q4
 * Implement front-end for Jade
 * Support newcomer engagement efforts by deploying models for flagging goodfaith newcomers
 * Deploy the first newcomer quality model to production by the end of Q4
 * Socialize the use of newcomer quality models in at least two tools by the end of Q4



Platform
Team Manager: Corey Floyd
 * Create a cohesive documentation portal to onboard new developers to our API
 * Create infrastructure for developing better structured documentation to make it easier to build easy to read
 * Developers can easily understand the contents of the portal and find the information they need
 * Allow developers to quickly get started building knowledge based applications using our APIs.
 * ✅ Build a prototype for the documentation portal
 * The portal is a hub for a thriving community of developers.
 * Develop a technical direction for the Wikimedia Platform to support Wikimedia Medium Term Plan
 * Enable the development of full featured Javascript web clients
 * Communicate the vision and plan for the Core Platform Team's work through the end of the FY resulting from PE
 * Develop a strategy to integrate Javascript frameworks into the MediaWiki platform to enable easy development
 * Improve the sustainability of MediaWiki and the ease of building on top of it
 * Quantify and reduce coupling in MediaWiki Core
 * Initiatives that the Core Platform Team begins are driven to completion
 * Allow for more confident refactoring of core code
 * Close out MCR work
 * MW Core Code is better logically decomposed into libraries, introduction of new cross dependencies is...
 * ✅ Product requirements for upcoming Core Platform Team initiatives are documented
 * Further decoupling efforts
 * Close out actor and comment migration
 * Limit vandalism requests by bad actors and guarantee levels of service through securing the API
 * Reduce the risk of vandalism by bad actors by limiting throughput of anonymous API calls
 * Completion and shipping of the OAuth 2.0 initiative Epic 1 and 2
 * Reduce the risk of vandalism by bad actors by enabling the ability to disable access of known API users

Architecture
Team Manager: Kate Chapman
 * Define target architecture for structured content so pieces of content can be more easily used to engage users.
 * ✅ Perform task analysis modeling with product managers help determine pain points and needed system capabilities
 * Engage stakeholders to present plan on system changes needed to better enabled structured data.
 * ✅ Develop proposal for modern system to enable structured data.
 * Present proposal to CTO and CPO to gain support for no longer focusing on building page building software
 * Make architectural decision process clear so teams have clear direction as to what decisions have been made and how to proceed.
 * Create plan for decision making process
 * Develop template for technical design and decisions
 * Engage stakeholders in decision making process for feedback.

Platform Engineeering
Team Manager: Mat Nadrofsky
 * Drive the Delivery of Q4 Platform Engineering Initiatives
 * Limit the ability for bad actors and misinformed users to impact the availability of our services
 * Enable developers to make system changes while maintaining a consistent stable experience
 * Help a team migrate their service to Kubernetes



Research
Team Manager: Leila Zia
 * MTP-Y1: Thriving Movement By the end of FY20, build the taxonomies of knowledge gaps to prepare the foundation for measuring and prioritizing knowledge gaps
 * Build three or more taxonomies of knowledge gaps
 * Present the taxonomies internally (through one or more presentations) and externally
 * Provide a comprehensive write-up of the taxonomies developed during the year.
 * MTP-Y1 Platform Evolution Secure and protect platform, and communities, in the free knowledge movement against the spread of disinformation and bad-actor risk
 * Develop and mature 2 security governance services as risk management and security awareness.
 * ✅ Organize at least one brainstorming session with internal and external stakeholders to identify and prioritize
 * Build 2 sets of Formal Collaborations to expand our capacity for working on prioritized disinformation projects
 * Limit vandalism requests by bad actors and guarantee levels of service through securing the API
 * ✅ Develop and mature 2 security engineering services as application security and privacy engineering.
 * Build a test model to address a specific type of disinformation
 * Develop a means to limit and disable the API access of bad actors without interrupting the access of other contributors
 * Wikimedia’s infrastructure and data is protected from common threats and vulnerabilities
 * Finalize the WMF Research portal/program page template for Meta
 * The Wikimedia movement has the tools, knowledge, and support to identify and respond to disinformation campaigns
 * Improve the sockpuppet detection model.
 * Build a test API (or other form of output) for the sockpuppet detection model in collaboration for Product.
 * Perform a large-scale analysis of citation quality in Wikipedia
 * The Wikimedia projects and communities have access to a global network of researchers who is deeply aware of Wikimedia's open questions and knows the data
 * Mentor 1-2 students as part of the first internship pilot in Research.
 * Mentor 1 student as part of Outreachy.
 * Organize the monthly Office hours
 * Wikimedia has increased presence and influence within the research communities to expand the network of Wikimedia researchers globally as well as to more broadly
 * Submit a paper about the model developed for detecting disinformation.
 * Submit a paper on reader demographics survey.
 * Co-organize the monthly Research Showcase
 * Submit a paper about research on aligning sections across Wikipedia languages.
 * ✅ Co-organize the annual Wiki Workshop 2020
 * Prepare for co-organizing the monthly Research Showcase
 * Characterize Wikipedia reader behavior to learn how to engage readers across the globe.
 * Train and evaluate one deep-learning model on reader sessions in comparison to navigation vectors
 * Initiate the analysis of readers' engagement with images in Wikipedia
 * ✅ For at least one wiki, create list of related article pages which co-occur in sessions involving articles on covid-19.
 * Build three or more taxonomies of knowledge gaps (the list of taxonomies: content, readership, contributorship, usage, and primary causes)
 * Build the first complete draft of the taxonomy of content gaps.
 * Build the first complete draft of the taxonomy of contributor gaps.
 * Build the first complete draft of the taxonomy of readership gaps.



Search Platform
Team Manager: Guillaume Lederrey
 * MTP-Y1: Thriving Movement Develop means for addressing knowledge gaps in Wikimedia projects
 * Apply existing Machine Learning and NLP techniques to four new underrepresented languages by the end of Q3.
 * Learning to Rank (LTR) applied to additional languages and projects to improve ranking (needs experimentation,...
 * Enhance search suggestions to allow for easier access to results
 * ✅ Glent method 0 (session reformulation) A/B tested and deployed by end of Q2
 * Provide better "did you mean" search suggestions to users
 * ✅ Glent method 1 (comparison to other users' queries) offline tested, tuned, A/B tested and possibly deployed end of Q2
 * Increase understanding of our work outside our team, and outside the Foundation
 * Increase non-technical understanding of our work outside our team, and outside the Foundation
 * Increase WD and WDQS in community
 * Improve search quality, especially for non-English wikis by prioritizing community requests
 * ✅ Positive feedback from speakers/community on changes made
 * Improve search quality, especially for non-English wikis
 * Complete homoglyph plugin that will help improve search results
 * Investigate query completion to guide users to better queries
 * ✅ Have a plan around using query completion for search
 * Build and deploy a testable prototype of query completion for commonswiki
 * Updates to WDQS are streaming in an event sourcing fashion so that WDQS isn't the bottleneck in the Wikidata edits.
 * Run streaming updater pipeline continuously
 * Support the SDAW project
 * Review of Commons query logs



Security
Team Manager: John Bennett
 * MTP-Y1: Platform Evolution Secure and protect platform, and communities, in the free knowledge movement against the spread of disinformation and bad-actor risk
 * Develop and mature 2 security governance services as risk management and security awareness.
 * ✅ Organize at least one brainstorming session with internal and external stakeholders to identify and prioritize the...
 * Build 2 sets of Formal Collaborations to expand our capacity for working on prioritized disinformation projects;...
 * Limit vandalism requests by bad actors and guarantee levels of service through securing the API
 * ✅ Develop and mature 2 security engineering services as application security and privacy engineering. New...
 * Build a test model to address a specific type of disinformation (to be determined in the previous step) by...
 * Develop a means to limit and disable the API access of bad actors without interrupting the access of other contributors,...
 * Wikimedia’s infrastructure and data is protected from common threats and vulnerabilities



Site Reliability Engineering
Directors: Mark Bergsma and Faidon Liambotis
 * Infrastructure operational issues are being identified, surfaced and escalated appropriately
 * ✅ Develop new alert notification, escalation and paging capabilities to accommodate the increased needs
 * The reachability of our infrastructure over the Internet is observable and understood
 * ✅ Errors from clients are surfaced to developers
 * ✅ Operational issues are being identified, surfaced and escalated appropriately
 * ✅ Deploy client error logging is to 1 wiki with error stats displayed on our operation dashboards
 * Operational issues are being identified, surfaced and escalated appropriately
 * ✅ Operational issues are being identified, surfaced and escalated appropriately
 * ✅ Errors from clients are surfaced to developers
 * ✅ Operational issues are being identified, surfaced and escalated appropriately
 * ✅ A standardized and more efficient logging format enabling easier interpretation of operational issues and mitigate...
 * Service owners, deployers and other stakeholders are able to develop, test, deploy, observe and maintain services without dependencies on other teams
 * ✅ Assist with adoption of at least 2 additional services into the Deployment Pipeline by service owners by end of Q2
 * Service owners, deployers and other stakeholders are able to develop, test, deploy, observe and maintain services, etc
 * Maintain and evolve developer tooling, testing infrastructure, validation environments, deployment infrastructure, etc.
 * Wikimedia’s infrastructure and data is protected from common threats and vulnerabilities
 * ✅ Ensure general backup service is migrated to new hardware infrastructure by end of Q2 and general backup...
 * Services are resilient to the failure of a single data center
 * Wikimedia’s data can be recovered reliably and quickly from accidental data loss
 * Improve the resiliency of the MediaWiki platform against outages and instability.
 * Wikimedia’s infrastructure and data is protected from common threats and vulnerabilities
 * ✅ On-board at least two new services to the new Single Sign-On infrastructure
 * ✅ Extend security vulnerability tracking for container images
 * ✅ Extend vulnerability tracking to container images
 * ✅ Improve the security of our web-based services by integrating into a common identity provider/SSO
 * ✅ Enable opt-in 2FA for web services SSO
 * ✅ Improve the security of our web-based services by integrating into a common identity provider/SSO
 * Our network routing and reachability is protected from malicious or accidental threats
 * Improve the security of our web-based services by integrating into a common identity provider/SSO
 * Typical (D)DoS attacks are detected, observable, and able to be responded to by any SRE
 * Wikimedia staff is able to perform common workflows faster and more efficiently, with fewer errors
 * ✅ Reduce the number of manual steps involved in the provisioning and decommissioning of new services by 1
 * ✅ Reduce the amount of manual work and complexity required to make configuration changes in network devices
 * Build automation that reduces manual toil and time required for MediaWiki database server maintenance
 * Reduce time required and avoidable errors during the generation of server's DNS records.
 * ✅ Drive the configuration of the networking infrastructure via automated means & ensure multiple team members are...
 * Reduce time required and avoidable errors during the generation of server's DNS records.
 * Reduce the amount of manual work and complexity required to make configuration changes in network devices
 * -Support expansion of automation through tools, libraries and processes.-
 * ✅ Have fewer services require root permissions for day-to-day automation tasks
 * ✅ Have fewer services require root permissions for day-to-day automation tasks
 * ✅ Network interface configuration for switches and routers is managed and driven by data
 * ✅ Increase participation in the Incident Coordinator process
 * ✅ Reduce time required and avoidable errors during the generation of server's DNS records.
 * Maintain and Improve Netbox through tools and processes
 * (Faidon) Wikimedia’s technical infrastructure is sustained at common industry standards and levels and maintained with low technical debt
 * ✅ Modernize and increase capacity of database infrastructure by end of Q2
 * ✅ Finish what we started: Cleanup remnants of HHVM from our infrastructure by end of Q2
 * Migrate core software components of the Deployment Pipeline to current major releases
 * Reduce the amount of unmaintained services and products running in production
 * ✅ Main database infrastructure is running stable on modern hardware with sufficient capacity for the next few years,...
 * ✅ Unused parts of the infrastructure are removed to reduce complexity and improve resource usage
 * SREs are able to perform maintenance on application object caching services without user impact
 * Wikimedia’s technical infrastructure is sustained at industry standard levels and adequately maintained
 * Improve resilience and modernize devtools services
 * Wikimedia’s traffic privacy and security implementations continue to track the leading edge of industry standards
 * ✅ Configuration is tested as code
 * ✅ Further the goal of making Parsoid the default wikitext engine for MediaWiki
 * Wikimedia’s technical infrastructure is sustained at common industry standards and levels and maintained
 * Wikimedia's caching infrastructure is robust, supports industry standard protocols, and maintained
 * Sunset Debian 8 and 9
 * (Mark) Wikimedia’s technical infrastructure is sustained at common industry standards and levels and maintained with low technical debt
 * ✅ Upgrade the Elastic/Logstash version to >= 7.2
 * ✅ Reduce the number of service clusters running a soon-to-be unsupported Debian release by 8
 * ✅ Services are able to intercommunicate in a reliable, secure and standardised way in our infrastructure [TODO: shared...
 * ✅ Ensure that the logging pipeline utilizes current software versions, and runs with minimal technical debt
 * ✅ Replace/renew the internal Certificate Authority (expires Jun 2020)
 * ✅ Rebuild the Amsterdam data center
 * ✅ Have esams and knams network stack up to date and up to our standard in order to prevent downtime due to old...
 * ✅ Ensure that the logging pipeline utilizes current software versions, and runs with minimal technical debt
 * ✅ Adopt Prometheus as the new standard for operational metrics to reduce operational overhead and enhance our...
 * ✅ A standardized and more efficient logging format enabling easier interpretation of operational issues and mitigate...
 * ✅ Improve our PKI infrastructure by creating a centrally managed PKI infrastructure and address the immediate...
 * ✅ Sunset Debian 8 and 9
 * ✅ Ensure that the logging pipeline utilizes current software versions, and runs with minimal technical debt



Technical Engagement
Team Manager: Birgit Müller

Developer Advocacy
Team Manager: Birgit Müller
 * Develop, test and evaluate different formats to build technical capacity in smaller wikis
 * ✅ Conduct workshop and document the technical challenges small wikis face in North America
 * Organize an online workshop series for Indic language small wikis
 * Create a hub for the Small Wiki Toolkits initiative
 * A starter kit for small wikis containing a recommended set of templates, Gadgets, bots, etc. is available by Q4
 * Write a report highlighting lessons learned from developing and testing different formats to build technical capacity in smaller wikis


 * Increase visibility & knowledge of technical contributions, services and consumers across the Wikimedia ecosystem
 * ✅ Establish Coolest Tool Award
 * Share stories and insights from the technical community
 * Increase knowledge on scope and breadth of technical contributions and contributors
 * Train people how to use Phabricator to increase acceptance and foster collaboration


 * Successfully run Wikimedia’s technical internship and outreach programs
 * Successfully coordinate Outreachy and GSOC
 * ✅ Mentor First Season of Google Season of the Docs.
 * ✅ Submit and hold session on Wikimedia's Tech internships at WikiCon Northamerica
 * ✅ Successfully coordinate Google Code-in 2019
 * Mentor 1 intern on the WikiContrib project via Outreachy round 20

Wikimedia Cloud Services
Team Manager: Bryan Davis
 * All Debian Jessie instances are removed/replaced in Cloud VPS hosted projects
 * ✅ Remove Debian Jessie from the Cloud VPS "toolsbeta" project
 * ✅ Remove Debian Jessie from the Cloud VPS "tools" project
 * Remove Debian Jessie from the Cloud VPS "openstack" project


 * Increase application security by hosting tools using unique hostnames rather than path based routing
 * ✅ Update front proxy to support host based routing
 * ✅ Create redirect system to preserve function of legacy URLs following conversion from path base to host based routing of each tool
 * Migrate all tools to host based routing
 * ✅ Update `webservice` to support host based routing
 * ✅ Migrate 5+ early adopter/beta tester tools to host based routing
 * Interwiki links support for $tool.toolforge.org


 * Upgrade Toolforge Kubernetes to 1.16
 * Update `webservice` to support k8s 1.16 APIs
 * Determine blockers for k8s 1.16 upgrade and assign as tasks/KRs to team
 * ✅ Fix psp API group to work with k8s 1.16
 * Deploy Kubernetes 1.16 in Toolforge


 * WMCS Infrastructure as a Service (IaaS)
 * Debian Jessie operating system deprecation
 * OpenStack platform upgrades
 * Galera cluster
 * CEPH instance storage
 * Fix Cloud VPS and Toolforge mail servers to work with the modern internet


 * WMCS Platform as a Service (PaaS)
 * Provide a more modern, secure, and performant PaaS experience for Toolforge tools
 * Increase quality of technical documentation for Toolforge and Cloud VPS users
 * PAWS Kubernetes rebuild'''