Wikimedia Technology/Goals/2017-18 Q4

From MediaWiki.org
Jump to navigation Jump to search
TriangleArrow-Left.svgQ3 Wikimedia Technology Goals, FY2017–18, Q4 (April - June) Q1TriangleArrow-Right.svg

Introduction[edit]

Purpose of this document[edit]

Goals for the Wikimedia Technology department, for the fourth quarter of fiscal year 2017–18 (April - June 2018). The goal owner in each section is the person responsible for coordinating completion of the section, in partnership with the team(s) and relevant stakeholders.

Goals for the Audiences department are available on their own page

Legend[edit]

Tech Goal categorizes work into one or more of these quadrants:

A Foundation level goals C Features that we build to improve our technology offering
B Features we build for others D Modernization, renewal and tech debt goals

ETA (Estimated Time of Arrival) fields may use the acronym EOQ (End of Quarter) or EOY (End of Year).

Status fields can use the following templates: In progress In progress, To do To do, N Postponed, Yes Done or Incomplete Partially done


Technology departmental programs[edit]

Program 1: Availability, performance, and maintenance[edit]

Overall goal owner: Mark Bergsma

  • Owner for Outcome 1: Mark Bergsma
  • Owner for Outcome 2: Ian Marlier
  • Owner for Outcome 3: Nuria Ruiz
  • Owner for Outcome 4: Bryan Davis
  • Owner for Outcomes 5 and 6: Greg Grossmeier
Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: All production sites and services maintain current levels of availability or better. Objective 1: Deploy, update, configure, and maintain production services (Traffic infrastructure, databases & storage, MediaWiki application servers, (micro)services, network, and miscellaneous sites & services) Update MediaWiki application server platform and prepare for PHP7
  • Migrate to ICU 57
  • Upgrade the MediaWiki application server cluster to Debian stretch
  • Stretch goal: Migrate MediaWiki dumps to PHP7 on Debian
  • Stretch goal: Merge video scalers into the job runners cluster and remove the image scalers
  • Site Reliability Engineering
    • Service Operations
    • Infrastructure Foundations
  • Release Engineering (Beta)
  • Community Liaisons (ICU migration)
D EOQ Yes Done
Outcome 1: All production sites and services maintain current levels of availability or better. Objective 1: Deploy, update, configure, and maintain production services (Traffic infrastructure, databases & storage, MediaWiki application servers, (micro)services, network, and miscellaneous sites & services) Stabilize Sanitarium
  • Migrate Sanitarium cluster from multi-source to multi-instance
  • Expand Sanitarium cluster with additional capacity and redundancy
  • Site Reliability Engineering
    • Data Persistence
D EOQ Yes Done
Outcome 1: All production sites and services maintain current levels of availability or better. Objective 1: Deploy, update, configure, and maintain production services (Traffic infrastructure, databases & storage, MediaWiki application servers, (micro)services, network, and miscellaneous sites & services) Release and deploy Debmonitor (patch management software)
  • Release v1.0 of Debmonitor
  • Deploy the web frontend in production
  • Deploy the agent in all production hosts
  • Site Reliability Engineering
    • Infrastructure Foundations
D EOQ Incomplete Partially done
Outcome 2: All users consistently experience systems that perform well: Objective 1: Maintain a comprehensive toolset to measure the performance of our platforms
  • Adopt use of histograms for important metrics
  • Continue monitoring performance of WMF properties
  • Establish team/project scorecards that measure important metrics in a clear, consistent, actionable way
  • Identify options for measuring mobile performance, and develop a roadmap for delivery in FY18/19
Performance EOQ In progress In progress
Outcome 2: All users consistently experience systems that perform well: Objective 2: Catch and address performance regressions in a timely fashion through automation
  • Finalize Linux WPT infrastructure Yes Done
  • Work with FRTech to improve the performance of CentralNotice In progress In progress
Performance FRTech EOQ
Outcome 2: All users consistently experience systems that perform well: Objective 3: Modernize our performance toolset. We will measure performance metrics that are closer to what users experience.
  • Decide on PHP7 sampling profiler tool, and prep for use in production. In progress In progress
  • Create a roadmap for measuring backend latency In progress In progress
  • Finish NavTiming2 keys, and create new dashboards that use them In progress In progress
Performance EOQ
Outcome 3: We have scalable, reliable and secure systems for data transport.: Objective 2: Consolidation of analytics Kafka infrastructure to tier-1 requirements, including TLS encryption task T152015:
  • Continue Port most (if not all) Kafka clients from Analytics to Jumbo, with TLS where appropriate task T175461Yes Done
  • Upgrade main Kafka clusters to 1.0 task T190853 Yes Done
  • Upgrade Kafka Burrow to 1.0 task T188719 Yes Done
  • Shutdown plain text production of webrequest data, allow tls only. Yes Done
  • STRETCH: Decommission analytics Kafka cluster task T183303 To do To do
Analytics Operations, Services EOQ Yes Done
Outcome 3: We have scalable, reliable and secure systems for data transport.: Objective 2: Maintenance and expansion of current Hadoop cluster to support new use cases that require more computational resources: Upgrade of Druid 0.11 task T164008 Analytics EOQ Yes Done
Outcome 3: We have scalable, reliable and secure systems for data transport.: Objective 3:' Software, hardware upgrades, and maintenance on analytics stack to maintain current level of service:
  • Refresh zookeeper nodes on eqiad task T182924 Yes Done
  • Make spark2 cluster default. Deploying all jobs that run spark in spark2. Task T159962 Yes Done
  • STRETCH GOAL: Cleanup CRON mailto standard error convention, maybe use cronic task T172532 To do To do
Analytics EOQ Yes Done
Outcome 3: We have scalable, reliable and secure systems for data transport. Objective 3: Software, hardware upgrades, and maintenance on analytics stack to maintain current level of service:

Event Data Platform initial project planning

  • Interviews with customers
  • Create Phabricator tickets, outline TODOs and timelines.
  • Start decision spike for JSON versus Avro
Analytics Operations, Services EOQ To do To do
Outcome 4: Wikimedia Cloud Services users can leverage a reliable and public Infrastructure as a Service (IaaS) product ecosystem for VPS hosting. Objective 2: Pay down technical debt and allow upgrading of the core OpenStack platform to modern, supported releases by replacing the current network topology layer with OpenStack Neutron, which has become the standard for most OpenStack deployments. Continue work towards full deployment of Neutron software defined network layer by upgrading all OpenStack components to the "Mitaka" release or newer in the labtest, labtestn, and main OpenStack clusters. WMCS C: Improving our offering

D: Tech debt

EOQ Yes Done
Outcome 5: We have effective and easy-to-use testing infrastructure and tooling for developers. Objective 1:  Maintain existing shared Continuous Integration infrastructure Migrate away from Nodepool - task T190097
  • Migrate MediaWiki PHPUnit tests to Shipyard (docker-based CI) (~40% of Nodepool usage) - task T183512
  • Add Composer support to Blubber - task T186547
  • Add Python/Tox support to Blubber - task T186545
  • Add Ruby/Gem/Bundler/Rake support to Blubber - task T188950
Release Engineering SRE, WMCS B: For others EOQ Incomplete Partially done

Program 2: MediaWiki[edit]

Goal owner: Tim Starling

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Stakeholders in MediaWiki development will have sense of progress and direction in MediaWiki. Objective 1: Develop a MediaWiki roadmap Planning for the Platform Evolution CDP MediaWiki/Audiences D Modernization, renewal and tech debt goals EOQ In progress In progress
Outcome 2: MediaWiki code quality will be improved. Objective 2: Break up large classes and source files Namespacization (task T166010) MediaWiki D Modernization, renewal and tech debt goals EOQ N Postponed
Outcome 3: MediaWiki security and stability will be improved. Objective 1: Address the backlog of action items that arise from security and downtime post-mortems Execute scripts to cease using unserialize() for MediaWiki text storage (task T181555) MediaWiki C Features that we build to improve our technology offering EOQ In progress In progress
Outcome 3: MediaWiki security and stability will be improved. Objective 1: Address the backlog of action items that arise from security and downtime post-mortems Investigate the use of Content-Security-Policy (CSP) headers in MediaWiki task T135963 MediaWiki C Features that we build to improve our technology offering EOQ N Not done (watching, work will be done by another team)

Program 3: Addressing technical debt[edit]

Goal owner: Jean-Rene Branaa

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: The amount of orphaned code that is running Wikimedia “production” services is reduced. Objective 2: Identify and find stewards for high-priority/high use code segment orphans - task T174091 Broad roll out of Code Stewardship model.
  • Update Maintainers/Developers page with currently known Code Stewards
  • Use Code Stewardship review process to address gaps
  • Deploy dashboard of Code Stewardship Coverage
  • Communicate progress to Tech Managers
Release Engineering
  • All engineering teams (participation needed but not required for this goal to be complete)
  • CPO/CTO (advisory role)
C: Tech Debt EOQ Yes Done
Outcome 2: Organizational technical debt is reduced. Objective 2: Define and implement a process to regularly address technical debt across the Foundation Roll out of technical debt reduction approaches.
  • Identify early adopter engineering teams to test approaches.
Release Engineering
  • Select group of engineering teams
C: Tech Debt EOQ Yes Done
Outcome 2: Organizational technical debt is reduced. Objective 3: Promote and surface important technical debt topics at large gatherings of Wikimedia developers (e.g., Hackathon) Promote and surface important technical debt topics at Hackathon in May 2018 Release Engineering Code Health Group, Technical Debt SIG C: Tech Debt EOQ Yes Done

Program 4: Technical community building[edit]

Overall goal owner: Bryan Davis

  • Owner for Outcome 1: Bryan Davis
  • Owner for Outcomes 2 and 4: Dario Taraborelli
  • Owner for Outcome 3: Cindy Cicalese
Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Becoming a technical contributor to the Wikimedia movement by creating and maintaining 'tools' (bots, webservices, etc) and other innovative solutions is easier than it has been historically because documentation is easier to find, more comprehensive, and descriptive of start to finish steps needed to solve common problems. Cloud Services product users feel comfortable sharing their knowledge with others as part of a community with a culture of sharing via documentation and mutual support. Objective 1: Collaborate with community to find volunteers willing to form a documentation Special Interest Group to update documentation of existing Cloud Services products
  • Produce a style guide to provide guidance for technical writing contributors
  • Identify at least one good example article for each documentation type (tutorial, how to, explanation, reference, etc) covered by the style guide
WMCS A: Foundation goal (Community)

B: Serving our audiences C: Improving our offering

EOQ Yes Done
Outcome 1: Becoming a technical contributor to the Wikimedia movement by creating and maintaining 'tools' (bots, webservices, etc) and other innovative solutions is easier than it has been historically because documentation is easier to find, more comprehensive, and descriptive of start to finish steps needed to solve common problems. Cloud Services product users feel comfortable sharing their knowledge with others as part of a community with a culture of sharing via documentation and mutual support. Objective 3: Increase community awareness of volunteer developed Tools

(Added for Q4; not in official plan)

WMCS Technical Collaboration A: Foundation goal (Community)

B: Serving our audiences C: Improving our offering

EOQ Yes Done
Outcome 3: Value-added vendors who provide services and support based on Wikimedia software and true 'third parties' who install and use software produced by the Wikimedia movement on their own computers are more confident in recommending, deploying, and extending Wikimedia FLOSS projects. Objective 1: Establish canonical point of contact for third-parties by promoting the existence of a dedicated technical liaison for software projects with support for third-party users Establish channels of communication with third-party developers (task T184606) MediaWiki Technical Collaboration B: Features we build for others EOQ In progress In progress
Outcome 3: Value-added vendors who provide services and support based on Wikimedia software and true 'third parties' who install and use software produced by the Wikimedia movement on their own computers are more confident in recommending, deploying, and extending Wikimedia FLOSS projects. Objective 2: Clarify the Foundation’s short- and long-term commitments to third-party users. Create, publish, and promote a multi-tiered, third-party support level system for Wikimedia software projects. Document the support level of existing FLOSS projects and ensure that the documented levels of support are delivered. Expand the set of bundled extensions to achieve a default MediaWiki experience that's comparable to Wikimedia sites (task T178349) MediaWiki Release Engineering B: Features we build for others EOQ In progress In progress
Outcome 3: Value-added vendors who provide services and support based on Wikimedia software and true 'third parties' who install and use software produced by the Wikimedia movement on their own computers are more confident in recommending, deploying, and extending Wikimedia FLOSS projects. Objective 2: Clarify the Foundation’s short- and long-term commitments to third-party users. Create, publish, and promote a multi-tiered, third-party support level system for Wikimedia software projects. Document the support level of existing FLOSS projects and ensure that the documented levels of support are delivered. Improve extension management (task T88596) MediaWiki B: Features we build for others EOQ In progress In progress
Outcome 3: Value-added vendors who provide services and support based on Wikimedia software and true 'third parties' who install and use software produced by the Wikimedia movement on their own computers are more confident in recommending, deploying, and extending Wikimedia FLOSS projects. Objective 2: Clarify the Foundation’s short- and long-term commitments to third-party users. Create, publish, and promote a multi-tiered, third-party support level system for Wikimedia software projects. Document the support level of existing FLOSS projects and ensure that the documented levels of support are delivered. Develop a business model to support MediaWiki 3rd party usage MediaWiki B: Features we build for others EOQ In progress In progress
Outcome 4: The collaboration with research in industry and academics is further scaled and supported. Objective 1: Organize and host the annual Wiki Research Workshop (task T171118)
  • Co-run the annual workshop on April 24 in Lyon, FR (task T191092)
  • Write a report for the workshop (task T191094)
Research Workshop co-organizers A: Foundation goal (Community)

B: Serving our audiences

EOQ Yes Done

Program 5. Scoring Platform (ORES)[edit]

Goal owner: Aaron Halfaker

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Tool developers and Product teams can innovate tools that use machine prediction to make wiki-work more efficient Read and write ORES predictions for article quality, draft quality, and draft topic into MediaWiki (task T190471) Scoring Platform EOQ Yes Done
Outcome 2: Volunteers are empowered to track trends in prediction bias and other failures of AI in the wiki. Objective 1: Develop best practices for using community input to improve/correct predictions Implement a method for reading JADE data into ORES (task T190472 Scoring Platform EOQ N Postponed
Outcome 2: Volunteers are empowered to track trends in prediction bias and other failures of AI in the wiki. Objective 1: Develop best practices for using community input to improve/correct predictions Write a systems paper about ORES highlighting our version of transparent AI (task T121719) Scoring Platform April 19th Yes Done
Outcome 1: Tool developers and Product teams can innovate tools that use machine prediction to make wiki-work more efficient Objective 1: Expand vandalism & good-faith detection models to more wikis (focus on Emerging Communities) Existing ORES service support and stability Scoring Platform Operations continuous Yes Done

Program 6. Streamlined service delivery[edit]

Goal owner: Greg Grossmeier (Mark Bergsma)

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 2: Developers are able to develop and test their applications through a unified pipeline towards production deployment. Objective 2: Prove viability of testing staged service containers alongside MediaWiki extension containers
  • Add Composer support to Blubber - task T186547
  • Small, standalone, MediaWiki containers built using Blubber
  • Build extension specific containers using MediaWiki image as base
Release Engineering SRE,

Services

C: Tech Debt

D. Improve our own feature set

EOQ N Postponed

Program 7. Smart tools for better data[edit]

Overall goal owner: Nuria Ruiz

Outcome 1 owner: Nuria Ruiz
Outcome 2 owner: Marko Obrovac
Outcome 3 owner: Bryan Davis
Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Foundation staff and community have better tools to access data. Objective 1: Wikistats 2.0 redesign Analytics EOQ To do To do
Outcome 1: Foundation staff and community have better tools to access data. Objective 1: Wikistats 2.0 redesign More resilient backend for wikistats: task T177965 EOQ Yes Done
Outcome 1: Foundation staff and community have better tools to access data. Objective 2: Better visual access to EventLogging data Enable automatic ingestion from eventlogging into druid for some schemas task T190855 Analytics EOQ N Not done
Outcome 1: Foundation staff and community have better tools to access data. Objective 1: Users see improvements on data computing and data quality.: Archive old geowiki data (editors per country) and make it easily available at WMF. task T190856 Analytics EOQ Yes Done
Outcome 1: Foundation staff and community have better tools to access data. Objective 1: Experiments with real-time data and community support for new datasets available Build public eventstream endpoint on top of revision score data for every edit task T167180 Analytics EOQ Yes Done
Outcome 2: Access to Wikimedia content and data with scalable APIs Objective 1: Revision storage scaling Support for language variants:
  • Resolve the on-going disk I/O performance issues
  • Incorporate the use case for language variants into the storage design
  • Expose the language variants for Parsoid HTML and related content to clients via the REST API
Service Platform SRE, Parsing team D: Tech debt EOQ In progress In progress
Outcome 4: Users see improvements on data computing and data quality. Objective 3: Build prototype for MediaWiki content processing. For example: ingest and process text on every wikipedia page to use later for analytics-style computations. STRETCH GOAL: Build prototype for MediaWiki content processing. For example: ingest and process text on every wikipedia page to use later for analytics-style computations. task T190858 Analytics EOQ To do To do
Outcome 4: Users see improvements on data computing and data quality. Objective 3: More efficient Bot filtering on pageview data. STRETCH GOAL: Build POC for bot classifier task T138207 Analytics EOQ To do To do

Program 8. Multi-datacenter support[edit]

Overall goal owner: Tim Starling

Outcome 1 owner: Tim Starling
Outcome 1, Objective 3 owner: Mark Bergsma
Outcome 2, Objective 1 owner: Marko Obrovac
Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Our audiences enjoy improved MediaWiki and REST API availability and reduced wiki read-only impact from data center fail-overs. Objective 1: MediaWiki support for having read-only “read” requests (GET/HEAD) be routed to other data centers Fix any remaining MediaWiki blockers to multi-DC deployment
  • Performance
  • MediaWiki
D: Tech Debt EOQ Incomplete Partially done
Outcome 1: Our audiences enjoy improved MediaWiki and REST API availability and reduced wiki read-only impact from data center fail-overs. Objective 2: Test an active/active deployment for read-only requests of the MediaWiki application platform and REST APIs
  • Deploy a clustering solution for memcached in production Yes Done
  • Implement read/read-write support in the Traffic layer In progress In progress
  • Investigate solutions for MySQL connection pooling Incomplete Partially done
  • Implement mcrouter to support object caching across data centers Incomplete Partially done
  • Track changes and document the process to depool the secondary data center and switch between the primary and secondary Yes Done
  • Site Reliability Engineering
    • Service Operations
    • Traffic
    • Data Persistence
    • Infrastructure Foundations
  • Performance
  • MediaWiki Platform
  • Services Platform
D: Tech Debt EOQ
Outcome 2: Backend infrastructure works reliably across data centers Objective 1: Reliable, multi-DC job processing Complete the JobQueue transition to EventBus
  • Port the remaining jobs over to EventBus
  • Enable support for private wikis
Services Platform
  • Service Operations
  • Analytics
D: Tech debt EOQ Yes Done

Program 9. Growing Wikipedia across languages via recommendations[edit]

Goal owner: Leila Zia

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Surface relevant information about the articles to editors at the time of editing with the goal of helping editathon organizers Objective 1: Build, improve, and expand algorithms that can provide more detailed recommendations to editors about how an article could be expanded. (task task T171224)
  • Improve section translation classifier (task T190770)
  • Improve section synonym classifier (task T190771)
  • Build the first version of section recommender by fusing the synonym and translator models (task T190772)
Research External collaborators B, C EOQ In progress In progress
Outcome 1: Surface relevant information about the articles to editors at the time of editing with the goal of helping editathon organizers Objective 2: Develop and gather design requirements for how the algorithms’ results should be exposed to the editors. (task T171225) Design to allow the user of the API or the gadget see and report issues in category sequences behind a section recommendation (task T190773) Research Pau, External collaborators B, C EOQ Template:Postponed
Outcome 2: Editors can benefit from improved recommendations exposed via recommendation API, in Content Translation, and the Editor Dashboard tool. Objective 1: Continue experimenting (and implementing when applicable) algorithmic improvements on article recommendation for creation (the service behind GapFinder and Suggestions feature in Content Translation tool) Improve the prioritization algorithm used in recommendation API (task T190774) Research B, C EOQ In progress In progress

Program 10. Public cloud services & support[edit]

Goal owner: Bryan Davis

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 3: Wikimedia community members, Foundation staff, and potential contributors are aware of the breadth of products and services offered by the Cloud Services team. Objective 1: Promote available services and products at relevant conferences, hackathons, and within the Wikimedia communities Present on Cloud Services products and initiatives at the Barcelona Hackathon, May 2018 WMCS A: Foundation goal (Community)

B: Serving our audiences

May Yes Done

Program 11. Improving citations across Wikimedia projects[edit]

Goal owner: Dario Taraborelli

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Quantitative research is available to help Wikipedia and Wikidata contributors focus and prioritize their sourcing efforts. Objective 1: Estimate what proportion of content in Wikipedia or Wikidata is unsourced and in need of citations. (task T171230)
  • Analyze citation dataset by category, topic, and language (task T190438) In progress In progress
  • Model for citation gap estimations. Stretch: starting to test (task T186279) Incomplete Partially done
Research External collaborations B, C EOQ
Outcome 1: Quantitative research is available to help Wikipedia and Wikidata contributors focus and prioritize their sourcing efforts. Objective 2: Collect and analyze clickthrough data for footnotes and external links to understand how readers interact with them (task T171231) Instrumentation, data collection, first round of descriptive analysis (task T191086) Research External collaborations B, C EOQ In progress In progress
Outcome 3: Outreach activities continue to ensure community and technical efforts to improve the structure and quality of citations are aligned Objective 1: Fundraise for, and host the 3rd annual meeting in the WikiCite series Host a thematic WikiCite track at the Wikimedia Hackathon in Barcelona (task T191087) Research B EOQ In progress In progress

Program 12: Grow contributor diversity[edit]

Goal owner: Leila Zia

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: We improve Wikipedia’s contributor diversity after designing and testing potential intervention(s). Objective 3: Run experiment(s) to assess whether the recommended design will have the desired outcome
  • Test the first version of the framework designed to increase retention in contributors (women) (task T190775)
  • Test the model for eliciting new editor interests (task T190776)
Research Community :) B, C EOQ In progress In progress

Product departmental programs[edit]

Note: When the draft annual plan was written, the Search Platform and Fundraising Tech teams were in Product. They were moved into Technology during the "tune-up" in June 2017.

Program 1: Make knowledge more easily discoverable[edit]

Goal owner: Erika Bjune

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: Readers are better able to discover and search for content. Objective 1: Implement advanced methodologies such as “learning to rank” machine learning techniques and signals to improve search result relevance across language Wikipedias.
  • Implement advanced query parser features In progress In progress
  • Explore potential applications for NLP to be applied in Search Yes Done
Search Platform C EOQ In progress In progress
Outcome 1: Readers are better able to discover and search for content. Objective 2: Improve support for multiple languages by researching and deploying new language analyzers as they make sense to individual language wikis.
  • Investigate language analyzers in ElasticSearch 6
  • Continue morphological library investigations and implementations: Slovak, Estonian, Korean
  • Continue general language support, including looking at individual language analyzer config options
Search Platform C EOQ Yes Done
Outcome 1: Readers are better able to discover and search for content. Objective 3: Advanced search capabilities (e.g., Wikidata Query Service, SPARQL queries) will be updated to support the more specific media search filters and the relationships to the topics they represent.
  • Incorporate Lexeme Forms into lexeme search mechanism
  • Gather Lexeme search requirements and begin prototyping, starting with search completion
Search Platform, WMDE C EOQ Yes Done
Outcome 1: Readers are better able to discover and search for content. Technical Debt/Maintenance Prepare CirrusSearch for Elasticsearch 6 (T194199) Search Platform D EOQ Yes Done
Outcome 2: Through improvements to the user experience of search interfaces, users are better able to find content in an intuitive and easy way. Objective 1: Investigate how to expand and scale Wikidata Query Service to improve its ability to power features on-wiki for readers
  • Productionize six new servers for WDQS, (see T178548)
  • Investigate memory allocation rate issues on WDQS
Search Platform WMDE C EOQ Yes Done

Program 7: Payment processor investigation and long-term strategy[edit]

Goal owner: Katie Horn

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome: Advancement and fr-tech find a solution that lowers or does not increase current maintenance costs. We are continuing with the Ingenico reintegration and scoping Contribution tracking Make Ingenico "campaign ready" Fr-tech Advancement operations and Banner team D EOQ In progress In progress

Program 8: Donor retention[edit]

Goal owner: Katie Horn

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome: In FY 16/17 four well-trained team members spent 106 hours on deduping related to the English campaign. One of those employees may not be available for this task next year and a new contractor may help. We still hope to reduce the hours spent. Switch to support of manual dedupe supporting manual dedupe and scoping next year Fr-tech Major Gifts, Donor Services, Email team B EOQ In progress In progress


Cross-departmental programs[edit]

Community Health[edit]

Segment 3: Research on harassment[edit]

Goal owner: Dario Taraborelli

  • Note: no specific goals for this quarter

Privacy, Security, and Data Management[edit]

Segment 2: Privacy and Data Management[edit]

Goal owner: Nuria Ruiz

Annual plan outcome Annual plan objective(s) Quarterly Work Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 2: To protect user data and uphold movement values, the Wikimedia Foundation continues compliance with best practices for data management
  • Support teams and departments as needed
  • Data Mapping - Pivoting data categories by type for analytics data
Analytics, Legal, Security Ongoing In progress In progress
Outcome 3: Maintain and enhance connection privacy and security Objective 2: Keep up with evolving public-facing TLS Standards and enhancements (e.g. HPKP, TLSv1.3, ciphersuites) Establish timeline and methodology for upcoming deprecation of non-forward-secret ciphers and TLSv1.0; begin execution of non-forward-secret ciphers deprecation. SRE/Traffic CE/Community Liasons D EOQ To do To do

Segment 2: Security[edit]

Note: When the annual plan was written, the Security team's goals were not part of the annual plan, but they still needed to be addressed.

Goal owner: John Bennett

Quarterly Work (or Goal) Primary Team(s) ETA Status
  • Hiring: Define job descriptions and start seeking other reqs Yes Done
  • Privacy: Create and train on WMF PGP keys N Not done
  • Ongoing security reviews as requested In progress In progress
  • Create Incident Response plan and hold Table Top exercise In progress In progress
  • Solidify compliance In progress In progress
Security EOQ

Structured Data on Commons[edit]

Segment 1: Database Integration[edit]

Goal owner: Tim Starling

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 2: Introduce Multi-Content Revisions Objective 2: Enable saving components to use the new backend. Update transaction management facilities to ensure changes to one or more types of content in a page are committed safely in the database and related systems.

Assist with the implementation of the MCR storage layer by completing the following tasks (phase 1):

MediaWiki Platform, WMDE B, C EOQ In progress In progress

Segment 2: Search integration and exposure[edit]

Goal owner: Erika Bjune

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 1: It is easier for people to discover, learn, and manage the free media stored on Commons and thereby incentivize higher contribution rates. Objective 1: Commons search will be extended via CirrusSearch and Elasticsearch and Wikidata Query Service, to support searching based on structured data elements describing media.
  • Search backend for file captions
  • Inform “full-search” specifications and begin prototyping
Search Platform, Multimedia C EOQ To do To do

Segment 4: Programs[edit]

Goal owner: Jonathan Morgan

Annual plan outcome Annual plan objective(s) Quarterly Work (or Goal) Primary Team(s) Dependencies Tech Goal ETA Status
Outcome 2: Develop a better understanding of existing needs for Structured Commons Objective 2: Write case studies and documentation for Commons and Wikidata projects that allow project development among Wikimedia Communities and allow us to identify gaps in existing tools (task T171252).
  • Publish Commons curator research report (task T175185) Yes Done
  • Conduct and publish Commons reuser research (task T190228) Incomplete Partially done
Research Community programs, Multimedia C Q1 FY18/19 Incomplete Partially done