Wikimedia Engineering/2015-16 Q2 Goals

From mediawiki.org
Q1 Wikimedia Engineering Goals, FY2015-16, Q2 (October–December) Q3

Purpose of this document: Goals for the Wikimedia Product and Technology groups (collectively, Wikimedia Engineering), for the second quarter of fiscal year 2015–16 (1 October 2015 – 31 December 2015). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress In progress, To do To do, N Postponed, Yes Done or Incomplete Partially done

Product[edit]

Goal setting process owner: Dan Garry

We build the anonymous path of discovery to a trusted and relevant source of knowledge.

Our focus will be on building on-top of our anonymous and trusted search stack to improve relevancy, expose additional content and leverage locally relevant knowledge.

For the below goals, the objectives are relatively finalised, but the key results and dependencies still need further definition before the goals are locked in.

Search[edit]

Objective Key result Dependency ETA Status
Improve language support for search.
  • Run A/B test for a feature that:
    • Uses a library to detect the language of a user's search query.
    • Adjusts results to match that language.
  • Determine from A/B test results whether this feature is fit to push to production, with the aim to:
    • Improve search user satisfaction by 10% (from 15% to 16.5%).
    • Reduce zero results rate for non-automata search queries by 10%.
  • Operations: Wikimedia Labs (maintaining Labs)
  • Discovery Analysis Team
Dec 2015 Yes Done
Make www.wikipedia.org a portal for exploring open content on Wikimedia sites.
  • Measure usage of existing portal.
  • Perform A/B tests to improve portal.
  • Decrease time each user spends searching by Y ms (exact number pending rollout of event logging schema on the portal).
  • Operations: Wikimedia Labs (maintaining Labs)
  • Community Engagement (support for communications around portal improvements)
  • Discovery Analysis Team
Dec 2015 N Not done

Wikidata Query Service[edit]

The Wikidata Query Service Team achieved its Q1 goal to release the Wikidata Query Service to production, with a public endpoint. In Q2, development on the project will be put on hold, user feedback about what features are missing will be collected, and usage of the service will be measured over the course of Q2; this will be used to assess whether further development will be required in Q3.

Maps & Geo[edit]

The Maps Team is currently on track with its Q1 goal to release the Maps tile service to production, with a public endpoint, in Q1. In Q2, development on the project will be put on hold, user feedback about what features are missing will be collected, and usage of the service will be measured over the course of Q2; this will be used to assess whether further development will be required in Q3.

Analysis[edit]

Objective Key result Dependency ETA Status
Improve understanding of user satisfaction for search by iterating and improving on the search satisfaction metric.
  • Design QuickSurvey to qualitatively assess user satisfaction with search.
  • Tie qualitative QuickSurvey results back to quantitative search satisfaction data.
  • Validate/invalidate current approach for measuring satisfaction.
  • Operations: Wikimedia Labs (maintaining Labs)
Dec 2015 N Not done
Support ongoing evaluation of usage of Wikidata Query and Maps services to decide on what's next for these services.
  • Continue maintenance of Wikidata Query Service and Maps dashboards.
  • Review user feedback on services.
  • Make decision on continued level of support for WDQS and maps service.
  • Operations: Wikimedia Labs (maintaining Labs)
Dec 2015 Yes Done
Create recurring performance indicator on referrer traffic from the primary search engines and determine what features may have largest impact on referrer metric.
  • Create dashboard which displays traffic to our sites broken down by major referrers.
  • Based on the dashboard data, make a list of recommendations for features which could be implemented to increase traffic from prominent referrers.
  • Operations: Wikimedia Labs (maintaining Labs)
Dec 2015 N Not done

Notes:

  • In response to a request from the Head of Research and the Vice President of Product, the Discovery Analysis Team dedicated one of its analysts at 33% capacity to the annual fundraising campaign to aid in ensuring its success. This reduction in anticipated capacity places some of the above goals, particularly the first one, at risk.

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department build collaborative, inclusive tools for creating and editing free knowledge. In Q2, the Department will focus on rationalising technologies and strengthening engineering practices whilst delivering on the key user workflows for editors across the product range.

Editing Departmental level goals
Objective Key result Dependencies ETA Status
Focus: The visual editor and other editing improvements released to less-active accounts and logged-out users on the English Wikipedia.
  • The visual editor available to more accounts and logged-out users on the English Wikipedia; opt-in is visible to all.
  • Cross-wiki notifications.
  • Add support for inserting and editing more kinds of rich content using visual editor.
  • Image upload as you edit from the toolbar in wikitext and the visual editors, and via drag-and-drop into the visual editor.
Community Liaisons

Communications

Design Research

EOQ Incomplete Part done; some parts delayed
Strengthen: Prototype and research suggestion interfaces for the visual editor and ContentTranslation.
  • Designs for "product"-ifying the suggested-edit features being prototyped and researched are ready to be implemented in Q3.
Design Research

Research & Data

EOQ Incomplete Partially done
Experiment: Experiment with other kinds of editing and content.
  • Incorporate two additional extensions to VisualEditor.
  • Research VisualEditor on mobile and use of video on Wikipedia.
Design Research EOQ Incomplete Partially done

VisualEditor Team[edit]

Involve more people than ever before in the collaborative effort of building free knowledge through improving the usability, stability and performance of VisualEditor, integrated with the developing reading experience.

Objective Key result Dependencies ETA Status
Focus: Increase use of VisualEditor by releasing VisualEditor to more accounts and anonymous users on English Wikipedia More editors use VisualEditor Community Liaisons EOQ Incomplete Delayed by scope increases
Strengthen: Improve support for math editing by improving the UX of the Math extension and researching its usability It's easier to create and edit formulæ from within VisualEditor EOQ Yes Done
Strengthen: Improve support for graph editing by improving the UX of the Graph extension and researching its usability It's easier to create and edit charts from within VisualEditor EOQ Yes Done
Experiment: Increase value of contributions by prototyping and researching integrating edit recommendations We have an informed plan for next steps for integrating edit recommendations Design Research

Research & Data

EOQ Incomplete Delayed by other priorities
Experiment: Improve VisualEditor on mobile devices by prototyping and researching using VisualEditor on mobile devices We have an informed plan for next steps for using VisualEditor on mobile devices Design Research EOQ Incomplete Partially done

Collaboration Team[edit]

Increase the participation, mentoring and awareness of activity on the site by bringing structured conversations to both talk pages and the editing experience by improving and deploying Flow where it can help most.

Objective Key result Dependencies ETA Status
Focus: Improve awareness of activity by providing a cross-wiki notifications feature Echo can deliver notifications between many wikis, allowing users to access their notifications from any content wiki on any other content wiki Security

Community Liaisons

Technical Operations

EOQ In progress In progress
Strengthen: Increase access to Flow by deploying and supporting an opt-in system for it Users have the ability enable Flow on their talk page Community Liaisons EOQ In progress In progress
Experiment: Increase engagement with notifications by prototyping and researching improvements to the notifications page We have an informed plan for the next steps for improving the notifications page Design Research EOQ In progress In progress

Multimedia Team[edit]

Enrich the content of our projects with more use of multimedia assets, by streamlining media contribution with UploadWizard, and evolving the rich editing experience to better suggest media.

Objective Key result Dependencies ETA Status
Focus: Increase media contribution by integrating media upload into VisualEditor's media dialog and drag-and-drop system Users can upload media files directly within VisualEditor by browsing or dragging-and-dropping EOQ Yes Done
Strengthen: Improve reliability by resolving UploadWizard bugs that cause users to start over Users can use UploadWizard to mass-upload without interruption EOQ Incomplete Partially done
Experiment: Improve multimedia editing workflow by prototyping non-destructive image editing Users can make basic changes to images such as crop, rotation, etc. directly on the wiki Performance EOQ Yes Done
Experiment: Improve value of content by researching the learnability of content to inform future prioritisation of feature development We have an informed plan for next steps for multimedia contribution features Design Research EOQ Incomplete Partially done

Language Team[edit]

Growing and accelerating projects with smaller content bases by providing and promoting enhanced Content Translation tools for translating content from projects with larger content bases.

Objective Key result Dependencies ETA Status
Focus: Improve user retention by adding additional 'suggestion' features such as customised lists and adding additional relevant notifications Increase in the number of translations per user Research & Data EOQ Yes Done
Strengthen: Improve reliability by complying with WMF infrastructure requirements as defined by Services, Tech Ops and Security teams ContentTranslation complies with WMF infrastructure requirements Ops, Services EOQ Yes Done
Strengthen: Improve reliability by fixing high-priority bugs that affect basic functionality Users can use ContentTranslation without interruption Parsing EOQ Yes Done
Experiment: Increase visibility for third parties by collecting and sharing parallel corpora of translation modifications API will be able to access the parallel corpora Ops EOQ Incomplete Partially done

Parsing Team[edit]

Increasing the integrity, accessibility and stability of content by providing Parsoid as both a bridge for editing wikitext documents, but also a rendering system for viewing and querying them.

Objective Key result Dependencies ETA Status
Focus: Improve editing experience and draw closer to Parsoid HTML for read views by implementing support for native Parsoid extensions Parsoid has an extension registration mechanism and native support for galleries EOQ Yes Done; ‎<gallery> will be implemented in Q3; ‎<translate> and testing extensions done
Strengthen: Improve mobile reading experience and draw closer to Parsoid HTML for read views by improving multimedia support in Parsoid Parsoid's HTML5 DOM spec and HTML generation have been updated for audio and video EOQ N Postponed; Deprioritized below other requests; Will be picked up in Q3
Strengthen: Modernise parsing infrastructure to use modern Web standards by replacing Tidy with the HTML5 parser in MediaWiki core MediaWiki has an HTML5-compliant parser for fixing PHP parser output EOQ Incomplete Partially done - rollout is more complex than anticipated; Incremental roll-out of smaller pieces
Experiment: Improve editing experience and take advantage of performance optimisation opportunities by prototyping some form of opt-in / opt-out system for well-balanced template output MediaWiki has a mechanism for templates to specify their output as balanced or not EOQ Incomplete Partially done; Implementation sketch in place; Prototypes to be done; Has to go through RFC process

Goal setting process owner: User:AGomez_(WMF), User:Khorn_(WMF)

Objective Key result Dependencies ETA Status
Focus: Support the Big English fundraiser Fundraising creative reaches their donation goals in December
  • Fundraising (creative) team
21/1 - 12/31 Yes Done
Strengthen: Get France campaign tech-ready Fundraising creative reaches donation goals in October (in France)
  • 3rd Party Integration Support (WorldPay)
  • Fundraising (creative) team
10/23 Yes Done
Experiment: Support Banner History rollout Fundraising creative is able to see patterns and problems with banner consumption, before December
  • Analytics (Ellery)
  • Fundraising (creative)
  • Performance engineering
  • Release engineering
10/1 - 10/31 Yes Done
Experiment: Start investigating SmashPig as its own code project

Rough estimate of the work involved to create the open source project -OR- a decision not to move forward with independent payment libraries

Unknown

EOQ N Postponed - Extremely disruptive conversations were endangering our mission-critical Q2 fundraising goals. Postponed for a less complicated time.
Objective Key result Dependencies ETA Status
Strengthen: Drive engagement by launching Link Preview on mobile web beta and Read More on web beta Engagement (5% increase in links clicked) in beta or on non-EN Wikis (Link Preview T111329, Read More T94906)
  • Community Liaisons
EOQ Incomplete Partially done

Read more is on beta, link preview is on hold

Experiment: Drive app retention via feeds and modern UX (notifications Q3) iOS App overhaul released

7 day 15% of users retained (up from 10% in Sept ‘15) (Feeds T104415, Modernization T104510)

  • Design Research
EOQ Incomplete Partially done

Notifications pilot in place.

Focus: Measure MediaWiki API usage, and data dump loads. Availability of usage metrics  on Hadoop warehouse for analysis. (T102079)
  • Partnerships
  • Analytics team
EOQ In progress In progress
Strengthen: Investigate migrating API traffic to OAuth. Decision on if migration is desirable; then measurable impact based on existing usage (currently unknown)
  • MediaWiki extension developers
  • Security team
EOQ To do To do
Strengthen: Release Auth Manager: MediaWiki plug-in security architecture Usage (# of services using new architecture) (T89459)
  • MediaWiki extension developers
  • Security team
EOQ In progress In progress
Experiment: Prototype services based architecture for web and apps Working web app with feature parity of current mobile web stable reading experience (not scalability, quality). (main task T113066, Wikimedia Developer Summit 2016 discussion T114542)
  • Services team
EOQ Yes Done

UI Standardization team[edit]

Goal setting process owner: Volker Eckl

Objective Key Result Dependency ETA Status
Basic stylesheet of accorded design properties

Basic stylesheet ("base.css") of generally accorded design properties/values to spec implemented in a central repo & accompanying HTML.

Designers on web-facing teams, community involvement EOQ In progress In progress
Enhance accessibility of
Living Style Guide (Blueprint) & WMF libraries
  • Color palette that is compliant to WCAG 2.0 guidelines at minimum level AA for Wikimedia projects' usage (including for use in basic stylesheet and Living Style Guide (Blueprint)).
  • Enhance OOjs UI & mediawiki.ui library to increase reasonable WAI-ARIA best practices accomplished in design and code.
Editing (OOjs UI), community involvement EOQ In progress In progress
Style Guide
  • Initial step: Understand what non-technical contributors need in their work
Design Research team EOQ In progress In progress
Objective Key results Dependency ETA Status
Strengthen: Establish baseline for measuring community satisfaction with WMF tech support Completed Tech Support survey across multiple Wikimedia projects and languages Learning and Evaluation support Mid-November Yes Done
Focus: Include Wikimedia communities in goal development and prioritization Completed cross-project Community Wishlist survey, with community members voting on priority EOQ In progress In progress
Focus: Address current top-priority community technical requests At least 3 mid to large-sized requests from existing wishlists EOQ In progress In progress

Technology[edit]

Goal setting process owner: Kevin Leduc

In progress In progress

Objective Key result Dependencies ETA Status
Focus: Use Hadoop infrastructure to replace computation of Wikistats Replicate 5 of wikistats most popular pageview reports defined here. Research (Erik Zachte) In progress In progress
Strengthen: Pageview API API is operationalized and ready to be queried by outside world.  A blog post is published on Wikimedia’s blog. Services Team Yes Done Blogpost
Experiment: Build infrastructure Scalable Event Bus A service to produce to kafka with a standardized schema is running in production. Services Team In progress In progress

Goal-setting process owner:  Rob Lanphier

(previously discussed/published at WMF, but not posted here until 2015-12-11)

Objective Key result Dependencies ETA Status
Focus: Prepare WikiDev '16 Demonstrate that we can develop a modern system in an inclusive, consensus-oriented, open manner.  Large number of diverse RfCs, with ample discussion prior to WikiDev '16 Wide participation in Engineering 2016-01-04 Yes Done - Preparation happened, summit resulted  :-)
Strengthen: Improve ArchCom utility Set up Architecture Committee to succeed.  Maintain one public IRC meeting per week
  • ArchCom members at >20% time
  • We may want/need to expand ArchCom
2015-12-31 Yes Done - 10 IRC RFC review meetings, plus 3 “Agenda bashing” meetings for WikiDev ‘16.  Most were at the usual time (14:00 PT)

Phab query for meetings

Experiment: ArchComm naming Think through our loaded naming scheme, possibly renaming bits (e.g. “Architecture Committee” and “RfC”) Consultation over naming 2015-12-31 N Not done - oh well

Goal-setting process owner: Ori Livneh

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The following is a summary of the 2015–16 Q2 Performance team goals.

Objective Key results Dependencies Estimated time of arrival Status
Availability Make MediaWiki code ready for concurrent deployment across multiple data centers. This is a milestone on the way to full deployment (T88445). TechOps EOD In progress In progress
Reporting We'll publish weekly performance reports that provide clear and compelling insights into important performance characteristics. None When ready Yes Done, but cadence is monthly for now.
Faster painting We want a 10% reduction in the time-to-first paint across desktop and mobile. None When ready Yes Done

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201516-q2 - More details at Wikimedia_Release_Engineering_Team/Goals/201516Q2

Key Performance Indicators

Objective Key result Dependency ETA Status
STRENGTHEN

Reduce CI wait time

Wait time KPI

Operations / WMF Labs EOQ N Not done
FOCUS

Reduce number of deploy tools from 3 to 2

  • Migrate all Service team owned services and MW deploys to scap3 - task T109926

Services / Operations

EOQ N Not done
FOCUS

Retire Gerrit and Gitblit in favor of Phabricator

  • Decommission Gitblit - task T111465 - N Not done
  • Code review RFC: creation, publication, discussion, feedback etc - task T114311 - Yes Done
(small) Operations / DevRel EOQ N Not done
FOCUS

Release MediaWiki 1.26

WMF Engineering

MW Technical Community

Mid-Nov Yes Done


In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data[edit]

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective Key results Dependencies ETA Status
FOCUS

Streamline ownership of research / data-related work

Publish a map for internal use of who is responsible for specific requests around research and analytics. (T112317)
  • Team / department leads
10/1-10/31 In progress In progress
STRENGTHEN

Revscoring integration

Bring revscoring to fruition to our users as a Beta Feature (score integration into RC feed) (T112856)
  • Ops
  • Design
  • Discovery
EOQ In progress In progress
STRENGTHEN

Productize article and link recommender systems

Article creation (T112321) and link (T112322) recommendation productized as services so they can be integrated with products and community tools.
  • Ops
  • Editing
EOQ In progress In progress
EXPERIMENT

Value-added research

Deliver research on historical data to quantify who adds value  to English Wikipedia (T99172)
  • Altiscale cluster
EOQ In progress In progress
EXPERIMENT

Reader segmentation research (stretch)

Deliver research on reader segments and behavioral patterns (T112326)
  • Analytics Dev (spiky cluster usage)
  • Research fellows
EOQ In progress In progress

Design Research[edit]

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective Key result Dependencies ETA Status
STRENGTHEN

Mentor people to do quality design research (those not on DR team).

One person outside the design research team does a complete design research project to DR team standards, expanding the number of people who do design research well.
  • Person outside the DR team having capacity to do a design research project.
EOQ Yes Done
STRENGTHEN

Design Research workshop to build collaboration capacity with product teams.

Workshop happens, and product teams learn more about collaboration with Design Research and vice versa.
  • Availability of workshop participants to attend (track leads, managers and individual contributors from each product track).
MidQ Yes Done
STRENGTHEN

Improve participant recruitment process

Implement changes to reduce steps in recruitment (of design research participants) process to make recruiting more efficient.
  • n/a
EOQ Yes Done
FOCUS

Collaborate with product teams to implement design research projects.

Design Research has done design research projects in collaboration with product teams to inform human centered product development.
  • All product teams we collaborate with
  • Research participants show up to research sessions (volunteers)
EOQ Yes Done
EXPERIMENT

Persona interviews

12 persona interviews completed
  • Design research participants show up to research sessions (volunteers)
EOQ Incomplete Partially done

Goal setting process owner: User:CSteipp (WMF)

We make life hard for the people who want to harm our sites or the people that use them.

Objective Key result Dependency ETA Status
Automated security static analysis of MediaWiki
  • Evaluate potential tools
  • If a suitable tool can be found, automatically scan core and at least one extension weekly.

None

EOQ Yes Done
Expand developer training
  • Present secure code and secure SDL training for community and staff
  • (stretch) Develop and present security training materials for DevOps and Mobile

None

EOQ In progress In progress
(stretch) Document and report initial metrics for security reviews
  • Metrics reported

None

EOQ To do To do

Goal setting process owner: Gabriel Wicke

For details, see the Phabricator task.

Objective Key result Dependencies ETA Status
Strengthen: Build out the REST API Provide cached & purged REST API entry points for at least two high-traffic API use cases, and work with users to speed up page loads. Reading, Editing

EOQ

Yes Done

Focus: Set up an event bus and change propagation solution Key change events like edits, renames, deletes available in a publish-subscribe event bus. Basic change propagation / pre-generation functionality for REST services operational. Analytics EOQ

In progress In progress

Experiment: Prototype an API-driven web front-end Gather information to inform our longer-term front-end architecture by prototyping a service-worker / node.js based front-end. Reading EOQ

Yes Done

Goal setting process owner: User:Mark Bergsma (WMF)

Objective Key result Dependencies ETA Status
Focus: Encrypt PII on cross-data center links
  • Encrypt cross-data center traffic for MySQL/MariaDB, Kafka, Cassandra, (Varnish) HTTP
  • Decommission udp2log (in favor of Kafka)
  • Analytics (Kafka, udp2log)
  • Fundraising (udp2log, Kafka)

EOQ

Yes Done

Strengthen Security: Strengthen infrastructure user authentication
  • Establish organisation-wide workflow for creation and deactivation of user accounts in Production
  • Implement (but don't enforce) two-factor authentication on SSH bastion hosts
  • Migrate unmaintained OpenDJ LDAP to OpenLDAP
  • HR & OIT (for workflow standardization)
  • Procurement (for 2fa hardware)
EOQ

Yes Done

Experiment: Support bare metal servers in Labs Allow provisioning of physical servers within the Labs network, governed by Labs Puppetmaster and LDAP user accounts EOQ

Yes Done

Strengthen Incident monitoring infrastructure
  • Upgrade incident monitoring and expand it to codfw Yes Done
  • Reduce monitoring check execution lag to 0 Yes Done
  • Create new abstractions for incident monitoring in Puppet Incomplete Partially done
EOQ Incomplete Partially done