Wikimedia Engineering/2016-17 Q1 Goals

From mediawiki.org
Q4 Wikimedia Engineering Goals, FY2016-17, Q1 (July–September) Q2

Purpose of this document: Goals for the Wikimedia Technology and Product departments, for the first quarter of fiscal year 2016–17 (July 2016 –September 2016). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders. Guidance and planning are in accordance with our Annual Plan and are subject to change based on feedback, testing and analysis.

Status fields can use the following templates: In progress In progress, To do To do, N Postponed, Yes Done or Incomplete Partially done

ETA fields may use the initialism EOQ (End of Quarter).

Product[edit]

We build the anonymous path of discovery to a trusted and relevant source of knowledge.

Our focus will be on building on-top of our anonymous and trusted search stack to improve relevancy, expose additional content and leverage locally relevant knowledge.

Goal setting process owner: Dan Garry

Team Objective Key result Dependency ETA Status
Search Improve search result relevance and enable fixes for multiple relevance issues by switching from tf–idf to BM25
  • Upgrade to Elasticsearch 2 completed last quarter
September 2016 Incomplete Partially done
Evaluate and devise plan for using cross-project indices, which would allow users to search across all projects in any given language
  • Plan generated for switching to using cross-project indices, with rough estimates of complexity, time, and resources involved.
  • Community Liaison
August 2016 Yes Done
Maps & Geo

Goal setting process owner: Yuri Astrakhan

All tracked in: #maps - More details at Wikimedia Interactive Team/Goals/201617Q1

Make <maplink> tag available for all wiki projects, and <mapframe> available everywhere except Wikipedia
  • Maplink available on all projects
  • Mapframe available everywhere except Wikipedia
  • Operations: Heavy operations participation will be needed to stabilize map server configuration and monitoring.
EOQ In progress In progress
Offer community a way to migrate away from GeoHack
  • Implement a Special page (for legacy browsers) that allows users to migrate away from Geohack
  • Extend maplink with configurable links to other mapping services information tab
  • Community liaison
EOQ In progress In progress
Wikipedia.org Portal Make the www.wikipedia.org portal a jumping off point to explore open content on Wikimedia sites.
  • Finish running A/B tests (that were started in Q4 2015-16) to decrease the portal's bounce rate while increasing visitor click-through into rich and interactive content within Wikipedia and Wikimedia.
  • Graduate successful A/B test(s) to production.
  • Graduate translated sister wiki project descriptive text into production.
  • Update article statistics frequently.
  • Operations: Wikimedia Labs (maintaining Labs for dashboarding)
  • Discovery Analysis Team
July – September 2016 Yes Done

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department build collaborative, inclusive tools for creating and editing free knowledge. Each team is working toward the department's annual goals which are currently being finalized.

Annual Goal Team Objective Outputs Outcomes Dependencies ETA Status
1: Maintain and incrementally improve current content creation and curation interfaces VE All our editing tools are pleasing and simple to use, being clear, consistent, and coherent between platforms and technologies A new wikitext editor, designed like the visual editor, integrated into its workflow, and with all its tools, is available as a Beta Feature on desktop Users can have a unified, consistent editing interface between mobile and desktop Web, and between visual and wikitext editors, if they opt-in. None required; validation of experience from Design Research would be appreciated, but is not a blocker. EOQ Incomplete Partially done — New tool still undergoing testing, production release in Q2.
2: Improve editing experience by reducing technical and product debt MM Begin support for the structured meta data work for Commons with the image annotations gadget. Migration of the well-loved Commons image annotations gadget into its own extension, with initial work on prototyping storing structured metadata in a well-formed way based on the forthcoming multi-content revision system. Users on Commons will continue to see image annotations as before, and pages will load slightly faster; in future, tools will be able to use annotations more readily and in more contexts. None of which we know. EOQ Incomplete Partially done — New tool still undergoing testing, production release in Q2.
P Begin support of wikis with multiple writing systems in Parsoid Initial goal: Parse LanguageConverter markup and represent as Parsoid DOM. The output will not (yet) be postprocessed into a single variant; that’s the next phase of the work.  It will just be a faithful representation of the syntax for those pages without “ugly markup” At the end of this phase of the work, the goal is to parse most -- not all -- markup on zhwiki. We will quantify the number of pages with problematic markup. None that we know of yet. EOQ Incomplete Partially done — Feature nearing completion, not yet ready for release.
D Make the styleguide a pleasant resource Overhauled, up-to-date online documentation in one place, addressing designers' & developers' needs. Accompanies technical documentation, not a replacement. Users have one place to go for guidelines on UI fundamentals (fonts, color palette, widget essentials, code examples). Welcoming place for contributors. Design Research, WMF designers, OOUI devs & consumers, TPG EOQ Incomplete Partially done — Documentation enhanced, new accessibility for colours and design in place.
3: Invest in new types of content creation, and new forms of curation and collaboration tools C Provide an edit-review process suited to the needs of new contributors, who, research shows, are particularly vulnerable to rejection, especially by powerful edit-review programs. To make a beginning towards this goal, we will create a process that enables edit-reviewers to identify the edits of good-faith new users, so that these edits can be reviewed separately. Our immediate goal is for edit reviewers to be able to identify good-faith new users whose edits need help. (Our ultimate goal, toward which this is a first step, is to provide a better new-user review experience on the theory that this will have an effect on new editor retention.) Support with using ORES from the Research and Data team; ORES deployment from the Ops team (ongoing); data analysis assistance from Editing Product EOQ Yes Done — Extensive tool & process design done with rounds of user testing, engineering now ready to commence.
l10n Stabilize Content Translation for going out of beta High priority blocker bugs further stabilizing the publishing process and content enrichment (like template adaptation) are resolved Content Translation is more stable in terms of article publishing, requiring less maintenance work from patrolers, and more ready to be enabled by default for all users. The Parsing team. EOQ Yes Done — Lots of stability bugs fixed. Improvements to block and inline template adaptation also mostly complete.

Goal setting process owner: User:Khorn_(WMF)

Technical work supporting the Wikimedia Foundation's fundraising efforts.

Objective Key Result Dependency ETA Status
Replace ActiveMQ
  • Replace it by the end of July
  • Stabilize for the remainder of the quarter
none New Queue solution on production by August 1st Yes Donenew service is live. currently fixing bugs
Better Emergency messaging
  • Tune up our monitoring and alert systems (part of T136169)
    • Create high-level error signals, which in the future can be plugged into icinga or fail2ban.
none EOQ N Postponed

de-prioritized due to queue work

Central Notice Banner sequence test https://phabricator.wikimedia.org/T135398 none EOQ In progress In progress de-prioritized due to queue work

Note: the new Queue was not deployed by August 1st. However it was live by EOQ and we are fixing bugs. We would have wanted it to be more stable by now. This delay pushed out all other work.

Objective Key result Dependency ETA Status
Web: Refine and move language switching improvements from beta to stable on most wikis (T137932) Easier language switching for our users is expected to lead to an increase in language switching
  • Community Liaisons
  • Language team (as partners)
EOQ Yes Done
Android: Navigation overhaul Increase engagement by centralizing more of the app's features into a single interface, and surfacing interrelated features more prominently.
  • Design
  • Design-Research
EOQ Yes Done
iOS: Notifications pilot Continue to improve medium-term retention by piloting content based notifications or other push mechanisms. This pilot might consist of a beta test, a/b test or other small scale roll-out.
  • Design
  • Design Research
Yes Done

Goal setting process owner: Danny Horn

Team focused on meeting the needs of active Wikimedia contributors for improved, expert-focused curation and moderation tools.

Objective Key result Dependency ETA Status
Increase core contributors' productivity Define a technical implementation plan for Cross-wiki watchlist (#4 on wishlist), in collaboration with core tech stakeholders
  • Performance
  • Ops
EOQ Yes Done
Improve contributors' ability to identify and fix copy violation problems on EN.WP Launch a CopyPatrol tool (#9 on wishlist), building on existing tools by Eran and Earwig EOQ Yes Done
Increase WikiProjects' productivity Release PageAssessments extension, to enable bot and gadget creators to build tools for WikiProjects EOQ Yes Done
Increase Community Tech investment in supporting broader range of programs and communities Define a prioritized wishlist for community-facing features and fixes identified in collaboration with the Program Capacity and Resources teams, for a shared backlog of Community Tech and Technical Collaboration
  • Program Capacity & Learning
  • Resources
  • Community Engagement
  • Technical Collaboration
EOQ Yes Done
Improve volunteer developers' productivity, and encourage them to publish source code Launch Striker, a tool that connects developers' LDAP account with their Wikimedia user account, and allows them to easily create git repositories associated with tool accounts.
  • Labs
  • Security
  • Ops
EOQ Yes Done
Reduce barriers for new volunteer developers Extend Striker to help new developers create an LDAP account and manage their SSH keys, removing barriers in the new account creation process
  • Labs
  • Security
  • Ops
EOQ In progress In progress
Encourage community maintenance of important tools Lead a community discussion to define a new policy for how users can take over abandoned tools
  • Tool Labs Community
EOQ Yes Done

Goal setting process owner: Kristen Lans

The Team Practices Group (TPG) supports the Wikimedia Foundation (WMF) teams in their continual improvement.

Mission: TPG believes that healthy teams reliably deliver customer value and innovative products. Through dedicated resourcing, coaching, and workshops at the Wikimedia Foundation, we facilitate the use of practices that support team health, with a strong emphasis on creating conditions of clarity, trust, and continuous improvement.

The Team Practices Group (TPG) was dissolved in 2017.

FY2017-18 First Quarter (Jul to Sep 2016)[edit]

Foundation Strategic Direction Annual Goal Quarterly Goal Output Dependencies Outcome
Annual Plan Program Annual Plan Goal Annual Plan Objective
Reach, Community, Knowledge Facilitate the use of practices and processes that support WMF teams Support teams needing embedded TPG support.

Support teams needing lightweight coaching or workshops.

Tracking: Ongoing TPG Work

Service levels of support are maintained:

Maintain net promoter score of at least 40 for embedded support.

- Design Research Teams are equipped with appropriate practices and processes to support their work.
Reach, Community, Knowledge Facilitate the use of practices and processes that support WMF teams Identify initiatives that could benefit from TPG support:

Support Annual Planning retrospective activities

Tracking: T138241

Service levels of support are maintained:

Key improvements to Annual Planning are identified and documented.

- Finance

- Budget Owners

Teams are equipped with appropriate practices and processes to support their work:

The organization is positioned to improve how it conducts Annual Planning.

Reach, Community, Knowledge Iterative and incremental support improvements Identify areas of greatest strength and improvement for TPG:

Identify a common pain point across teams we interact with, and implement potential solutions.

Tracking: T139630

TPG understands areas for improving its services:

Document the problem(s) we identified, the solution(s) we tried, what we learned, and how effective our efforts were.

- TPG-supported teams

- Design Research

TPG engages in continual improvement:
  • Shared knowledge across TPG is increased.
  • Teams' ability to deliver value is improved.
Reach, Community, Knowledge Iterative and incremental support improvements Identify areas of greatest strength and improvement for TPG:

Plan: Light engagement survey

Tracking: T139549

TPG understands areas for improving its services:

Experiment with collectiong metrics for light engagements

- Light Engagement customers

- Design Research

TPG engages in continual improvement:

We start to understand how satisfied our light engagement customers are, and how we can achieve a high impact with low effort.

Technology[edit]

Goal setting process owner: Nuria Ruiz

These goals are preliminary until the end of June 2016, goals are in priority order.

Feedback can be sent to analytics@lists.wikimedia.org (public e-mail list)

Objective Key result Dependency Outcome/Rationale/Value Proposition Status
Operational Excellence
  • Better response times pageview API task T124314 (carry-on goal from last quarter)
  • Kafka: Upgrade Kafka on non-analytics cluster: task T138265
  • Kafka: Get mirrormaker puppetized task T138267
  • Better redundancy pageview API
Set of tasks needed to keep lights up and update/upgrade existing systems to maintain and improve current level of service Yes Done
Wikistats 2.0 (ongoing goal) .Ticket: task T130256.
  • MediaWiki changes to publish data to fullfil schemas task T138268
  • Solve scalability and query issues with high cardinality of dimensions. task T138269
  • Host edit data on Druid (or other datastore?)
Enables Easy consumption of edit data. At this time consuming edit data from database or dumps is cumbersome Yes Done
Public Event Stream (ongoing goal). Task: task T130651. Details: here POC for a public Event Stream Note: carry-on goal from last quarter Yes Done
Productionize Druid Pageview Pipeline and UI (pivot) on Druid task T138261 Easy access for WMF-internal users to Druid query UI. Users can get to pageview data without having to write SQL Security, Operations Carry on for next quarter (q2) To do To do

Goal-setting process owner:  Rob Lanphier


Objective Key result Dependencies ETA Status
Focused support for Wikimedia Security, assisting with hiring and implementing experiments such as assembling a security-focused working group (T123606) and creating a workable framework for security postmortems (T123753) Scaling our security expertise Emergent leadership and wide participation EOQ Yes Done

Much of the work was done by Darian Patrick, who stepped up after Chris Steipp's departure. I helped Wikimedia Security with bug triage meetings where I could.

Implement Fellowships program Clear guidance from our most senior engineers, while giving them the autonomy to be effective for the movement Editing EOQ N Postponed - I intend to regroup and discuss this with our future Wikimedia Foundation CTO

Goal-setting process owner: Ori Livneh

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The following is a summary of the Performance Team Goals.

Objective Key result Dependency ETA Status
Deploy PerformanceInspector EOQ In progress In progress
Deploy Thumbor
  • Service-based thumbnailing re-architecture in production - task T121388
Ops EOQ In progress In progress
Front-end performance
  • Optimize critical rendering path - task T127328 (Reusable style modules; consolidate script queues)
  • Support other teams and products.
In progress In progress
Multi-DC On-going work - task T88445
  • ..
In progress In progress
ResourceLoader Improve documentation, make code easier to debug, improve error-detection.

Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201617-q1 - More details at Wikimedia Release Engineering Team/Goals/201617Q1

Objective Key result Dependency Team members ETA Status KPI (if applicable)
Phase out Ubuntu Precise
  • N Not done - Replace primary production Continuous Integration host (gallium) - task T95757
  • Yes Done - Upgrade Phabricator database servers to Maria10/Jessie - task T138460
  • Yes Done - Upgrade Beta Cluster database servers to Maria10/Jessie - task T138778
  • Yes Done - Move Gerrit off of ytterbium - task T125018
  • Operations
  • Gallium
    • Lead: Antoine
    • Tyler
    • Zeljko
  • Phabricator
    • Lead: Mukunda
  • Beta Cluster
    • Lead: Dan
    • Mukunda
  • Gerrit
    • Lead: Chad
EOQ Incomplete Partially done Three fewer production Ubuntu Precise production machines (and two fewer Beta Cluster hosts).
Reduce Technical Debt
  • Perform a technical debt analysis of software and services maintained by WMF Release Engineering - task T138225
  • None
  • Lead: Greg
  • Everyone
EOQ Yes Done By the end of Q1 WMF Release Engineering will have a prioritized list of technical debt work that should be done. This will allow us to prioritize this work accordingly alongside our other user-requested work in the future.
Streamline deployments

(Train 2.0)

Stretch goal
  • Convert our production deployment strategy to use long-lived branches - task T89945
 
  • TechOps for any needed caching changes.
  • MediaWiki expertise for the needed changes to MW internals.
  • Lead: Mukunda
  • Chad
  • Tyler
EOQ N Not done By the end of Q1 deployments will be made from long-lived branches versus week-specific ones.


In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data[edit]

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective Key result Dependency ETA Status
FOCUS

Revscoring in production

  • ORES extension deployed on 6 wikis (T140002)
  • Write a comprehensive story on ORES (covering productization and research reports) (T140429)
  • Release historical article score data; explore score preparation for ingestion into ElasticSearch (T135684)
  • Ops/DBA
  • Comms
EOQ Yes Done
EXPERIMENT

Open Notebooks Infrastructure (T140430)

  • Define, monitor and ensure PAWS availability of not less than 0.5% less than Labs' NFS availability over a rolling 30d window
  • Release a set of notebooks showcasing analysis that can be performed on PAWS by 2nd month of Q1
  • Publish an announcement and call-to-action targeted at research community in 3rd month of Q1
  • Ops (Labs)
EOQ In progress In progress
FOCUS

Discussion modeling

  • Design and evaluate attack and aggressiveness models on article talk comments (T139703)
  • Release notebooks; write up and present results (T139704)
  • None
EOQ In progress In progress
FOCUS

Productize the article recommendation API

  • Stable version of production-ready article recommendation API (T140431)
  • None
EOQ In progress In progress
STRENGTHEN

Research landing page (stretch)

  • Complete information architecture, audience analysis and draft of preliminary contents for a landing page for Wikimedia Research
  • Design Research
EOQ In progress In progress

Design Research[edit]

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective Key result Dependency ETA Status
FOCUS

Product Research

  • Deliver design requirements, heuristic evaluations, and user study findings for key products across Reading, Editing, and Discovery.
  • Product teams: Mobile Apps, Desktop & Web, VE, Collaboration, Discovery
end of Q1 Yes Done
FOCUS

Generative Research

  • Analyze and share results of Mexico, Nigeria, and India contextual inquiries T132799
  • New Readers project concept development and evaluation and collaboration on product iteration (in collaboration with Reading, Global Reach, Communications) based on contextual inquiries in South Africa, Mexico, Nigeria and India. Explore and evaluate 1-3 concepts coming out of the contextual inquiries with the goal of deciding on 1 concept to focus on by the end of the quarter. This work will involve researching what has been done in both wiki projects and in the market in general around the product area we decide to focus on as well as the possibility of new concepts and/or products. T129201 T132800 T132801
  • Reading, Global Reach, Communications (teams) and ReBoot (external contractor)
mid Q1 Yes Done
STRENGTHEN

Research capacity building

  • Implement user testing tools to support moderated and unmoderated user testing on desktop and mobile (apps and web)
  • Experiment with recruiting research participants through Wikimedia/Wikipedia social media channels
  • funding for testing tool (annual subscription or a la carte)
  • Communications team
end of Q1 Yes Done
EXPERIMENT

Research data mapping (stretch)

  • Document the sources of user data we manage, how we store it, and what potentially personally identifiable information it contains.
  • Ensure all archived user study data is retained and shared in compliance with WMF data retention policy and privacy policy.
  • Draft data access guidelines to inform our future data collection and dissemination practices.
  • Security team, Legal team, Research & Data team
end of Q1 Yes Done

Goal setting process owner: Darian Patrick

More details at Wikimedia Security Team/Goals 2016-2017#Q1_.28July-Sept_2016.29

Objective Key result Dependency ETA Status
Improve two-factor authentication usability Removal of impediments to adoption and use Results of usability survey, conducted with input from Design Research and Legal EOQ N Postponed
Draft and release job descriptions for new Security Team staff and begin hiring process Increase team capacity Input from Tech. Mgt. and other key stakeholders EOQ Yes Done
Draft Security Team onboarding documents/handbook New hire and on-demand resources for day-to-day operations EOQ Incomplete Partially done

Goal setting process owner: Gabriel Wicke

Objective Key result Dependency ETA Status
Protect sensitive user information
  • Minimize code with direct access to sensitive user information by creating an authentication service prototype, to be used as an AuthManager backend in Q2.
Security, Operations EOQ Partly successful. Prototyped, full implementation pending storage backend discussion & security hiring.
EventBus & change propagation
  • Finish RESTBase update job migration.
  • Support use cases for Discovery, Reading & Wikidata, replacing custom jobs where it makes sense.
  • Refine our CDN purging infrastructure to reduce purge request volumes & improve reliability (with operations).
Operations EOQ Driver / Kafka upgrade took longer than anticipated. Finished RESTBase update migration & enabled other use cases like ORES. Work with collaboration & reading teams on new use cases is ongoing & will continue in Q2. Discussed CDN purge options & contributed patches, but do not yet have comprehensive scaling strategy in place.

Other progress:

Objective Key Result Dependency ETA Status
Move remaining Wikimedia Labs operations functionality from Wikitech to Horizon
  • Move Puppet class and role management to Horizon from Wikitech
  • Move Hiera management to Horizon from Wikitech
  • Allow all registered users to browse project / instance info N Not done
EOQ Yes Done
Modernize Puppet configuration management infrastructure
  • Upgrade current puppetmasters to jessie
  • Upgrade to puppet 3.7
  • Scale Puppetmasters to multiple machines in eqiad and codfw
  • Install and migrate to PuppetDB, preparing for Puppet 4
EOQ Yes Done
Build a Varnish xkey vmod and purging schema that works for our needs
  • Design a basic plan for how XKey k=v work and how they're encoded in PURGE traffic
  • Ensure xkey code can realistically handle this for our scenario (including keys with many many objects, index size scaling, etc)
  • Build the packaged vmod for varnish4 so that it's ready for experimentation and higher-level application key-design work
EOQ Yes Done
Progress Varnish 4 migration
  • Upgrade cache_upload cluster to Varnish 4
EOQ Yes Done
Experimental Prometheus metrics monitoring
  • Deploy redundant prometheus servers in codfw and eqiad
  • Fetch application metrics in each datacenter from >= 50% of MySQL servers and essential host metrics from >= 10% of hosts
  • Poll metrics with >= 1/min frequency, and with a retention of >= 4 months
  • Implement Prometheus multi-dimensional metrics (grouping/tagging) for hosts and databases
EOQ Yes Done
Stretch: Experiment with Kubernetes for services in production
  • Install Kubernetes in production
  • Evaluate and experiment with alternative networking solutions
EOQ Yes Done