Wikimedia Engineering/2016-17 Q3 Goals

From mediawiki.org
Q2 Wikimedia Engineering Goals, FY2016-17, Q3 (January - March) Q4

Purpose of this document: Goals for the Wikimedia Technology and Product Development department, for the second quarter of fiscal year 2016–17 (January 2017 – March 2017). The goal setting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.

Status fields can use the following templates: In progress In progress, To do To do, N Postponed, Yes Done or Incomplete Partially done

ETA fields may use the initialism EOQ (End of Quarter).

Product[edit]

The Product department, alongside the community, builds, improves and maintains the user facing features on all the Wikimedia projects. The software, interfaces, apps, and other systems and applications the team supports are essential means by which contributors and readers connect with our sites. The work of the team’s three “audience” groupings – Discovery, Editing, and Reading – is interrelated and enables a channel for progressive engagement with the users of the Wikimedia projects.

Our mission is to make the wealth of knowledge and content in the Wikimedia projects easily discoverable.

Goal setting process owner: Dan Garry

Team Objective Key result Dependency ETA Status
Search Add real-time index updates to completion suggester and improve overall stability and performance of search by upgrading to ElasticSearch 5
  • Upgrade search cluster to ElasticSearch 5
  • Add real-time index updates to completion suggester
  • Community Engagement, Team Practices Group, Analytics
EOQ Incomplete Partially done - upgrade completed, real time updates in progress
Improve support for different languages by researching and deploying new language analyzers
  • Perform research spikes to find new analyzers for different languages
  • Test new analyzers to see if they are improvements
  • Deploy new analyzers that are found to be an improvement
  • Community Engagement, Team Practices Group, Analytics
EOQ In progress In progress
Prototype and test overhaul UI of search page to add cross-wiki searching
  • Have a working prototype in Labs
  • Run at least 2 A/B tests on Persian, Italian, Catalan and Polish Wikipedias
  • Feedback from community Wikipedias: Persian, Polish, Italian and Catalan
EOQ Yes Done
Interactive Stabilize maps and graphs code base
  • Stabilize and fix high priority bugs
  • Note: this goal was added to the goals list mid-quarter
  • Community Engagement
TBD In progress In progress

Wikidata Query Service does not have any dedicated goals this quarter. The service will continue to be maintained by Stas and Guillaume, and Stas will work on user feature requests as time permits. Additionally, the dedicated hardware for the service will be upgraded to support the continued growth and use of the service.

The Analysis team (Mikhail and Chelsy) is in the final stage of adopting Analytics' ReportUpdater infrastructure. As part of that migration, they are starting to forecast usage of Discovery's services and products to detect when services and products might be significantly under- or over-utilized (relative to the usage our models predicted).

Wikipedia.org Portal team will not have any dedicated goals this quarter other than continuing to maintain the page by fixing critical bugs and performing regular statistics and translations updates. No other work on the page is planned in Q3.

Goal setting process owner: User:Trevor Parscal (WMF)

The Editing Department build collaborative, inclusive tools for creating and editing free knowledge.

Also see the more detailed progress on these goals which are based on Program 4 of the Annual Plan

Annual Goal Team Objective Output (deliverable) Outcome Dependencies ETA Status
1: Maintain and incrementally improve current content creation and curation interfaces VisualEditor Visual diffs for the visual editor Provide visual diffs for all users inside the visual editor. Editors can see the changes they and others have made without needing to know the intricacies of wikitext.
  • Editing Design
  • Community Liaisons
EOQ Yes Done, with more features to come
2: Improve editing experience by reducing technical and product debt Parsing Address technical debt in wiki content Parsoid has a linter that can identify errors in wikitext. We are going to deploy a Linter extension to the Wikimedia cluster that collects these errors and surfaces them to editors. Editors will be able to identify markup errors and use their existing processes to fix them. But, the Linter extension itself has client-side JS to highlight the section of wikitext with the error to make it easier for editors to fix.
  • Editing Design
EOQ Incomplete Partially done, deployed on small wikis
Multimedia 3D model file support Pick up the work on 3D file support by getting the 3d2png service and 3D extension into production, and working with Reading on MediaViewer integration. Editors can upload and transclude 3D files on articles, which readers can view statically on-page or interactively in MediaViewer/etc..
  • Security
  • Performance
  • Services
  • Reading
  • Community Liaisons
EOQ Incomplete Partially done, work carried over
3: Invest in new types of content creation, and new forms of curation and collaboration tools Collaboration Provide an edit-review process suited to the needs of new contributors—while improving efficiency for edit reviewers in general. Add ORES tests, optimized ORES data display, a new filtering UI and a “New User” filter to the recent changes page (as a beta). Release the ReviewStream machine-readable feed incorporating ORES and other useful edit-review data. More powerful edit filtering generally and, in particular, the ability to identify damaging but good-faith edits by new users. Faster performance for downstream review tools and the ability to more easily incorporate newcomer-friendly features.
  • Analytics
  • Services
  • Research & Data
  • Community Liaisons
EOQ Incomplete Partially done, deployed on some wikis
Language Stabilize ContentTranslation for going out of Beta Replace the contenteditable editing component of Content Translation to VisualEditor and preserve the current CX functionalities. Content Translation is more stable in terms of article publishing, requiring less maintenance work from patrollers, and more ready to be enabled by default for all users.
  • VisualEditor
  • Parsing
  • Editing Design (for OOjs UI)
EOQ Incomplete Partially done, work carried over

Goal setting process owner: User:Khorn_(WMF)

Objective Key result Dependency ETA Status
Paypal Express Checkout integration possibly campaign ready Paypal EOQ Incomplete Partially done, work carried over
Start Ingenico reintegration Begin dev work ingenico EOQ Yes Done
Central Notice Banner Sequence feature ship some version of this To be added EOQ Incomplete Partially done, work carried over
Team Objective Key result Dependency ETA Status
Reading Infrastructure Empower editors to create mobile-friendly templates by allowing <style> tags to be attached to templates. The TemplateStyles extension is deployed in production and there are no unaddressed major community concerns. Security, Editing, Community Liaisons, Performance, Traffic EOQ To do To do
Reading Infrastructure Determine next steps for the ORES platform technology and its support. A report (or reports) assessing the software architecture and software quality of the ORES platform (service, extension, and so forth), with recommendations covering areas such as code, configuration management, hosting, data persistence, security & privacy, testability, and a plan for viable human and automated support of the platform and its future intended use. Research & Data, probably Editing EOQ To do To do
Reading Web Provide optimal printed content Update print styles for mobile and desktop web and deploy on all projects in preparation for using browser-based pdf rendering New Readers (for mobile only), community liasons EOQ To do To do
Assess the possibility of providing an offline-capable app and unifying content through various services

Create a written report planning the technical and tactical implementation of building and releasing an offline-capable app that unifies content consumption under reading services

Services EOQ To do To do
iOS App Improve the relevancy and engagement with top read and notifications features Integrate and evaluate trending edits API for notifications and Explore feed Services EOQ To do To do
Android app Improve user confidence and set a solid stage for future feature development. Make significant improvements to offline functionality (reading lists) and offline navigation. None EOQ To do To do
New Readers Hand off online to offline prototypes to the web team for moving to production. A report which summarizes research (remote and on the ground) and community feedback.

Requirements defined and documented in partnership with Web team.

Design research (inc. vendor), Reading web EOQ To do To do

Goal setting process owner: Danny Horn

Team focused on meeting the needs of active Wikimedia contributors for improved, expert-focused curation and moderation tools.

Objective Key result Dependency ETA Status
Partner with community on identifying and prioritizing needed improvements
  • Report back to community on analysis of top 10 wishes from Community Wishlist Survey
  • Host on-wiki discussions on refining the definitions of reported problems, validating proposed solutions
EOQ Yes Done
Improve core contributors' productivity Build a prototype of the cross-wiki watchlist feature
  • Performance
  • Operations
EOQ N Not done
Improve core contributors' productivity Complete development and ship features and fixes related to two wishes in the Wishlist Survey top 10 EOQ To do To do
Improve workflows for Tool Labs developers Extend Striker to allow developers to create a shared tools account and collect data to be used for right to fork policy and tool discovery. EOQ N Postponed
Improve workflows for Tool Labs developers Define evaluation criteria for evaluating Platform as a Service (PaaS) solutions (carried over from Q2)
  • Labs team
EOQ N Postponed
Encourage community maintenance of important tools Help community bootstrap committee to enforce right to fork and abandoned tool policies.
  • Tool Labs Community
EOQ Yes Done
Improve community culture and reduce incidence of harassment Hire and onboard new staff for the anti-harassment project EOQ To do To do

The Team Practices Group believes that healthy teams reliably deliver customer value and innovative products. Through dedicated resourcing, coaching, and workshops at the Wikimedia Foundation, we facilitate the use of practices that support team health, with a strong emphasis on creating conditions of clarity, trust, and continuous improvement.

Goal setting process owner: Kristen Lans

Workflow Working with Deliverables/Impact

Wikidata[edit]

Goal Setting and process owner is Lydia Pintscher.

Wikidata is Wikimedia's structured data project and knowledge base. Our goal is to support all the other Wikimedia projects with structured data and provide a top-notch knowledge base to everyone who needs structured data about the world.

Wikidata adapt their programms for WMDE’s new annual plan in 2017. These goals are:

    • Goal 1: Continue to assure the technical and social sustainability of Wikidata (software and community) as the central, structured knowledge data base for the Wikimedia movement and beyond. (Sustainability of Wikidata)
    • Goal 2: Continue to increase the reach of Wikidata into the Wikimedia projects. (Wikidata for Commons, Wiktionary, Wikipedia)

Here you can find all objectives for next quarter.

Objective Key result Dependency ETA Status
Sustainability of Wikidata

Improve the usability of Wikidata.

Research migration to OOUI framework to be more in line with the rest of the Wikimedia projects and at the same time improve our input widgets. Improve the keyboard navigability of the user interface. Create new datatype for linking to geoshape and data pages on Wikimedia Commons.

EOQ In progress
Sustainability of Wikidata

Improve the documentation around Wikidata.

Organize an event to support the community in order to improve documentation and help pages for regular users.

  • Community
EOQ In progress
Sustainability of Wikidata

Keep the community involved, happy and connected.

Start organizing WikidataCon, an event build for and by the Wikidata community for Wikidata’s 5th birthday.

  • Community
EOQ In progress
Wikidata for Commons

Provide structured data support for multimedia files to Wikimedia Commons.

Publish Federation prototype in order to let the community try it out and work more towards Multi-Content-Revisions together with WMF.

  • WMF engineering and database expertise.
EOQ In progress
Wikidata for Wiktionary

Support for storing lexicographical data in order to support Wiktionary.

Publish the Lexeme prototype in order to let the community try it out. Roll out the support for automated sitelinks to Wiktionary. Publish the technical data model specification.

  • Community and WMF security team
  • Input & Implementation supported by Discovery team
EOQ In progress
Wikidata for Wikipedia

Allow editing of Wikidata’s data directly from Wikipedia.

Publish next version of click-dummy to get a new round of feedback from the community.

EOQ In progress
Wikidata for Wikipedia

Make ArticlePlaceholder pages discoverable by search engines in order to help small Wikipedias grow their reach.

Trial run on at least one Wikipedia to test the impact.

  • Operations Team
EOQ In progress
Wikidata for Wikipedia

Make using Wikidata data on the other Wikimedia wikis easier.

Allow automatically adding authority control templates to articles via an empty system message at the end of content pages that editors can overwrite. Improve the statement parser function further (if there is time).

EOQ In progress
Wikidata for Wikipedia

Automated creation/update of list articles based on queries to Wikidata.

Experimenting with different ways to make it easier to write and edit queries (if there is time).

EOQ In progress

Technology[edit]

Goal setting process owner: Nuria Ruiz

Objective Key result Tech Goal Dependency ETA Status
Improve tools for data display for the Foundation and community. Better tools to access data. Wikistats 2.0 Creation of Visual language for replacement of http://stats.wikimedia.org task T152033 . Mocks and Wireframes for UI and Information Architecture of new site when it comes to edit data. A. Org Level Priority. B. Serving our Audiences. Design - Community Liaisons, in order to engage with community and be able to provide early feedback. Yes Done Visuals can be seen here: task T157827
Improve tools for data display for the Foundation and community. Better tools to access data. Wikistats 2.0 Productionize mediawiki edit reconstruction pipeline to enable calculation of edit metrics for community and WMF for all wikis since the start of the projects task T152035 A. Org Level Priority. B. Serving our Audiences. Yes Done
Improve data computing and release. Real Time. PublicEventStreams Launch task T13065 Start of deprecation of RCStream with the result of less code and less resources used to publish an edit updates feed for the community B. Serving our Audiences. Ops Yes Done
Improve data computing and release. A/B testing framework design Design Document for a more solid AB testing infrastructure. Carry along from last quarter task T135762 B. Serving our Audiences. Ops, Research Yes Done Design document available in meta
Operational Excellence Set of tasks needed to keep lights up and update/upgrade existing systems to maintain and improve current level of service. In order of priority.
  • Install new Pageview API nodes (carry-on as we have not provisioned hardware last quarter): task T144833
  • Expand Cluster with 8 new Hadoop Nodes task T152713
  • Kafka Doc Revamp: task T150277
  • Deprecate old reportcard as part of removing old dashboard infrastructure from labs task T146308
  • CDH upgrade. Value proposition: new spark for edit reconstruction task T152714
  • Alarm on heap utilization task T88640
C and D. Tech Debt and improve our own feature set To do To do

Speed is Wikipedia's killer feature. ("Wiki" means "quick" in Hawaiian.) As the Wikimedia Foundation’s Performance team, we want to create value for readers and editors by making it possible to retrieve and render content at the speed of thought, from anywhere in the world, on the broadest range of devices and connection profiles.

The following is a summary of the Performance Team Goals.

Objective Key result Quadrant
Outreach
  • Assist Tech Ops with second datacenter switchover.
  • Write blog post about "hidden tabs incident" - T153048
  • Continue preparation work for running Active-Active datacenters. - T88445
B: Customer goal
Monitoring
  • Set up monitoring of performance metrics using Grafana 4 alerts and notifications - T153166
C: Feature
Improvement D: Tech debt
Knowledge
  • Research isolated browser testing environment and proxy T153360
C: Feature


Goal setting process owner: Greg Grossmeier

All tracked in: #releng-201617-q3 - More details at Wikimedia Release Engineering Team/Goals/201617Q3

See also Wikimedia Release Engineering Team/Project/Planning for more items that are not quarterly goals.

Objective Key result Dependency Team members ETA Status Tech Goal Quad.
Train 2.0
  • Move MW+Extension deploys to scap3 - task T114313 (part 2)
    • Build fanout support in scap3 - task T121276
    • Assess impact of moving to `scap deploy` for all parts
  • Operations
  • Chad (Lead)
  • Tyler
  • Mukunda
  • Dan
EOQ Yes Done C: Tech feature

A: Annual goal

Continuous Integration Infrastructure modernization (i.e., Deployment Pipeline)
  • WMF Labs
  • Operations
  • Tyler (Lead)
  • Antoine
  • Dan
EOQ N Not done B: Customer goal

C: Tech feature

Phabricator Improvements Re-evaluate and improve search functionality in Phabricator - task T146843
  • (Minimal) Discovery
  • Mukunda (Lead)
  • Chad
EOQ Yes Done D: Tech Debt


In Wikimedia Research we use qualitative and quantitative methods to provide strategic insights and technological solutions to the movement and the Foundation, to foster innovation and to inform the development of new products.

Research & Data[edit]

Goal setting process owner: Dario Taraborelli [ Wikimedia Research goals overview ]

Objective Key result Dependency ETA Status
FOCUS

Revscoring

  • A report (or reports) assessing the software architecture and software quality of the ORES platform (service, extension, and so forth).
  • Complete a paper draft on ORES socio-technical AI
  • Reading Infrastructure
EOQ In progress In progress
FOCUS

Stub recommendations

  • Design a model predicting articles that need to be expanded in English Wikipedia and expand to other languages
  • Prototype recommender design
  • Formal collaborators (EPFL)
EOQ In progress In progress
EXPERIMENT

Recommender system evaluation

  • Develop and pilot a methodology for comparative evaluation of search/recommendation algorithms
  • Reading
  • Design Research
EOQ In progress In progress
STRENGTHEN

Annual workshops and outreach

  • Host a session on "Artificial Intelligence to build and navigate content"" at the Wikimedia Dev Summit 2017
  • Organize the annual Wiki Research Workshop at WWW '17
  • Organize the WikiCite '17 event
  • Co-organizers
  • Conference hosts
EOQ In progress In progress

Design Research[edit]

Goal setting process owner: Abbey Ripstra [ Wikimedia Research goals overview ]

Objective Key result Dependency ETA Status
FOCUS

Generative research

New Readers next steps
  • Evaluative research in country for offline concepts
  • Concept generation and evaluation of affordability findings
  • Consultations with Comms on awareness findings

Audience project

  • Collaborate on next steps
  • New Readers teams
EOQ Yes Done
STRENGTHEN

Participant outreach

  • Add 1,000 new participants to the Design Research database
  • Leverage WMF social media accounts
  • Start a physical campaign
  • Communications
  • Design researchers (Product)
EOQ In progress In progress
FOCUS

Understanding editing culture

  • Document content curation workflows on Wikimedia Commons to inform development micro-contribution features T152248
  • Perform and publish analysis of the quality of new editor support provided at Wikipedia Teahouse T132809
  • Perform and publish expanded analysis of the impact of Teahouse on new editor retention T127523
  • Research and Data
EOQ In progress In progress
EXPERIMENT

Recommender Evaluation

  • Develop and pilot a methodology for comparative evaluation of search/recommendation algorithms
  • RelatedPages study plan T142009
  • Dev Summit proposal T149373
  • Research and Data
  • Reading
EOQ In progress In progress

Goal setting process owner: Darian Patrick

Objective Key result Dependency Team members ETA Status
Improve wiki account authentication
  • Provide insight into active sessions
  • Improve security messages to users (login, reset, etc.)
  • Reduce possibility for account takeover
  • TBD
  • Darian
  • Brian
  • Sam
EOQ To do To do


Goal setting process owner: Gabriel Wicke

Objective Key result Dependency ETA Status
REST API 1.0 (B/D) Graduate the REST API 1.0 from beta to stable production. Stabilize individual end points, improve documentation, and write a blog post announcement. EOQ Done
Container infrastructure (B/C) Establish requirements for a container-based development, continuous integration, production, and third party install infrastructure. Identify issues, and investigate potential solutions. Operations, Release Engineering EOQ Done

We also plan to spend significant time on:

  • Change propagation and EventBus infrastructure expansion.
  • RESTBase storage scaling.
Team Objective Key result Dependency ETA Status
Core Ops

Data center switchover (prep)

  • Audit & prepare MediaWiki application servers, Swift, ElasticSearch, RESTBase, EventBus and Parsoid for switchover to codfw
  • Partially orchestrate the datacenter switchover, reducing the number of steps of individual merges and manual commands required
  • Improve and implement MediaWiki cache warmup procedures from current ad-hoc method
  • Performance & Availability
  • Release Engineering (Scap, MediaWiki configuration)
  • Services (RESTbase)
  • Discovery (Search)
  • Parsing (Parsoid)
  • Analytics (EventBus)
  • Community Liaisons
EOQ Yes Done

Security: Enhance and unify system user account handling

  • Enhance account handling to cross-validate data sources data.yaml and LDAP
  • Simplify account/group membership handling
  • Review & audit existing accounts and groups
EOQ Yes Done

Stretch: Kubernetes

  • Experiment with at least one service running on Kubernetes in production
EOQ Yes Done

Migrate all production systems away from Precise

  • Upgrade/migrate all remaining Precise production hosts to Trusty or later
April 24th 2017 (Precise End Of Support) Yes Done
Labs

Deprecate Precise

  • Precise based services in Tool Labs are discontinued
  • Precise instances are purged from all projects
EOQ Yes Done
Investigate Neutron migration path
  • Experiment: Initial OpenStack Neutron PoC deployment in Labtest
EOQ In progress In progress

Traffic

Site selection for Asia Cache DC

  • Consult formally with Legal on location (country)
  • Evaluate vendor/building options in top locations
  • Vendor review of top choices with Procurement
  • Final selection of City, Vendor, Building for Asia Cache DC
  • Legal
  • Finance (Procurement)
  • DC Ops
EOQ Yes Done