Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation.

See also:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF  goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software

Previous months:
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March

Item syntax:
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]

Examples:
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Operations

 * Data Centers
 * Deploy 2nd Transit (internet) network carrier @ EQIAD [Leslie][DONE]
 * ipv4 transit is now live. working on ipv6 next.[DONE]
 * Lucene Search deployment@EQIAD (parallel test)[Peter/Asher/Jeff][DONE]
 * Lucene Search @ EQIAD rollout [Peter/Asher/Jeff][DONE]
 * New Nagios server@EQIAD [Leslie][done but now upgrading to Icinga][DONE]
 * DB Schema changes - SHA1 requirements for post r1.19 [Asher] [DONE]
 * upload.wikimedia.org@eqiad using varnish (replacing squid)[Mark]
 * Mark has been testing his Varnish hashing redirector (added hashing persistence). Currently upload@eqiad is serving about 100% traffic. Also added 8 more varnish servers (to 16).[DONE]
 * Mark fixed 3 memory leaks but still seeing leakage[DONE]
 * Caveat: Varnish@upload deployed. Varnish has some inherent data persistency issue in additonal to some unidentified leaks. Will need more work.
 * db9/db10 upgrade (db for etherpad, RT, Bugzilla, ..)
 * Servers Bellin & Blondel are being setup for replication & eventual rollout later this month
 * new UDP2log infrastructure @ EQIAD
 * prepare server Oxygen for Diederik [Mark] [DONE]
 * enable udp2log service [Peter]


 * Swift
 * Prep Swift infrastructure to be ready for 'Originals' (Commons) deployment - Testing [Ben/Aaron]
 * Start seeding SWIFT cluster with Originals


 * Wikimedia Labs
 * New compute node for capacity [Ryan] [DONE]
 * Labs features rollout
 * Build out LVS in Labs [Ryan/Mark]
 * Create squid instance in Labs [Ryan/Sara]
 * Add resilience to Lab Controller service by building redundant Controller services in EQIAD which includes DNS, LDAP, MySQL [Sara]


 * Data Dumps
 * EQIAD test/parallel runs [DONE]
 * Test suites for XML Dumps [on-going WIP]
 * own MQ server

Editing tools

 * Visual editor
 * demo and sandbox to use for CE [Trevor] [DONE]
 * complete backlog on data model [Roan] [IN PROCESS]
 * contentEditable Linear Model integration [Trevor/Rob] (text nodes, serializer, etc.) [IN PROCESS]
 * UI for contentEditable [Rob] (Surface View, toolbar, menu, etc.) [IN PROCESS]


 * Parsoid
 * Try to nail down as much of the bugs on the todo in order to get a testing model (template/parser function/table/attribute)[Gabriel] [IN PROCESS]
 * Get VE round-tripping info, source, tracking, RFDa (annotations) [Gabriel] [IN PROCESS]

Participation and editor engagement
ation [DONE]
 * Article feedback
 * Abuse filter integration:
 * Testing of one filter on English Wikipedia [DONE]
 * Deployment of multiple filters on English Wikipedia [April 30]
 * Feature this Post, Mark as resolved [April 26]
 * Additional features for oversighters/monitors [April 26]
 * Edit this page - Option 4 (CTA instead of feedback form)
 * documentation.
 * Central Feedback Log: Evaluate options [IN PROCESS]
 * Begin design work on streamlined version of Option 1 [IN PROCESS]
 * Finalize Checkuser requirements for Feedback posts [IN PROCESS]


 * Feedback Dashboard
 * Patch releases (permalinks, e-mail change, AFTv4 change, bugs in abuse filter) [DONE]
 * Minor changes to support Dario/Giovanni's research [after April 16 PT deployment]
 * First internal research review [DONE]


 * Page Triage
 * Prototype deployment on English Wikipedia (in parallel with Special:NewPages) [w/o April 23]
 * Work on NO_INDEX (pending RFC)
 * Curation toolbar (temporary name)
 * Complete first design iteration [DONE]


 * Echo/Messages
 * Examine hooks into current MW infrastructure for notifications
 * Look into PHP default synchronous notifications
 * Examine LQT3 code [IN PROCESS]


 * Misc EE
 * CR on AbuseFilter and Moodbar (Andrew) [IN PROCESS]

Multimedia tools

 * TimedMediaHandler
 * Verify labs is ready for testing (issues with beta on labs) [IN PROCESS]
 * Begin testing on labs setup


 * [UploadWizard]
 * No current work planned

Feature support

 * Wikipedia Education Program
 * Complete testing in Wikimedia Labs and incorporate community feedback - http://education.wmflabs.org/ [Jeroen, QA via Chris] [IN PROCESS]


 * Fundraising
 * Get recurring GlobalCollect up [IN PROCESS]
 * Git migration deployment [IN PROCESS]

MediaWiki infrastructure

 * Gadgets 3.0
 * Includes review of Salvatore's "Gadgets user preferences" work


 * ResourceLoader
 * Gadgets 2.0 deployed after MW 1.19 (dependent on REL1_19 schedule) [in progress]
 * Set up a first working setup in WMF labs [Roan, Timo]

Analytics Team

 * Deployment Wikipedia Zero and Teahouse filters
 * Deployment of Beta 2 Reportcard
 * Deployment of geocoding of editors
 * Configuration of analytics servers: Oxygen and stat1
 * Continued work on Reportcard

Internationalization/Localization

 * Internationalization and localization tools
 * Universal language selection UI (for desktop and mobile) [i18n team + UI/UX contractors (Pau, Arun)]; [in progress]
 * Translate extension performance improvements and notifications enhancements [Niklas, Santhosh]
 * Narayam feedback functionality - adding more languages [Santhosh, Amir]
 * Language impact assessment/metrics measurement [i18n team]
 * WebFonts maintenance [Santhosh]
 * Office Hours: Scheduled for next week on #wikimedia-office on Wednesday April 18

Editor Engagement Experimentation (E3)
(http://hire.jobvite.com/Jobvite/Job.aspx?j=oi7iWfwn&c=qSa9VfwQ)
 * This new team focused on engineering for experimentation around strategies to reverse stagnating/declining participation in Wikimedia’s projects will be effective as of April 16. This team will be a cross-functional team with resources from the Community and Engineering/Product departments tasked specifically with conducting small, rapid experiments designed to improve editor retention. We already know some of the fixes that will solve the editor retention problem, and we're working on those already. The purpose of *this* team will be to identify the fixes we don't yet know about. This engineering team will report to Alolita Sharma with two new software developer positions to be hired in the current fiscal year. See job listings for the following positions:
 * Senior Software Developer (http://hire.jobvite.com/Jobvite/Job.aspx?j=oHDiWfwi&c=qSa9VfwQ)
 * Software Developer - FrontEnd (http://hire.jobvite.com/Jobvite/Job.aspx?j=opDhWfwZ&c=qSa9VfwQ)
 * Software Developer - BackEnd

Product
[DONE]'
 * Mobile Navigation product spec [Phil] [DONE]
 * Advanced Upload Wizard WLM Spec (exploring whether WLM features are desireable/feasible for initial release of mobile photo uploads)

Design

 * Upload Basic Workflow [Lindsey]
 * Mobile Navigation [Lindsey]

Mobile Web

 * [Jon, Yuvi, Arthur, Max]
 * New look & feel
 * Full screen search (without preview) [Jon]
 * Prod
 * Footer [Jon]
 * Prod
 * Reference reveal
 * Prod
 * New collapsible sections [Jon]
 * Prod
 * Contact us [Jon+Arthur]
 * Beta
 * Remove dependence on the output buffer in prep for MF -> Core move [DONE]
 * Begin refactor of MF in prep for core migration
 * Moving MF rendering to a skin [DONE]
 * Performance testing
 * Lazy loading of article sections

Wikimedia Apps

 * Begin testing GPS API [Yuvi & Max]
 * Wikipedia Android app
 * Release V1.1.1 [DONE]
 * Release V1.2 beta ("the decoupling")
 * Wikipedia iOS app
 * Release V3.1.1 [DONE]
 * Release V3.1.2 Bugfix [DONE]
 * RC1 [DONE]
 * RC2 [DONE]
 * Release V3.2 beta ("the decoupling")


 * Community discussion about standardizing templates [Amgine]

Wikipedia Zero

 * Digi Malaysia(Telenor) rollout [Preilly+Ops]
 * Futher partner rollouts in Africa & Asia

Wikipedia over SMS/USSD

 * WMF Production Rollout for Vumi over Gtalk [Patrick]

Special projects

 * Kiwix on Android
 * Improving performance of openZim PhoneGap plugin
 * ePub Collections extension [Ops+PP]

MediaWiki Core

 * MediaWiki 1.19/Roadmap
 * MediaWiki 1.19 tarball release [Sam]
 * Release Candidate in April [DONE]


 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle begins
 * Complete 1.20wmf1 deployment [in progress]
 * Start 1.20wmf2 deployment [DONE]


 * Continuous integration
 * Improve beta labs so that it's suitable/trusted for testing (target completion in May) [Antoine, Chris]
 * Begin developing framework for integration testing [Antoine, Chris McMahon]
 * possibly Selenium - see Chris's mail to wikitech-l
 * TestSwarm reintegration with Jenkins [Antoine] [in progress]


 * Git/Conversion [Chad/Antoine/Sumana]
 * First deployment from git mainline development branch [DONE]
 * See MediaWiki 1.20/Roadmap section above.
 * Updating deployment documentation [Chad/Roan] [mostly done]
 * Update all opt-in extensions from svn to git [DONE]
 * Bugfixing (see bug dependencies)


 * SwiftMedia
 * Make File.php write files to Swift in production on transformation (thumbnails) [Aaron] [move to May]
 * Start deployment work on originals [move to May]
 * Bugfixing and cleanup on thumbnails [ongoing]


 * Lua scripting
 * Initial embedded Lua interpreter implementation [DONE]
 * Lua-via-shell implementation for third party deployments [DONE]


 * IPv6 support
 * Prepare for IPv6 Launch (June 6th) [Aaron,Mark?] [move to May]
 * rc_ip schema change [need to investigate if this is needed for June 6]
 * other schema changes


 * MathJax [Brion/TBD]
 * experimental MathJax rendering mode for Math extension almost ready to deploy [Brion]
 * needs IE 6-8 fix
 * Deployment with 1.20wmf1; enabling on MediaWiki.org for experimental testing [Brion/Aaron] [DONE]
 * Site-wide enabling as opt-in preference [Brion/Aaron] [move to May]


 * OAuth
 * Begin specification work [Chris Steipp] [DONE]

Analytics

 * Report card
 * Prototype first release -- Focus on new UI layer [partial]
 * First April release http://lists.wikimedia.org/pipermail/analytics/2012-March/000003.html [DONE]


 * Log collection pipeline
 * 3 new filters: 2 Wikipedia Zero and one Teahouse filter [DONE]

Engineering Community Team

 * Wikimedia blog maintenance
 * Theme changes, git environment setup, setup, testing, plugins


 * 'Summer of Code 2012
 * Proposal evaluation and acceptance/rejection, starting community bonding period


 * Bug Squad formation and training

QA

 * Test environment

May 2012

 * General engineering process checkin (IRC)

Tech Ops

 * Convert Upload cache from using squid to varnish (phase 2)
 * upload.wikimedia.org @ Tampa
 * Swift
 * Swift infrastructure rollout for 'Originals' (Commons) deployment - [Ben/Aaron]
 * Deploy servers for Analytics Cluster
 * Deploy Memcached or equivalent technology (e.g. membase) @EQIAD [Mark/Asher]
 * ESAM router replacement (from Foundary to Juniper) [Mark/Leslie]
 * Prep infrastructure stack to be ipv6 ready
 * LVS
 * squid
 * Fundraising infrastructure buildup@ EQIAD
 * own firewall
 * own payment clusters
 * migrate applications
 * OTRS Upgrade

---

Editing tools

 * Visual editor
 * Complete backlog on data model [Roan] [IN PROCESS]
 * Finish contentEditable Linear Model integration [Trevor/Rob] (text nodes, serializer, etc.) [IN PROCESS]
 * Finish UI integration for contentEditable [Rob] (Surface View, toolbar, menu, etc.) [IN PROCESS]
 * Complete data model
 * Complete API for inspector and toolbar
 * [End of May] Sandbox deployment
 * [June] Release candidate with round tripping.
 * [July] API for pluggable nodes?
 * [July] mediawiki.org deployment of VE


 * Parsoid
 * Try to nail down as much of the bugs on the todo in order to get a testing model (template/parser function/table/attribute)[Gabriel] [IN PROCESS]
 * Get VE round-tripping info, source, tracking, RFDa (annotations) [Gabriel] [IN PROCESS]
 * Start Parsoid-VE integration

Participation and editor engagement

 * Article feedback
 * [June] Complete development of features for AFTv5
 * [June] Potential deployment of AFT to broader list of articles on enwp
 * [June] Refactoring of code
 * [July] Final features for enwp release: browser support, i18n, mobile …


 * Feedback Dashboard
 * Patch releases (permalinks, e-mail change, AFTv4 change, bugs in abuse filter) [DONE]
 * Minor changes to support Dario/Giovanni's research [after April 16 PT deployment]
 * Additional research available


 * Article Creation Workflow
 * Template changes to draft flow [Early-Mid May]
 * Deployment of A/B to x% of new users to English Wikipedia [Early -Mid May]


 * Page Triage
 * Prototype deployment on English Wikipedia (in parallel with Special:NewPages) [May 2]
 * Work on NO_INDEX
 * Curation toolbar (temporary name)
 * Continue design iterations
 * NoIndex
 * Curation toolbar: labs deployment


 * Echo/Messages
 * Examine hooks into current MW infrastructure for notifications
 * Look into PHP default synchronous notifications
 * Finalize decision on LQT3
 * [June] Build PHP default synch notifications
 * [June] Look into queue based infrastructure extension
 * [July] Start replacing current MW notifications with common notifications infrastructure


 * Misc EE
 * CR on AbuseFilter and Moodbar (Andrew) [IN PROCESS]

Multimedia tools

 * TimedMediaHandler
 * [May] Final code review of new changes and deployment onto Wikimedia Foundation sites
 * [June] Release


 * [UploadWizard]
 * No current work planned

Feature support

 * Wikipedia Education Program
 * Complete testing in Wikimedia Labs and incorporate community feedback - http://education.wmflabs.org/ [Jeroen, QA via Chris] [IN PROCESS]


 * Fundraising
 * Get recurring GlobalCollect up [IN PROCESS]
 * Git migration deployment [IN PROCESS]

MediaWiki infrastructure

 * Gadgets 3.0
 * Includes review of Salvatore's "Gadgets user preferences" work


 * ResourceLoader
 * Gadgets 2.0 deployed after MW 1.19 (dependent on REL1_19 schedule) [in progress]
 * Set up a first working setup in WMF labs [Roan, Timo]

---

MediaWiki Core

 * Lua scripting
 * Return value expansion
 * [move to April] Working CPU limits
 * Providing access to the frame object to Lua
 * [move to April] Deployment to labs
 * Bugfixing and polish
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * Complete 1.20wmf02 deployment
 * Complete 1.20wmf03 deployment


 * Continuous integration
 * Complete beta labs so that it's suitable/trusted for testing (target completion in May) [Antoine, Chris]
 * Continue developing framework for integration testing [Antoine, Chris McMahon]
 * possibly Selenium - see Chris's mail to wikitech-l
 * TestSwarm reintegration with Jenkins [Antoine]


 * Git/Conversion [Chad/Antoine/Sumana]
 * Upgrading to Gerrit 2.3 + Gerrit bugfixing
 * GitHub mirroring (one-way read-only mirror, no pull requests)
 * Moving more projects to Git [Chad - ongoing]


 * SwiftMedia
 * Begin deployment work on originals
 * Bugfixing and cleanup on thumbnails [ongoing]
 * Deploy concurrent file operations - rewrite of external CloudFiles library [Aaron]


 * OAuth
 * Continue specification work [Chris Steipp]


 * CentralAuth
 * Bugfixing [Chris Steipp]


 * Memcache client replacement [Chris Steipp]


 * Replicated session handling [Chris Steipp]

Analytics

 * Report card


 * Log collection pipeline

TL;DR

 * Start Summer of Code 2012 students [Sumana]


 * Prepare tutorials for Berlin Hackathon 2012 [Sumana, contractor]


 * Wikimedia blog maintenance [Guillaume]

QA

 * Improve beta labs so that it's suitable/trusted for testing (target completion in May) [Antoine, Chris]

---

Internationalization/Localization

 * Internationalization and localization tools
 * Universal language selection UI - implementation of proposed design (for desktop and mobile) [i18n team + UI/UX contractors (Pau, Arun)]; [in progress]
 * Translate extension enhancements [Niklas, Santhosh]
 * Narayam maintenance - continuing to add more languages for onscreen keymaps [Santhosh, Amir]
 * WebFonts maintenance [Santhosh]
 * Language impact assessment/metrics measurement [i18n team]; [in progress]
 * Office Hours: Scheduled for next week on #wikimedia-office on Wednesday May 16 (3rd Wednesday of the month)

Editor Engagement Experimentation (E3)

 * This new team focused on engineering for experimentation around strategies to reverse stagnating/declining participation in Wikimedia’s projects. This team is cross-functional with resources from the Community and Engineering/Product departments tasked specifically with conducting small, rapid experiments designed to improve editor retention. The experiments product backlog will be published by the team on a public wiki (meta or en.wp) and the technical documentation will be published on mediawiki.org for all experimental features.

See job listings for the following positions: (http://hire.jobvite.com/Jobvite/Job.aspx?j=oi7iWfwn&c=qSa9VfwQ)
 * Senior Software Developer (http://hire.jobvite.com/Jobvite/Job.aspx?j=oHDiWfwi&c=qSa9VfwQ)
 * Software Developer - FrontEnd (http://hire.jobvite.com/Jobvite/Job.aspx?j=opDhWfwZ&c=qSa9VfwQ)
 * Software Developer - BackEnd

Design

 * Language Selection [Lindsey, consultation with Pau]

Mobile / Offline

 * First MobileFrontend feature related to mobile uploads (e.g. media upload beta).

Platform

 * Prepare IPv6 support for IPv6 Launch (June 6th)
 * Revisit Gerrit, git-review, and other code review/workflow tools (3 months after the git switch) to see whether to switch to Phabricator or another code review tool
 * Configuration management overhaul [Chad]
 * Evaluate WikiFactory from Wikia

Engineering Community Team

 * Summer of Code 2012
 * midterm evaluations


 * blog maintenance

Tech Ops

 * Swift
 * Swift replicas @ EQIAD

Mobile / Offline

 * Mobile Article Feedback feature definition [Phil]
 * Minor Editing [Phil]
 * Offline saving [Phil]

Features

 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Re-energize LiquidThreads (rename it?!) (may need refactoring for notification streams)
 * Respond to open design questions
 * Photo/media moderation tools; additional patrolling and moderation tooling
 * Help features
 * Mentoring features
 * Other real-time features, chat
 * Dependent on MediaWiki 1.20 deployment date, ResourceLoader 2 -- mostly the Gadgets-extension work which is (as of Feb 2012) in a branch.
 * Gadgets 2.0?

Platform / Dev-Ops

 * Extension management -- depends on config mgmt revamp

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance

Parking lot and todos

 * Details of git migration (extension mgmt, permissions, code review tool, Gerrit-Jenkins integration, etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf Wikimedia Platform Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles


 * Replicated session handling
 * HipHop deployment (deferred because Facebook has a different shiny new thing)
 * Extension:Configure (merge with config management?)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)
 * Wikidata (WMDE)
 * BlameMaps (WMDE)
 * graphical UI on top of improved backend for configuration management
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Wikimedia operations history and architecture document


 * Disk-backed object cache
 * DB sharding


 * Interwiki transclusion
 * WMF cannot commit the time to merge this.