Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation. See also: Previous months:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF  goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March
 * /2012/April
 * /2012/May
 * /2012/June
 * /2012/July

Item syntax: Examples:
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance
 * Recruitment and encouragement of volunteers
 * Monthly report and other engineering communications

Technical Operations

 * Deploy Memcached @TAMPA [Mark/Asher/Leslie]
 * Provision Memcached servers @ EQIAD [RobH}
 * Swift
 * Move to new 404 handler (Aaron/Ben) [DONE]
 * Cutover Originals (multi-media) on Swift - Tampa [Aaron/Ben]
 * Write new originals to Swift [DONE]
 * Reads from Swift with NFS as backup [ongoing, deployed to test*, mw.org]
 * Full switch-over (disable NFS)
 * upgrade Tampa Swift cluster to 1.5 from 1.43 [Ben]
 * build up Eqiad Swift and start replication [RobH / Ben]
 * Fundraising infrastructure buildup@ EQIAD
 * build & deploy MQ server
 * deploy Payment Cluster
 * deploy applications
 * complete packaging, puppetising and testing of Image-Scalers @ Tampa [Peter/Mark/Faidon]
 * Limited Apache-on-Precise deployment @ Tampa [Asher/Peter/Mark]
 * Provision Application servers @ Eqiad
 * Varnish
 * deploy Varnish with persistent store [Mark] [testing on mobile]
 * deploy Varnish with fixes to streaming video performance issues [Mark] [delayed]
 * NetApp /home deployment [Mark]
 * Install new MX80 @ Evoswitch [mark]
 * Labs deployment in eqiad, OpenStack release upgrade [Ryan/Faidon]

Mobile / Offline

 * Transition to the team to using Mingle [DONE]
 * Release beta1 WLM App [DONE]
 * Release beta2 WLM App
 * Message out to community [DONE]
 * Create marketing materials [DONE]
 * Deploy WMF WLM API server [DONE]
 * Turn on GeoData for test wiki [DONE]
 * Continue Zero testing with new mobile partner [DONE]
 * Continue SMS/USSD testing with new mobile partner [DONE]


 * Evaluate additional features
 * Mobile Article Feedback feature definition [Phil]
 * Minor Editing [Phil]
 * New Zero Launches

Features

 * VisualEditor
 * API (implementation of 3 features) [IN PROCESS]
 * Feature implementation and bugfixing on VE according to 3 month plan [IN PROCESS]
 * Assessment of C/C++ implementation of Parsoid [IN PROCESS]
 * Parsoid fixes for release (December date) [ONGOING]


 * New Pages Feed (fka Page Triage)
 * Public soft launch of curation toolbar feature (link on Special:NewPages, curation toolbar by default): [DONE]
 * Other potential changes to be prioritized (e.g., message to article author, visual changes to New Pages Feed, etc.) [DONE]
 * Increase visibility of Special:NewPagesFeed (e.g., notice on Special:NewPages) [Week of Aug 6]
 * Additional socialization [ONGOING]


 * Article feedback
 * Add feedback pages to watchlist [DONE]
 * assessment of scalability issues surrounding AFTv5 [DONE]
 * performance/db related changes to AFTv5 queries before scaling [IN PROCESS]
 * bugfixing and small feature enhancements [ONGOING]
 * Other issues for enwp release?


 * Echo_(Notifications) & Flow
 * Deployment of prototype for Mediawiki.org [Werdna/Vibha] [DONE]
 * UI and bugfixing of Echo [ONGOING]
 * see Performance engineering and Database Sharding for prep work on Echo & Flow


 * ResourceLoader 2 -- mostly the Gadgets-extension work which is (as of June 2012) in a branch.
 * Testing existing gadgets on WMF labs for breakage [Timo] [IN PROCESS]
 * "Gadgets 3.0": Includes getting a version deployed/integrated, then a review of Salvatore's "Gadgets user preferences" work, then a merge of Salvatore's work
 * Set up a first working setup in WMF labs [Roan, Timo]


 * TimedMediaHandler
 * Bugfixing of player on test2
 * Deployment of transcoding boxes
 * Possible work on writing transcodes exclusively to Swift
 * Create production deployment plan


 * Fundraising Engineering
 * Auditing and Reconciliation (expect to complete mid-late August) [IN PROCESS]

i18n & R/D

 * E3
 * Clicktracking/Analytics
 * E3 analytics, Clicktracking re-design [IN PROGRESS]
 * Post-edit Feedback
 * Data analysis [IN PROGRESS]
 * Community Portal on en.wp
 * Enabling clicktracking on Community Portal page on en.wp [DONE]
 * UX improvements for Community Portal page on en.wp [DONE]
 * Account Creation User Experience
 * UI/UX mockups [IN PROGRESS]
 * Development [IN PROGRESS]
 * Clicktracking enablement - first iteration [DONE]
 * Internationalization/Localization
 * Universal Language Selector
 * Prototype [IN PROGRESS]
 * User Experience / UI testing [ONGOING]
 * Translate UX
 * Prototype [IN PROGRESS]
 * User Experience / UI testing [IN PROGRESS]
 * Translate API
 * API definition [IN PROGRESS]
 * TTMServer
 * Labs setup of SOLR review pending from Ops; otherwise ready for deployment
 * Web Fonts
 * Adding languages [ONGOING]
 * Narayam
 * Adding input methods for other languages [ONGOING]
 * Project Milkshake
 * Design specification, API development [IN PROGRESS]
 * Language search
 * Initial analysis [DONE]
 * Prototype initial concept [IN PROGRESS]
 * Integration into ULS [IN PROGRESS]

MediaWiki Core

 * Lua scripting
 * Deploy to test2 [DONE]
 * Deploy to mediawiki.org
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf9 [DONE]
 * 1.20wmf10
 * Continuous integration
 * On hold in August
 * Git/Conversion [Chad/Antoine]
 * Finalize decision on Gerrit vs other tools [Brion, David, Chad] [DONE]
 * Improvements to project listing screen upstream [IN PROGRESS]
 * Research current Lucene status upstream & begin pressing on that [Chad]
 * Create build environment so we can deploy custom builds [IN PROGRESS]
 * Upgrade to 2.5
 * Overhaul puppet config for gerrit and get labs functioning [IN PROGRESS]
 * Mirror mediawiki-core to Github (waiting on Gerrit 2.5) [Chad]
 * SwiftMedia
 * See "Swift" section in Ops
 * OAuth
 * Finalize specifications, identify/recruit developers [ChrisS] [on hold]
 * Admin tools development
 * CentralAuth and AbuseFilter bugfixing [Chris Steipp] [ongoing]
 * Global AbuseFilter rules [Chris Steipp] [ongoing]
 * Global AbuseFilter variables/throttling [Chris Steipp]
 * CentralAuth mass account blocking [Chris Steipp] (possibly September)
 * Site performance
 * Assessment of PHP 5.4 and suhosin removal [DONE] [Tim/Faidon/Asher]
 * Ready to move to PHP 5.4 when Ops is ready
 * Incremental architectural improvements
 * Category collation
 * Pt wiki fix ready for deployment by Tim - window scheduled for Tuesday, August 21
 * Database sharding
 * Initial prototype work (Aaron/Asher/Matthias) [IN PROCESS]
 * Awaiting review by Tim
 * Wikidata deployment
 * Code review of extensions [Sam, Tim]

QA

 * Start helping test NewPagesFeed/Page Curation Toolbar  [Chris] [DONE,ONGOING]
 * Enable and configure useful extensions on beta labs (AFTv5, NewPagesFeed, Curation Toolbar, etc.) for testing [Antoine/Chris][IN PROGRESS] (final step, deploying to beta via Jenkins/git was waiting on gluster upgrade)
 * possible Sept test event for PageTriage/NewPagesFeed/Curation Toolbar [Chris][IN PROGRESS] (final decision by Aug 17)
 * miscellaneous projects as required: EtherEdit testing, TMH testing, documentation, small projects [ONGOING]
 * Hiring for QA Engineer, Bug Wrangler and Volunteer Eng Coordinator

Analytics

 * Cache Server Log Aggregation Plan [Dave, Otto] [IN PROGRESS]
 * Complete testing Kafka
 * Complete architectural suggestions document
 * Present to Ops, get signoff, throw party
 * Hand off Limn to new engineer [Dave] [pending]
 * Hire new engineer (citation needed!)
 * Prototype of Data Warehouse [Dave, Otto] [pending]
 * Test CDH4 on Cassandra 1.x
 * Test DSE

Engineering Community Team

 * Bug Wrangling [Bug Wrangler + volunteers]
 * Activate Tech Ambassadors Network [Guillaume]
 * Google Summer of Code wrapup [Sumana]
 * Planning events [Sumana]
 * Hiring

Technical Operations

 * Install new passive optics system between the 2 datacenters in Tampa
 * CWDM Optics to increase bandwith capacity [Leslie]
 * Wide Apache-on-Precise test deployment @ Tampa [Asher/Peter/Mark]
 * Limited ImageScalers-on-Precise deployment @ Tampa [Asher/Peter/Mark]
 * Swift
 * Complete Eqiad Swift replication [RobH / Ben]

MediaWiki Core

 * Memcache client replacement
 * Deploy memcache client to Apaches (RT 3155) - possibly blocked on Precise deployment to eqiad
 * Configuration change (to be scheduled)
 * Git/Conversion (Chad/Antoine/Sumana)
 * Have last of extensions out of SVN and into Git
 * Have Lucene work underway upstream for 2.6 release cycle
 * Finish replicating all repos to GitHub
 * Begin researching how to get pull requests back in
 * Have gitweb replacement in place (probably GitBlit)
 * OAuth
 * Discuss at tech days/all staff [ChrisS]
 * Swift
 * See Swift section in Ops
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf11
 * 1.20wmf12
 * Continuous integration
 * Admin tools development
 * Wikidata deployment
 * Code review of extensions [Sam, Tim]
 * Site performance

QA

 * use beta cluster hosting latest version of prod extensions (AFTv5, NewPagesFeed, etc.) for testing [Antoine/Chris] [IN PROGRESS]
 * engaged with all Features testing [CHRIS] [ONGOING]
 * contractor at work(?) [IN DISCUSSION]

Analytics

 * Dance Party

Engineering Community Team

 * GSoC wrapup & postmortem [Sumana + volunteers]
 * Bug Wrangling [Bug Wrangler + volunteers]
 * Online or in-person QA trainings for the community [Volunteer Engineering Coordinator]

Features

 * New Pages Feed (fka Page Triage)
 * Additional socialization and public announcement/launch
 * ACW: TBD


 * Article feedback
 * (TBD pending assessment of scalability isssues)


 * Echo_(Notifications) & Flow
 * UI and bugfixing of Echo [ONGOING]
 * Next set of features TBD

i18n & R/D

 * E3
 * Clicktracking/Analytics
 * E3 analytics, Clicktracking re-design [IN PROGRESS]
 * Account Creation User Experience
 * Development [IN PROGRESS]
 * Internationalization/Localization
 * ULS
 * Prototype [IN PROGRESS]
 * User Experience / UI testing [ONGOING]
 * Translate UX
 * Prototype [IN PROGRESS]
 * User Experience / UI testing [IN PROGRESS]
 * Translate API
 * API definition [IN PROGRESS]
 * TTMServer
 * Labs setup of SOLR review pending from Ops; otherwise ready for deployment
 * Web Fonts
 * Adding languages [ONGOING]
 * Narayam
 * Adding input methods for other languages [ONGOING]
 * Project Milkshake
 * Design specification, API development [IN PROGRESS]
 * Language search
 * Prototype initial concept
 * Integration into ULS

Technical Operations

 * Deploy Memcached @EQIAD [Mark/Asher]
 * Deploy Etherpad Lite [Daniel/RobH]
 * Deploy Mingle [RobH/DZ]
 * Deploy eqiad application serving (Oct 15)

Features

 * New Pages Feed (fka Page Triage)
 * Additional feature development (only must-haves)
 * Article Creation Workflow
 * revisit for release
 * Changes to visual design to make ACW consistent with Design Toolkit
 * Echo_(Notifications)
 * Continued development of prototype for Mediawiki.org [Werdna/Vibha]
 * Global preferences for echo delivery
 * centralization and cross wiki notification delivery

Engineering Community Team

 * India hackathon in the fall? [Sumana, Engineering Outreach Coordinator]
 * UCOSP idea-wrangling
 * Community testing events

Parking lot and todos

 * Category collation
 * Patch waiting on Liangent for more work for full deployment


 * Queuing infrastructure work for Flow (who added this?)
 * Enable $wgHtml5 (bug 27478)
 * Configuration management overhaul [Chad]
 * Evaluate WikiFactory from Wikia
 * Extension management -- depends on config mgmt revamp
 * Details of git migration (extension mgmt, permissions, code review tool, Gerrit-Jenkins integration, etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf Wikimedia Platform Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles
 * Replicated session handling
 * HipHop deployment (deferred because Facebook has a different shiny new thing)
 * Extension:Configure (merge with config management?)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)
 * Wikidata (WMDE)
 * BlameMaps (WMDE)
 * graphical UI on top of improved backend for configuration management
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Wikimedia operations history and architecture document
 * Disk-backed object cache
 * DB sharding
 * Help features
 * Mentoring features
 * Other real-time features, chat
 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Gadgets 2.0?
 * Interwiki transclusion
 * WMF cannot commit the time to merge this.
 * Replicated session handling [Tim Starling]
 * Relicense our Lucene contributions [future Lucene search engineer]