Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation. See also: Previous months:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March
 * /2012/April
 * /2012/May
 * /2012/June
 * /2012/July
 * /2012/August

Item syntax: Examples:
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance
 * Recruitment and encouragement of volunteers
 * Monthly report and other engineering communications

Technical Operations

 * Datacenter migration
 * Memcached/Redis
 * Configure Redis server @TAMPA [Mark/Asher/Leslie]
 * Deploy MediaWiki changes to handle sessions from Redis@Tampa [Tim]
 * Deploy MediaWiki changes to dual-write all Memcache entries to Memcached and Redis@Tampa [Tim]
 * Disable Memcached@Tampa [Tim]
 * Configure Redis server @Eqiad [Mark/Asher/Leslie]
 * Test Redis replication from Tampa to Eqiad [Asher/Tim]
 * Wide Apache-on-Precise test deployment @ Tampa [Asher/Peter/Mark]
 * complete packaging, puppetising and testing of Image-Scalers @ Tampa [Peter/Mark/Faidon] [DONE]
 * Limited ImageScalers-on-Precise deployment @ Tampa [Asher/Peter/Mark]
 * Swift
 * Full switch-over (disable NFS) [ONGOING]
 * Initiate Eqiad Swift replication [RobH / Faidon / Ariel] [started but riddled with issues; working on plan B]
 * complete migration of the rest of contents from ms7 to Swift [Aaron]
 * Fundraising infrastructure buildup@ EQIAD [Jeff G]
 * migrate services (logging, puppet, deployment system) from silicon to boron/indium [DONE]
 * move silicon to frack, re-image to provision/test activemq
 * deploy ganglia for new cluster [DONE]
 * test/tune/deploy new payments infrastructure [completed production test; on-going tuning]
 * document new infrastructure [DONE]
 * Varnish
 * deploy Varnish with persistent store [Mark] [testing on mobile] - [DONE]
 * deploy another 8 instances to provide netwrok throughput capacity - [DONE]
 * deploy Varnish with fixes to streaming video performance issues [Mark] - serving 10% traffic now; moving the rest in the coming week
 * Provision & Deploy Poolcounter servers@eqiad [DONE]
 * Eqiad Deployment Host
 * Provision [DONE]
 * migrate from Tampa

Mobile / Offline

 * WLM bug fix v1.2.4 [DONE]
 * WLM bug fix v1.2.5 [DONE]
 * Wind down WLM
 * Sticky cookie issues
 * Begin backlog planning for new nav [DONE]
 * Beta updates
 * Left side nav
 * Watchlists: add article to watchlist (no view on mobile)
 * Login
 * Dynamic Sections
 * Move beta features -> prod
 * Analyze WLM data to guide next upload steps
 * Plan backlog for image triage
 * Hackathon planning [IN-PROGRESS]
 * pickup any remaining redirects [office, chapters, ...]
 * Get featured in the Google Play store for our Wikipedia App

Visual Editor

 * VisualEditor
 * API (implementation of 3 features) [IN PROCESS]
 * Feature implementation and bugfixing on VE according to 3 month plan [IN PROCESS]
 * Parsoid
 * C++ implementation of Parsoid [IN PROCESS]
 * Parsoid fixes for release (December date) [ONGOING]


 * Page Curation (fka Page Triage)
 * Keyboard shortcuts: spec [TOO COMPLEX]
 * Increase visibility of Special:NewPagesFeed (e.g., notice on Special:NewPages) [DONE]
 * Additional socialization [ONGOING]
 * Final deployment [DONE]


 * Article feedback
 * Link with page protection [IN PROCESS]
 * Filtering of comments [TBD]
 * No further feature development, waiting on sharding work


 * Echo_(Notifications) & Flow
 * Project planning for Notifications (in preparation for E2 team to start work once Page Curation is finished) [IN PROCESS]
 * Initial set of interviews (with editors at Foundation to gather initial set of use cases) [9/26]
 * UI and bugfixing of Echo [ONGOING]
 * see Performance engineering and Database Sharding for prep work on Echo & Flow


 * E3
 * Clicktracking/Analytics
 * E3 analytics, Clicktracking re-design [IN PROCESS]
 * Post-edit Feedback 2 [IN PROCESS]
 * Data analysis once experiment is complete [IN PROCESS]
 * Community Portal on en.wp
 * Continued click-tracking work on Community Portal (funnel analysis) [IN PROCESS]
 * Further UX improvements for Community Portal page on en.wp [TBD]
 * Account Creation User Experience
 * UI/UX mockups [DONE]
 * Development [IN PROCESS]
 * Clicktracking enablement - first iteration [DONE]

Multimedia

 * UploadWizard
 * One sprint devoted to UploadWizard bugs for WLM [Kaldari/Holmquist] [DONE]
 * Integration of Flickr GSoC work (requires operational infrastructure change) [COMPLETE/BLOCKED]
 * TimedMediaHandler
 * Bugfixing of player on test2 [IN PROGRESS] (Michael Dale)
 * Deployment of transcoding boxes

Support

 * ResourceLoader 2 -- mostly the Gadgets-extension work which is (as of June 2012) in a branch.
 * Testing existing gadgets on WMF labs for breakage [Timo] [IN PROCESS]
 * "Gadgets 3.0": Includes getting a version deployed/integrated, then a review of Salvatore's "Gadgets user  preferences" work, then a merge of Salvatore's work
 * Discussion of global scripts hosting [IN PROCESS]


 * Fundraising Engineering
 * Add Adyen payment processor [IN PROCESS]
 * Refactor PayPal, Moneybookers processors [IN PROCESS]

MediaWiki Core

 * Datacenter migration (see TechOps section above)
 * Git/Conversion (Chad/Antoine)
 * Have last of extensions out of SVN and into Git [IN PROGRESS]
 * Upstream development of wildcard replication
 * Improved project listing for 2.5 [Chad] [DONE]
 * Stage Gerrit 2.5 for testing
 * OAuth
 * Discuss at tech days/all staff [ChrisS]
 * Swift
 * See Swift section in Ops
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf11 [DONE]
 * 1.20wmf12 [ONGOING]
 * Continuous integration
 * Admin tools development
 * Wikidata deployment
 * Code review of extensions [Sam, Tim, Chris, Chad] [IN PROGRESS]
 * Site performance

QA

 * use beta cluster hosting latest version of prod extensions (AFTv5, NewPagesFeed, etc.) for testing [Antoine/Chris] [IN PROGRESS] 
 * engaged with all Features testing [CHRIS] [ONGOING]
 * contractor at work(?) [IN DISCUSSION]

Analytics
See also: Analytics/Roadmap


 * (Kraken) Cache Server Log Aggregation Plan [dsc, Otto] [DONE]
 * (Kraken) Set up Cassandra cluster, get it working with Hadoop. [otto + dsc]
 * Load in sample data sets. [otto]
 * Tee the udp2log stream into Kraken. [otto + dsc]
 * First-pass at Hive/Pig Jobs [dsc + otto]
 * (Kraken) Puppetize Kraken [otto] [ongoing]
 * (Legacy Log Collection) Add support for new domain names in webstatscollector (blog, etc) [diederik]
 * (Data) Create Data Release Practices Task Force [diederik]
 * (Limn) Bootstrap Dan [dan + dsc] [DONE]
 * (Limn) Refactor charting to use d3 [dan + dsc]
 * Initial Prototype with Options UI
 * (Limn) Mirror GitHub to Gerrit [dsc]
 * (Limn) Coke ( for Coco) task to create symlinks into   from an existing data repository (such as, say, analytics/reportcard/data) [dsc]
 * Coke task to download and setup dummy testing data for ease of development [dsc]
 * (Limn) Migrate Dario's dashboards to Limn [dsc]
 * (Limn) Support the Global Dev dashboard [evan] [ongoing]
 * (Limn) Support the Gerrit Stats dashboard [diederik] [ongoing]

Engineering Community Team

 * GSoC wrapup & postmortem [Sumana + volunteers]: blog post to post 27 Sept
 * Bug Wrangling [Bug Wrangler + volunteers]
 * Preparation for upcoming events [Sumana, Technical Contributor Coordinator]
 * Tech Ambassadors Network [Guillaume]
 * Volunteer product management program [Guillaume]

i18n/L10n

 * Internationalization/Localization
 * Universal Language Selector
 * Version 1 development [IN PROGRESS]
 * User Experience / UI testing [ONGOING]
 * Technical documentation [TBD]
 * Project Lassi: Translate UX Improvements
 * User Experience / UI testing [IN PROGRESS]
 * Prototype development [IN PROGRESS]
 * Translate API
 * API definition [On Hold]
 * TTMServer
 * Labs setup of SOLR review DONE
 * Testing of SOLR / TTMServer DONE
 * Deployment to 7 wikis IN PROGRESS
 * Web Fonts
 * Adding languages [ONGOING]
 * Narayam
 * Adding input methods for other languages [ONGOING]
 * Project Milkshake
 * Design specification, API specification, development [IN PROGRESS]
 * Technical documentation [TBD]
 * Language search
 * Initial analysis [DONE]
 * Prototype initial concept [IN PROGRESS]
 * Integration into ULS [IN PROGRESS]

Technical Operations

 * Redis
 * Deploy Memcached/Redis @ Tampa [Mark/Asher/Tim]
 * Replicate Redis to EQIAD [Mark/Asher/Tim]
 * Deploy Etherpad Lite [Daniel/RobH]
 * Deploy Mingle [RobH/DZ]
 * Deploy eqiad application serving (Oct 15)

Features

 * New Pages Feed (fka Page Triage)
 * Descriptive statistics on NPF (revised dashboard on toolserver) [Dario] [IN PROCESS]
 * Bug fixes
 * Additional socialization and public announcement/launch
 * Article Creation Workflow
 * TBD, depending on sharding work
 * Changes to visual design to make ACW consistent with Design Toolkit [TBD]
 * Echo_(Notifications)
 * Continued development of prototype for Mediawiki.org [Werdna/Vibha]
 * Selection of one use case (or a limited set, e.g., user talk notifications) and map across different areas of product (e.g., web display, preferences, email, bundling, etc.)
 * re-deployment of prototype for Mediawiki.org [Werdna/Vibha] [IN PROCESS]
 * UI and bugfixing of Echo [ONGOING]
 * Global preferences for echo delivery [TBD]
 * centralization and cross wiki notification delivery [TBD]
 * (see Performance engineering for infrastructure prep work:)
 * queueing & Echo
 * Database Sharding & Flow


 * Article feedback
 * performance/db related changes to AFTv5 queries before scaling, + sharding (see Database sharding) [IN PROCESS]

Multimedia

 * TimedMediaHandler
 * Bugfixing of player on test2 [IN PROGRESS] (Michael Dale)
 * Deployment of transcoding boxes
 * Create production deployment plan

Mobile / Offline

 * Move beta features -> prod
 * Analyze WLM data to guide next upload steps
 * Plan backlog for image triage
 * pickup any remaining redirects [office, chapters, ...]

MediaWiki Core

 * Datacenter migration (see TechOps section above)
 * Git/Conversion (Chad/Antoine)
 * Upgrade to 2.5
 * Mirror mediawiki-core to GitHub (waiting on Gerrit 2.5) [Chad]
 * Have last of extensions out of SVN and into Git
 * Finish replicating all repos to GitHub
 * Begin researching how to get pull requests back in
 * Have gitweb replacement in place (probably GitBlit)
 * OAuth [ChrisS]
 * Swift
 * See Swift section in Ops
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf13
 * 1.20wmf14
 * Continuous integration
 * Admin tools development
 * Wikidata deployment
 * Code review of extensions [Sam, Tim]
 * Site performance
 * Lua scripting [Tim]

QA

 * use beta cluster hosting latest version of prod extensions (AFTv5, NewPagesFeed, etc.) for testing [Antoine/Chris] [IN PROGRESS] 
 * engaged with all Features testing [CHRIS] [ONGOING] 
 * contractor at work(?) [IN DISCUSSION]

Analytics
See also: Analytics/Roadmap


 * (Kraken) Set up Maven / Somatype Artifact Repository (continuous integration) [dsc]
 * WMF Maven parent pom [dsc]
 * (Kraken) Puppetize Kraken [otto] (Ongoing)
 * (Kraken) Set up JMX monitoring -- needs to be our LAN [otto + dsc]
 * (Kraken) Get Storm set up [dsc + otto]
 * Start work on ETL topology [dsc]
 * (Kraken) Hardware reinstallation -- Depends on Ops [otto]
 * (Kraken) Get to consensus with Ops regarding logging of the firehose [dsc + otto]
 * Research needed: test running cli JVM producers does not cause extra load [otto]
 * (Legacy Log Collection) udp2log filters
 * Update filters for Wikipedia Zero [otto] (Ongoing)
 * Filter by X-Carrier headers. [otto + asher + diederik]
 * udp-filter to filter by http status. [otto]
 * (WikiStats) Reduce backlog regarding Wikistats traffic (squid etc) scripts [stefan]
 * (WikiStats) Make wikistats more robust (MoM validations) [ezachte]
 * (WikiStats) Add Blackbox testing to WikiStats [diederik + ezachte]
 * (Ops & Maintenance) Access/support requests for stat1, stat1001 [otto] (Ongoing)
 * (Ops & Maintenance) Migrate Reportcard off Labs onto stat1001 -- reportcard.wikimedia.org [otto + dsc]
 * (Ops & Maintenance) Maintenance of oxygen/emery/locke [otto] (Ongoing)
 * (Data) Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dsc] (Ongoing)
 * (Data) Start pushing datasets to AWS [diederik]
 * (Data) Finalize scripts to massively compact dammit.lt data [erik]
 * Blogpost about what awesome stuff you can do with this [diederik + ?]
 * (Limn) Refactor charting to use d3 [dan + dsc]
 * Feature Parity with Dygraphs (plus bugfixes, etc)
 * (Limn) Bugfixes (like Save-As, UI Error Notifications, ...) [dan]
 * (Limn) Improve Limn wiki, docs, & guides (esp those shameful screenshots) [dan]
 * (Limn) UI support for remote datasets via proxy [dsc + dan]
 * (Limn) Support the Global Dev dashboard [evan] (Ongoing)
 * (Limn) Support the Gerrit Stats dashboard [diederik] (Ongoing)
 * (Limn) Deploy reportcard / gerrit-stats on stat1001 (aka, "the Debian packaging discussion") [otto + dsc]

Engineering Community Team

 * GSoC wrapup & postmortem [Sumana + volunteers]
 * Bug Wrangling [Bug Wrangler + volunteers]
 * Online or in-person QA trainings for the community [Technical Contributor Coordinator]
 * Community testing events [Technical Contributor Coordinator]
 * Preparation for upcoming events [Sumana, Technical Contributor Coordinator]
 * India hackathon in the fall? [Sumana, Technical Contributor Coordinator]
 * Tech Ambassadors Network [Guillaume]
 * Volunteer product management program [Guillaume]
 * UCOSP idea-wrangling

MediaWiki Core

 * Git/Conversion (Chad/Antoine)
 * Have Lucene work underway upstream for 2.6 release cycle [Chad]

Engineering Community Team

 * Event [Sumana, Technical Contributor Coordinator]
 * Bangalore hackathon [Sumana, Technical Contributor Coordinator]

Parking lot and todos

 * Install Lua (Scribunto) on other Wikimedia wikis (probably sometime in 2013; see Goals)
 * Category collation
 * Patch waiting on Liangent for more work for full deployment


 * Queuing infrastructure work for Flow (who added this?)
 * Configuration management overhaul [Chad]
 * Evaluate WikiFactory from Wikia
 * Extension management -- depends on config mgmt revamp
 * Details of git migration (extension mgmt, permissions, code review tool, Gerrit-Jenkins integration, etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf Wikimedia Platform Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles
 * Replicated session handling
 * HipHop deployment (deferred because Facebook has a different shiny new thing)
 * Extension:Configure (merge with config management?)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)
 * Wikidata (WMDE)
 * BlameMaps (WMDE)
 * graphical UI on top of improved backend for configuration management
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Wikimedia operations history and architecture document
 * Disk-backed object cache
 * DB sharding
 * Help features
 * Mentoring features
 * Other real-time features, chat
 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Gadgets 2.0?
 * Interwiki transclusion
 * WMF cannot commit the time to merge this.
 * Replicated session handling [Tim Starling]
 * Relicense our Lucene contributions [future Lucene search engineer]