Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation. See also: Previous months:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March
 * /2012/April
 * /2012/May
 * /2012/June
 * /2012/July
 * /2012/August

Item syntax: Examples:
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance
 * Recruitment and encouragement of volunteers
 * Monthly report and other engineering communications

Technical Operations

 * Datacenter migration
 * Memcached/Redis
 * Configure Redis server @TAMPA [Mark/Asher/Leslie]
 * Deploy MediaWiki changes to handle sessions from Redis@Tampa [Tim]
 * Deploy MediaWiki changes to dual-write all Memcache entries to Memcached and Redis@Tampa [Tim]
 * Disable Memcached@Tampa [Tim]
 * Configure Redis server @Eqiad [Mark/Asher/Leslie]
 * Test Redis replication from Tampa to Eqiad [Asher/Tim]
 * Wide Apache-on-Precise test deployment @ Tampa [Asher/Peter/Mark]
 * complete packaging, puppetising and testing of Image-Scalers @ Tampa [Peter/Mark/Faidon] [DONE]
 * Limited ImageScalers-on-Precise deployment @ Tampa [Asher/Peter/Mark]
 * Swift
 * Full switch-over (disable NFS) [ONGOING]
 * Initiate Eqiad Swift replication [RobH / Faidon / Ariel] [started but riddled with issues; working on plan B]
 * complete migration of the rest of contents from ms7 to Swift [Aaron]
 * Fundraising infrastructure buildup@ EQIAD [Jeff G]
 * migrate services (logging, puppet, deployment system) from silicon to boron/indium [DONE]
 * move silicon to frack, re-image to provision/test activemq
 * deploy ganglia for new cluster [DONE]
 * test/tune/deploy new payments infrastructure [completed production test; on-going tuning]
 * document new infrastructure [DONE]
 * Varnish
 * deploy Varnish with persistent store [Mark] [testing on mobile] - [DONE]
 * deploy another 8 instances to provide netwrok throughput capacity - [DONE]
 * deploy Varnish with fixes to streaming video performance issues [Mark] - serving 10% traffic now; moving the rest in the coming week
 * Provision & Deploy Poolcounter servers@eqiad [DONE]
 * Eqiad Deployment Host
 * Provision [DONE]
 * migrate from Tampa

Mobile / Offline

 * WLM bug fix v1.2.4 [DONE]
 * WLM bug fix v1.2.5 [DONE]
 * Wind down WLM
 * Sticky cookie issues
 * Begin backlog planning for new nav [DONE]
 * Beta updates
 * Left side nav
 * Watchlists: add article to watchlist (no view on mobile)
 * Login
 * Dynamic Sections
 * Move beta features -> prod
 * Analyze WLM data to guide next upload steps
 * Plan backlog for image triage
 * Hackathon planning [IN-PROGRESS]
 * pickup any remaining redirects [office, chapters, ...]
 * Get featured in the Google Play store for our Wikipedia App

Visual Editor

 * VisualEditor
 * API (implementation of 3 features) [IN PROCESS]
 * Feature implementation and bugfixing on VE according to 3 month plan [IN PROCESS]
 * Parsoid
 * C++ implementation of Parsoid [IN PROCESS]
 * Parsoid fixes for release (December date) [ONGOING]


 * Page Curation (fka Page Triage)
 * Keyboard shortcuts: spec [TOO COMPLEX]
 * Increase visibility of Special:NewPagesFeed (e.g., notice on Special:NewPages) [DONE]
 * Additional socialization [ONGOING]
 * Final deployment [DONE]


 * Article feedback
 * Link with page protection [IN PROCESS]
 * Filtering of comments [TBD]
 * No further feature development, waiting on sharding work


 * Echo_(Notifications) & Flow
 * Project planning for Notifications (in preparation for E2 team to start work once Page Curation is finished) [IN PROCESS]
 * Initial set of interviews (with editors at Foundation to gather initial set of use cases) [9/26]
 * UI and bugfixing of Echo [ONGOING]
 * see Performance engineering and Database Sharding for prep work on Echo & Flow


 * E3
 * Clicktracking/Analytics
 * E3 analytics, Clicktracking re-design [IN PROCESS]
 * Post-edit Feedback 2 [IN PROCESS]
 * Data analysis once experiment is complete [IN PROCESS]
 * Community Portal on en.wp
 * Continued click-tracking work on Community Portal (funnel analysis) [IN PROCESS]
 * Further UX improvements for Community Portal page on en.wp [TBD]
 * Account Creation User Experience
 * UI/UX mockups [DONE]
 * Development [IN PROCESS]
 * Clicktracking enablement - first iteration [DONE]

Multimedia

 * UploadWizard
 * One sprint devoted to UploadWizard bugs for WLM [Kaldari/Holmquist] [DONE]
 * Integration of Flickr GSoC work (requires operational infrastructure change) [COMPLETE/BLOCKED]
 * TimedMediaHandler
 * Bugfixing of player on test2 [IN PROGRESS] (Michael Dale)
 * Deployment of transcoding boxes

Support

 * ResourceLoader 2 -- mostly the Gadgets-extension work which is (as of June 2012) in a branch.
 * Testing existing gadgets on WMF labs for breakage [Timo] [IN PROCESS]
 * "Gadgets 3.0": Includes getting a version deployed/integrated, then a review of Salvatore's "Gadgets user  preferences" work, then a merge of Salvatore's work
 * Discussion of global scripts hosting [IN PROCESS]


 * Fundraising Engineering
 * Add Adyen payment processor [IN PROCESS]
 * Refactor PayPal, Moneybookers processors [IN PROCESS]

MediaWiki Core

 * Datacenter migration (see TechOps section above)
 * Git/Conversion (Chad/Antoine)
 * Have last of extensions out of SVN and into Git [IN PROGRESS]
 * Upstream development of wildcard replication
 * Improved project listing for 2.5 [Chad] [DONE]
 * Stage Gerrit 2.5 for testing
 * OAuth
 * Discuss at tech days/all staff [ChrisS]
 * Swift
 * See Swift section in Ops
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf11 [DONE]
 * 1.20wmf12 [ONGOING]
 * Continuous integration
 * Admin tools development
 * Wikidata deployment
 * Code review of extensions [Sam, Tim, Chris, Chad] [IN PROGRESS]
 * Site performance

QA

 * Publish latest versions of production extensions (AFTv5, NewPagesFeed, etc.) to Beta Labs automatically from master [Antoine/Chris] [DONE]
 * Have production extensions configured properly with Settings files managed via gerrit [DONE]
 * Engaged with all Features testing [Chris] [ONGOING]

Analytics
See also: Analytics/Roadmap


 * (Kraken) Cache Server Log Aggregation Plan [dsc, Otto] [DONE]
 * (Kraken) Set up Cassandra cluster, get it working with Hadoop. [otto + dsc]
 * Load in sample data sets. [otto]
 * Tee the udp2log stream into Kraken. [otto + dsc]
 * First-pass at Hive/Pig Jobs [dsc + otto]
 * (Kraken) Puppetize Kraken [otto] [ongoing]
 * (Legacy Log Collection) Add support for new domain names in webstatscollector (blog, etc) [diederik]
 * (Data) Create Data Release Practices Task Force [diederik]
 * (Limn) Bootstrap Dan [dan + dsc] [DONE]
 * (Limn) Refactor charting to use d3 [dan + dsc]
 * Initial Prototype with Options UI
 * (Limn) Mirror GitHub to Gerrit [dsc]
 * (Limn) Coke ( for Coco) task to create symlinks into   from an existing data repository (such as, say, analytics/reportcard/data) [dsc]
 * Coke task to download and setup dummy testing data for ease of development [dsc]
 * (Limn) Migrate Dario's dashboards to Limn [dsc]
 * (Limn) Support the Global Dev dashboard [evan] [ongoing]
 * (Limn) Support the Gerrit Stats dashboard [diederik] [ongoing]

Engineering Community Team

 * GSoC wrapup & postmortem [Sumana + volunteers]: blog post to post 27 Sept
 * Bug Wrangling [Bug Wrangler + volunteers]
 * Preparation for upcoming events [Sumana, Technical Contributor Coordinator]
 * Tech Ambassadors Network [Guillaume]
 * Volunteer product management program [Guillaume]

i18n/L10n

 * Internationalization/Localization
 * Universal Language Selector
 * Version 1 development [IN PROGRESS]
 * User Experience / UI testing [ONGOING]
 * Technical documentation [TBD]
 * Project Lassi: Translate UX Improvements
 * User Experience / UI testing [IN PROGRESS]
 * Prototype development [IN PROGRESS]
 * Translate API
 * API definition [On Hold]
 * TTMServer
 * Labs setup of SOLR review DONE
 * Testing of SOLR / TTMServer DONE
 * Deployment to 7 wikis IN PROGRESS
 * Web Fonts
 * Adding languages [ONGOING]
 * Narayam
 * Adding input methods for other languages [ONGOING]
 * Project Milkshake
 * Design specification, API specification, development [IN PROGRESS]
 * Technical documentation [TBD]
 * Language search
 * Initial analysis [DONE]
 * Prototype initial concept [IN PROGRESS]
 * Integration into ULS [IN PROGRESS]

Technical Operations

 * Redis
 * Deploy Memcached/Redis @ Tampa [Mark/Asher/Tim]
 * Replicate Redis to EQIAD [Mark/Asher/Tim]
 * Deploy Etherpad Lite [Daniel/RobH]
 * Deploy Mingle [RobH/DZ]
 * Deploy eqiad application serving (Oct 15)
 * Labs

Features

 * New Pages Feed (fka Page Triage)
 * Descriptive statistics on NPF (revised dashboard on toolserver) [Dario] [IN PROCESS]
 * Bug fixes
 * Additional socialization and public announcement/launch
 * Article Creation Workflow
 * TBD, depending on sharding work
 * Changes to visual design to make ACW consistent with Design Toolkit [TBD]
 * Echo_(Notifications)
 * Continued development of prototype for Mediawiki.org [Werdna/Vibha]
 * Selection of one use case (or a limited set, e.g., user talk notifications) and map across different areas of product (e.g., web display, preferences, email, bundling, etc.)
 * re-deployment of prototype for Mediawiki.org [Werdna/Vibha] [IN PROCESS]
 * UI and bugfixing of Echo [ONGOING]
 * Global preferences for echo delivery [TBD]
 * centralization and cross wiki notification delivery [TBD]
 * (see Performance engineering for infrastructure prep work:)
 * queueing & Echo
 * Database Sharding & Flow


 * Article feedback
 * performance/db related changes to AFTv5 queries before scaling, + sharding (see Database sharding) [IN PROCESS]

Multimedia

 * TimedMediaHandler
 * Bugfixing of player on test2 [IN PROGRESS] (Michael Dale)
 * Deployment of transcoding boxes
 * Create production deployment plan

Mobile / Offline

 * Move beta features -> prod
 * Analyze WLM data to guide next upload steps
 * Plan backlog for image triage
 * pickup any remaining redirects [office, chapters, ...]

MediaWiki Core

 * Datacenter migration (see TechOps section above)
 * Git/Conversion (Chad/Antoine)
 * Upgrade to 2.5
 * Mirror mediawiki-core to GitHub (waiting on Gerrit 2.5) [Chad]
 * Have last of extensions out of SVN and into Git
 * Finish replicating all repos to GitHub
 * Begin researching how to get pull requests back in
 * Have gitweb replacement in place (probably GitBlit)
 * OAuth [ChrisS]
 * Swift
 * See Swift section in Ops
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * 1.20wmf13
 * 1.20wmf14
 * Continuous integration
 * Admin tools development
 * Wikidata deployment
 * Code review of extensions [Sam, Tim]
 * Site performance
 * Lua scripting [Tim]

QA

 * Continue refining beta labs performance [Antoine/Chris/NewQA]
 * Begin work on Watir-based browser testing framework
 * Create at least one useful end-to-end test for important Wikipedia feature, with automatic execution (Jenkins-driven) and results reporting [Chris/NewQA]

Analytics
See also: Analytics/Roadmap


 * (Kraken) Set up Maven / Somatype Artifact Repository (continuous integration) [dsc]
 * WMF Maven parent pom [dsc]
 * (Kraken) Puppetize Kraken [otto] (Ongoing)
 * (Kraken) Set up JMX monitoring -- needs to be our LAN [otto + dsc]
 * (Kraken) Get Storm set up [dsc + otto]
 * Start work on ETL topology [dsc]
 * (Kraken) Hardware reinstallation -- Depends on Ops [otto]
 * (Kraken) Get to consensus with Ops regarding logging of the firehose [dsc + otto]
 * Research needed: test running cli JVM producers does not cause extra load [otto]
 * (Legacy Log Collection) udp2log filters
 * Update filters for Wikipedia Zero [otto] (Ongoing)
 * Filter by X-Carrier headers. [otto + asher + diederik]
 * udp-filter to filter by http status. [otto]
 * (WikiStats) Reduce backlog regarding Wikistats traffic (squid etc) scripts [stefan]
 * (WikiStats) Make wikistats more robust (MoM validations) [ezachte]
 * (WikiStats) Add Blackbox testing to WikiStats [diederik + ezachte]
 * (Ops & Maintenance) Access/support requests for stat1, stat1001 [otto] (Ongoing)
 * (Ops & Maintenance) Migrate Reportcard off Labs onto stat1001 -- reportcard.wikimedia.org [otto + dsc]
 * (Ops & Maintenance) Maintenance of oxygen/emery/locke [otto] (Ongoing)
 * (Data) Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dsc] (Ongoing)
 * (Data) Start pushing datasets to AWS [diederik]
 * (Data) Finalize scripts to massively compact dammit.lt data [erik]
 * Blogpost about what awesome stuff you can do with this [diederik + ?]
 * (Limn) Refactor charting to use d3 [dan + dsc]
 * Feature Parity with Dygraphs (plus bugfixes, etc)
 * (Limn) Bugfixes (like Save-As, UI Error Notifications, ...) [dan]
 * (Limn) Improve Limn wiki, docs, & guides (esp those shameful screenshots) [dan]
 * (Limn) UI support for remote datasets via proxy [dsc + dan]
 * (Limn) Support the Global Dev dashboard [evan] (Ongoing)
 * (Limn) Support the Gerrit Stats dashboard [diederik] (Ongoing)
 * (Limn) Deploy reportcard / gerrit-stats on stat1001 (aka, "the Debian packaging discussion") [otto + dsc]

Engineering Community Team

 * GSoC wrapup & postmortem [Sumana + volunteers]
 * Bug Wrangling [Bug Wrangler + volunteers]
 * Online or in-person QA trainings for the community [Technical Contributor Coordinator]
 * Community testing events [Technical Contributor Coordinator]
 * Preparation for upcoming events [Sumana, Technical Contributor Coordinator]
 * India hackathon in the fall? [Sumana, Technical Contributor Coordinator]
 * Tech Ambassadors Network [Guillaume]
 * Volunteer product management program [Guillaume]
 * UCOSP idea-wrangling

MediaWiki Core

 * Git/Conversion (Chad/Antoine)
 * Have Lucene work underway upstream for 2.6 release cycle [Chad]

Engineering Community Team

 * Event [Sumana, Technical Contributor Coordinator]
 * Bangalore hackathon [Sumana, Technical Contributor Coordinator]

Technical Operations

 * Labs
 * Push OpenStackManager changes to show SSH fingerprints for instances [Ryan Lane] (not a high-priority item, will probably be December 2012 or later)
 * Add DNS support to Nova (in essex, but we must migrate to using it)
 * Add Gluster support to Nova (Gluster plugin is mostly written but we don't really have plugin support until folsom, so will happen in Q3 when we are ready to upgrade to Folsom)

Parking lot and todos

 * Install Lua (Scribunto) on other Wikimedia wikis (probably sometime in 2013; see Goals)
 * Category collation
 * Patch waiting on Liangent for more work for full deployment
 * Queuing infrastructure work for Flow (who added this?)
 * Configuration management overhaul [Chad]
 * Evaluate WikiFactory from Wikia
 * Extension management -- depends on config mgmt revamp
 * Details of git migration (extension mgmt, permissions, code review tool, Gerrit-Jenkins integration, etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf Wikimedia Platform Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles
 * Replicated session handling
 * HipHop deployment (deferred because Facebook has a different shiny new thing)
 * Extension:Configure (merge with config management?)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)
 * Wikidata (WMDE)
 * BlameMaps (WMDE)
 * graphical UI on top of improved backend for configuration management
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Wikimedia operations history and architecture document
 * Disk-backed object cache
 * DB sharding
 * Help features
 * Mentoring features
 * Other real-time features, chat
 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Gadgets 2.0?
 * Interwiki transclusion
 * WMF cannot commit the time to merge this.
 * Replicated session handling [Tim Starling]
 * Relicense our Lucene contributions [future Lucene search engineer]