Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation. See also: Item syntax: Examples:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance
 * Recruitment and encouragement of volunteers
 * Monthly report and other engineering communications

Features

 * VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
 * IE in VE [IN PROGRESS]
 * bug fixing on Visual Editor [ONGOING]
 * limited production release on French, Hebrew Wikipedia [IN PROGRESS]
 * 5-month plan for VE [late Jan]
 * Node.js parsoid to completion [ONGOING]
 * [Parsoid testing framework for semantic/superficial regressions|http://parsoid.wmflabs.org:8001] [IN PROCESS]
 * Parsoid architectural review (parser team, tim)
 * i18n support in Parsoid (page links, etc) [IN PROCESS]
 * assessment for next steps in Parsoid [IN PROCESS]
 * Echo_(Notifications) (Kaldari, Benny, Werdna, Luke Welling, Nischay, Krenair) (roadmap)
 * Continued development of prototype on Mediawiki.org [ONGOING]
 * UI and bugfixing of Echo [ONGOING]
 * HTML e-mail (Luke) [IN PROCESS]
 * Flyout + archive refactor [IN PROCESS]
 * New features on MediaWiki.org: new editor notifications, deduping logic [Jan 28]
 * i18n issues on Echo
 * JobQueue abstraction (zmq) [Luke] [IN PROCESS]
 * JobQueue abstraction (Redis) [Luke] [IN PROCESS]
 * Jan 30 community testing event [Quim, Chris McMahon] [IN PROCESS]
 * [Flow] (Werdna, JamesF, Brandon, Matthias, LukeW, Isarra)
 * Global user changes [JamesF] [IN PROCESS]
 * Kickoff [IN PROCESS]
 * reach out to Wikia [Terry, JamesF]
 * Article feedback (Matthias)
 * performance/db related changes to AFTv5 queries [Matthias] [IN PROCESS]
 * code review for performance/db patch [pending]
 * i18n messaging update [IN PROCESS]
 * Other minor changes prior to full English Wikipedia deployment [Jan]
 * refactored deployment on enwiki + dewiki pilot (may involve db pruning) [Late Jan/earlyFeb]
 * vertically partitioned database deploy [Feb]
 * New Pages Feed (fka Page Triage) (Kaldari, Benny, Nischay)
 * Bug fixes (no more feature development) [ONGOING]
 * E3 (Ori, S, MattF, Luke)
 * Clicktracking/Analytics
 * Funnel tracking of Community Portal [ONGOING]
 * Funnel tracking of Onboarding [IN PROCESS]
 * Account Creation User Experience
 * Transition to EventLogging (off of ClickTracking) incl. permanent campaign support [IN PROCESS]
 * Productization of ACUX [IN PROCESS]
 * Onboarding
 * Javascript bugs in guided tour [DONE]
 * get Guided Tour ready for deployment incl. creating onboarding tour [IN PROCESS]
 * version two of Special:GettingStarted [IN PROCESS]
 * tagging edits made through onboarding [IN PROCESS]
 * Other Account Creation Funnels
 * 22k Donor Survey email [IN PROCESS]
 * Contribution banner [DONE]
 * Donor Thank You email [DONE]
 * CAPTCHA testing [IN PROCESS]
 * MicroDesign
 * other edit window templates [IN PROCESS]
 * (see Agora)

Technical Operations

 * Datacenter migration (eqiad)
 * Updating the scap/git deploy scripts to deal with multi-datacenter
 * deploy redis and memcached@eqiad [done]
 * deploy parser cache@eqiad
 * deploy apache app servers@eqiad
 * deploy API servers@eqiad
 * deploy jobrunner servers@eqiad
 * database failover from Tampa to Eqiad
 * deploy parsoid servers@eqiad
 * setting up & testing Ceph@eqiad
 * deploy sharded parser cache @ tampa - pc2 & pc3
 * Deploy Etherpad Lite [Daniel/RobH]

Mobile

 * Quarterly planning session with team: focus on photo uploads (e.g., commons app, productizing mobile photo uploads) [DONE]
 * Getting MobileFrontend onto beta labs
 * Mobile Web
 * Beta
 * Photo Upload bug fixes [DONE]
 * Watchlist (reader + recent changes views) [IN-PROGRESS]
 * Thumbnail A-Z view [DONE]
 * Watchlist CTA [DONE]
 * Account Creation [DONE]
 * Login [DONE]
 * Mobile diff [DONE]
 * History
 * Production
 * Watchlist (reader + recent changes views)
 * UI Tweaks
 * Login
 * Mobile diff
 * Account Creation
 * Kill section 0 [DONE]
 * Use full resource loader in production [DONE]
 * Better resource loader support
 * Solr Data replication [DONE]
 * Solr Monitoring

MediaWiki Core

 * Datacenter migration (see TechOps section above) [Tim, Aaron, Sam, ChrisS, Brad, Antoine]  39106
 * git-deploy migration (43338) and file path issues for eqiad migration (43340) [Brad, Antoine, Sam, ChrisS]
 * Remove production MediaWiki dependency on NFS (43495) [Aaron, Tim]
 * Git/Conversion (Chad)
 * LDAP fixes for our Gerrit use (current patch) [Chad]
 * Upgrade to 2.6 [Chad]
 * Replace gitweb with GitBlit [Chad]
 * MediaWiki 1.21/Roadmap
 * MediaWiki 1.21/wmf7 - ongoing, done on Mon 14 Jan
 * MediaWiki 1.21/wmf8
 * Admin tools development [Chris]
 * Build and deploy Global RenameUser extension [CSteipp]
 * Lua scripting [Tim, Chad, Brad]
 * Language library [Tim] [DONE]
 * Unicode library [Brad] [DONE]
 * Search
 * Onboarding new engineer
 * Wikidata client deployment to Hungarian Wikipedia
 * Jan 14: initial rollout

Multimedia

 * TimedMediaHandler
 * Bugfixing (Jan)

QA

 * QA and testing
 * Datacenter migration, Echo
 * See also QA/Roadmap
 * Browser testing
 * Datacenter migration automation: IN PROCESS
 * Beta labs (beta cluster)
 * Add specific reviewer in Gerrit on SQL changes [IN PROCESS] [Antoine]
 * Continuous integration

Analytics
See also: Analytics/Roadmap

Kraken

 * Storm pom [dsc]
 * Storm ETL
 * Get Storm set up [dsc + otto] (Dec) [DONE]
 * Storm ETL bolt for GeoIP [dsc + otto]
 * Storm ETL bolt for HDFS Import [dsc + otto]
 * HDFS Import enforces product-code directories and permissions [dsc + otto] (Jan)
 * Storm ETL bolt for Anonymization [dsc + otto]
 * Storm ETL bolt for Kafka Checkpointing [dsc + otto] (Jan)
 * Puppetize Kraken [otto] (Ongoing)
 * Create oozie workflows for pageview {Wikipedia Zero / Mobile} job counts [diederik + otto]
 * Finish setting up nexus repository [diederik + otto]
 * Collect user requirements for funnel analysis [diederik]
 * Create funnel analysis tool [diederik + dan]
 * Create debian package for dClass [stefan + dvl]

Limn

 * Refactor charting to use d3 [dan + dsc] (Jan)
 * Feature Parity with Dygraphs (plus bugfixes, etc) (Jan)
 * Bugfixes (like Save-As, UI Error Notifications, ...) [dan] (Jan)
 * Migrate Dario's dashboards to Limn [dan] (Jan)
 * Improve Limn wiki, docs, & guides (esp those shameful screenshots) [dan] (Jan)
 * Coke ( for Coco) task to create symlinks into   from an existing data repository (such as, say, analytics/reportcard/data) [dan] (Jan)
 * Coke task to download and setup dummy testing data for ease of development [dsc] (Jan)
 * UI support for remote datasets via proxy [dsc + dan] (Jan)
 * Deploy reportcard / gerrit-stats on stat1001 (aka, "the Debian packaging discussion") [dvl + dan] (Jan)
 * Support the Global Dev dashboard [evan] (Ongoing)
 * Support the Gerrit Stats dashboard [diederik] (Ongoing)

Legacy Log Collection

 * udp2log filters
 * Update filters for Wikipedia Zero [otto] (Ongoing)
 * Filter by X-Carrier headers. [otto + asher + diederik] (Jan) [Waiting for Fundraiser to finish]'
 * udp-filter to filter by http status. [otto] (Jan) [Waiting for Fundraiser to finish]

WikiStats

 * Create new mobile pageview report [stefan + dvl]
 * Fix bugs in wikistats [stefan + erik +dvl] ONGOING

Infrastructure

 * Access/support requests for stat1, stat1001 [otto] (Ongoing)
 * Maintenance of oxygen/emery/locke [otto] (Ongoing)
 * Migrate Reportcard off Labs onto stat1001 -- reportcard.wikimedia.org [otto + dsc] (Jan)

Data Releases

 * Blogpost about what awesome stuff you can do with the AWS datasets [diederik] (Jan) (Ongoing)
 * Deploy alpha of prototype pageview metrics API on stat1001 [diederik/dario/andrew] (Jan)
 * Pageview Metrics API work [Andrew] - DONE
 * Create Data Release Practices Task Force [diederik/dario -- waiting for feedback from Dario] (Jan)
 * Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dan] (Ongoing)
 * Get Amazon to take back their instance! [diederik]

Engineering Community Team

 * Bug Wrangling [Andre + volunteers]
 * working especially on reducing backlog of unprioritized issues
 * mentoring Valerie in bug triage
 * Mentorship programs - ongoing [Quim, Sumana, Andre]
 * Outreach Program for Women kickoff -- DONE
 * First bug day, upcoming in January [Andre, Valerie]
 * Creation of useful list of MediaWiki enterprise users/vendors: in progress, target ~Jan 11 [Mariya, Sumana]
 * LevelUp kickoff
 * Matchmaking target: Jan 11
 * Volunteer coordination and outreach
 * Technical communications [Guillaume]
 * Kickoff of first tech collaboration experiment: more engagement in Tech/Ambassadors
 * Misc Engineering community proposals' followup [Sumana]
 * Code of Conduct [Sumana] -- in process - target 11 Jan

Features

 * VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
 * experimentation into C/C++ version
 * Echo_(Notifications)
 * Global preferences for echo delivery [Deferred to 2013]
 * centralization and cross wiki notification delivery [Deferred to 2013]
 * first experimental release on the English Wikipedia (February 2013)
 * Article feedback (Matthias)
 * Full deployment to English Wikipedia [2013]

MediaWiki Core

 * Git/Conversion (Chad)
 * MediaWiki 1.21/Roadmap
 * MediaWiki 1.21/wmf9
 * MediaWiki 1.21/wmf10
 * Admin tools development [Chris]
 * Lua scripting [Tim, Chad, Brad]
 * Install Lua (Scribunto) on other Wikimedia wikis ( see Goals)
 * Detailed deployment plan [TBD] [volunteer product management task shepherded by Sumana]
 * Wikidata client deployments (wikis TBD)

Multimedia

 * TimedMediaHandler [Jan Gerber]
 * Ramping up new engineers

QA

 * Ongoing test automation coverage [Chris, Zeljko]
 * Echo testing [Chris, Quim, Fabrice]
 * Visual Editor testing (possible) [James F., Chris]
 * More community testing! (format TBA) [Quim, Chris]

Analytics
See also: Analytics/Roadmap

Page views

 * Add a new report to stats.wikimedia.org that replaces the report at http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm and uses the squid log data files as input. [Stefan + Dvl]
 * Migrate mobile stats reporting to Kraken. [Otto + Diederik + (Stefan)]
 * Create monthly mobile target pageview numbers to use in the ReportCard. [Howie + Diederik]

Funnel Analysis

 * Simple funnel analysis tool [Diederik + Dan]
 * Automate funnel analysis results distribution [Diederik + Otto]

Tracking usage alpha and beta mobile site

 * Provide storage of this data in Kraken. [Otto + Dsc]
 * Convert EventLogging data to Avro for use in Kraken (using EventLogging schemas) [Dsc]
 * Provide raw data to data analysts in timely and easily accessible manner. [Otto + Dvl + Dsc]

Wikipedia Zero

 * Automatic and recurring (hourly / daily / weekly) reporting of pageviews per carrier. [Otto + Evan + Diederik]
 * Provide pageview counts which accurately reflect requests from Opera browsers. [Otto + Diederik]
 * Deploy new Wikipedia Zero filters on per request basis.[Otto]

Engineering Community Team

 * Bug Wrangling [Andre + volunteers]
 * reducing backlog of unprioritized issues
 * improving Bugzilla application
 * mentoring Valerie in bug triage
 * Mentorship programs - ongoing [Quim, Sumana, Andre]
 * Outreach Program for Women
 * Bug Days (triage) [Andre, Valerie]
 * Outreach to enterprise users & vendors [Mariya, Sumana]
 * Product management re data dumps (or other target product TBA) [Mariya, Sumana]
 * LevelUp metamentoring -- Sumana, ongoing
 * Volunteer coordination and outreach
 * QA outreach & coordination [Quim]
 * User group coordination [Quim]
 * Technical communications [Guillaume]
 * Followup on first tech collaboration experiment: more engagement in Tech/Ambassadors

Parking lot and todos

 * Configuration management overhaul [Chad]
 * Fundraising flesh-out
 * HipHop deployment (deferred because Facebook has HipHopVM)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * BlameMaps (WMDE/Luca de Alfaro; see proof of concept)
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Disk-backed object cache
 * Relicense our Lucene contributions [future Lucene search engineer]
 * OAuth [ChrisS]
 * related to potential API redesign (Wikia)
 * Changes to visual design to make ACW consistent with Design Toolkit [TBD]
 * Features parking
 * Queuing infrastructure work for Echo (probably using Aarons JobQueue abstraction)
 * Article creation workflow deferred
 * Gadgets 2.0?
 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Git/Gerrit
 * Have Lucene work underway upstream for 2.6 release cycle [Chad]

Archive
Previous months:
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March
 * /2012/April
 * /2012/May
 * /2012/June
 * /2012/July
 * /2012/August
 * /2012/September
 * /2012/October
 * /2012/November
 * /2012/December