Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation. See also: Item syntax: Examples:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance
 * Recruitment and encouragement of volunteers
 * Monthly report and other engineering communications

Features

 * VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
 * Initial Parsoid architectural review (parser team, tim) [DONE]
 * IE in VE [IN PROGRESS]
 * bug fixing on Visual Editor [ONGOING]
 * limited production release on French, Hebrew Wikipedia [IN PROGRESS]
 * 5-month plan for VE [late Jan]
 * Node.js parsoid to completion [ONGOING]
 * [Parsoid testing framework for semantic/superficial regressions|http://parsoid.wmflabs.org:8001] [IN PROCESS]
 * i18n support in Parsoid (page links, etc) [IN PROCESS]
 * assessment and timeline for next steps in Parsoid [IN PROGRESS]


 * Echo_(Notifications) (Kaldari, Benny, Werdna, Luke Welling, Nischay, Krenair) (roadmap)
 * Continued development of prototype on Mediawiki.org [ONGOING]
 * UI and bugfixing of Echo [ONGOING]
 * HTML e-mail (Luke) [IN PROCESS]
 * Flyout + archive refactor [IN PROCESS]
 * New features on MediaWiki.org: new editor notifications, deduping logic [Jan 28]
 * Bundling "[IN PROCESS]''
 * Dismiss and preferences "[IN PROCESS]''
 * i18n issues on Echo [ONGOING meetings after Jan 18]
 * JobQueue abstraction (zmq) [Luke] [IN PROCESS]
 * JobQueue abstraction (Redis) [Luke] [IN PROCESS]
 * Jan 30 community testing event [Quim, Chris McMahon] [IN PROCESS]


 * Flow (Werdna, JamesF, Brandon, Matthias, LukeW, Isarra)
 * Kickoff [DONE]
 * reach out to Wikia [Terry, JamesF] (Jan 17) [DONE]
 * Global user changes [JamesF] [IN PROCESS]
 * investigation into ContentHandler support [LukeW, Matthias]
 * look into Wikia's implemenbtation of MessageWall [Werdna]


 * Article feedback (Matthias)
 * performance/db related changes to AFTv5 queries (Matthias) [DONE]
 * discussion on DataModel wrapper (Asher, LukeW, Matthias, S) [Jan 25]
 * code review of AFTv5 (S Page, LukeW) [IN PROCESS]
 * i18n messaging update [IN PROCESS]
 * Other minor changes prior to full English Wikipedia deployment (Matthias) [IN PROCESS]
 * refactored deployment on enwiki + dewiki pilot (may involve db pruning) [Feb]
 * vertically partitioned database deploy [Feb/March]


 * New Pages Feed (fka Page Triage) (Kaldari, Benny, Nischay)
 * Bug fixes (no more feature development) [ONGOING]


 * E3 (Ori, S, MattF, Luke)


 * Clicktracking/Analytics
 * Funnel tracking of Community Portal [ONGOING]
 * Funnel tracking of Onboarding [IN PROCESS]
 * data/cohort API prototype (Dario,RyanF) "[DONE]"
 * data/cohort API further work (Dario,RyanF,Dan) [ONGOING]


 * Account Creation User Experience
 * Transition to EventLogging (off of ClickTracking) incl. permanent campaign support [IN PROCESS]
 * Productization of ACUX [IN PROCESS]


 * Onboarding
 * Javascript bugs in guided tour [DONE]
 * get Guided Tour ready for deployment incl. creating onboarding tour [Jan 17]
 * version two of Special:GettingStarted [IN PROCESS]
 * tagging edits made through onboarding [DONE]


 * Other Account Creation Funnels
 * 22k Donor Survey email [DONE]
 * Contribution banner [DONE]
 * Donor Thank You email [DONE]
 * CAPTCHA testing [IN PROCESS]


 * MicroDesign
 * other edit window templates [ON HOLD]
 * (see Agora)

Technical Operations

 * Datacenter migration (eqiad) - "[done]"
 * Updating the scap deploy scripts to deal with multi-datacenter [ done]
 * deploy redis and memcached@eqiad [done]
 * deploy parser cache@eqiad [done]
 * deploy apache app servers@eqiad [done]
 * deploy API servers@eqiad [done]
 * deploy jobrunner servers@eqiad [done]
 * deploy imagescalers [done]
 * database failover from Tampa to Eqiad [done]
 * deploy parsoid servers@eqiad [done]
 * deploy tmh (time media handler] @eqiad [done]
 * setting up & testing Ceph@eqiad [wip]
 * deploy sharded parser cache @ tampa - pc2 & pc3 [done]

Mobile

 * Quarterly planning session with team: focus on photo uploads (e.g., commons app, productizing mobile photo uploads) [DONE]
 * Getting MobileFrontend onto beta labs
 * Mobile Web
 * Beta
 * Photo Upload bug fixes [DONE]
 * Watchlist (reader + recent changes views) [DONE]
 * Thumbnail A-Z view [DONE]
 * Watchlist CTA [DONE]
 * Account Creation [DONE]
 * Login [DONE]
 * Mobile diff [DONE]
 * History
 * Production [IN PROCESS]
 * Watchlist
 * Login
 * Mobile diff
 * Account Creation
 * Kill section 0 [DONE]
 * Use full resource loader in production [DONE]
 * Better resource loader support
 * Solr Data replication [DONE]
 * Solr Monitoring

MediaWiki Core

 * Datacenter migration (see TechOps section above) [Tim, Aaron, Sam, ChrisS, Brad, Antoine]  39106
 * git-deploy migration (43338) - postponed
 * file path issues for eqiad migration (43340) [Brad, Antoine, Sam, ChrisS] - done
 * Remove production MediaWiki dependency on NFS (43495) [Aaron, Tim] - done
 * Git/Conversion (Chad)
 * LDAP fixes for our Gerrit use (current patch) [Chad]
 * Upgrade to 2.6 [Chad]
 * Replace gitweb with GitBlit [Chad]
 * MediaWiki 1.21/Roadmap
 * MediaWiki 1.21/wmf7 - done
 * MediaWiki 1.21/wmf8 - ongoing
 * Admin tools development [Chris]
 * Build and deploy Global RenameUser extension [CSteipp]
 * Lua scripting [Tim, Chad, Brad]
 * Language library [Tim] [DONE]
 * Unicode library [Brad] [DONE]
 * Search
 * Onboarding Ram - ongoing
 * Wikidata client deployment to Hungarian Wikipedia
 * Jan 14: initial rollout - done

Multimedia

 * TimedMediaHandler
 * Bugfixing (Jan)

QA

 * QA
 * Datacenter migration, Echo
 * See also QA/Roadmap
 * Browser testing
 * Datacenter migration automation: IN PROCESS
 * Beta labs (beta cluster)
 * Add specific reviewer in Gerrit on SQL changes [IN PROCESS] [Antoine]
 * Continuous integration

Analytics
See also: Analytics/Roadmap

Kraken

 * Storm pom [dsc]
 * Storm ETL
 * Get Storm set up [dsc + otto] (Dec) [DONE]
 * Storm ETL bolt for GeoIP [dsc + otto]
 * Storm ETL bolt for HDFS Import [dsc + otto]
 * HDFS Import enforces product-code directories and permissions [dsc + otto] (Jan)
 * Storm ETL bolt for Anonymization [dsc + otto]
 * Storm ETL bolt for Kafka Checkpointing [dsc + otto] (Jan)
 * Puppetize Kraken [otto] (Ongoing)
 * Create oozie workflows for pageview {Wikipedia Zero / Mobile} job counts [diederik + otto]
 * Finish setting up nexus repository [diederik + otto]
 * Collect user requirements for funnel analysis [diederik]
 * Create funnel analysis tool [diederik + dan]
 * Create debian package for dClass [stefan + dvl ]  Debian package available here
 * Creating a JNI wrapper for dClass. The aim is to do device detection in Java and Apache Pig[stefan + dvl ] [JNI wrapper available here https://github.com/wikimedia/kraken/tree/master/src/main/java/org/wikimedia/analytics/dclassjni]

Limn

 * Refactor charting to use d3 [dan + dsc] (Jan)
 * Feature Parity with Dygraphs (plus bugfixes, etc) (Jan)
 * Bugfixes (like Save-As, UI Error Notifications, ...) [dan] (Jan)
 * Migrate Dario's dashboards to Limn [dan] (Jan)
 * Improve Limn wiki, docs, & guides (esp those shameful screenshots) [dan] (Jan)
 * Coke ( for Coco) task to create symlinks into   from an existing data repository (such as, say, analytics/reportcard/data) [dan] (Jan)
 * Coke task to download and setup dummy testing data for ease of development [dsc] (Jan)
 * UI support for remote datasets via proxy [dsc + dan] (Jan)
 * Deploy reportcard / gerrit-stats on stat1001 (aka, "the Debian packaging discussion") [dvl + dan] (Jan)
 * Support the Global Dev dashboard [evan] (Ongoing)
 * Support the Gerrit Stats dashboard [diederik] (Ongoing)

Legacy Log Collection

 * udp2log filters
 * Update filters for Wikipedia Zero [otto] (Ongoing)
 * Filter by X-Carrier headers. [otto + asher + diederik] (Jan) [Waiting for Fundraiser to finish]'
 * udp-filter to filter by http status. [otto] (Jan) [Waiting for Fundraiser to finish]

WikiStats
( Still need to implement part of the pageview count logic, afterwards will be waiting for the Mobile Team to provide us with a way of deduplicating pageviews)
 * Create new mobile pageview report [stefan + dvl ].
 * Module to efficiently parse and filter log entries
 * Fix bugs in wikistats [stefan + erik +dvl] ONGOING

Infrastructure

 * Access/support requests for stat1, stat1001 [otto] (Ongoing)
 * Maintenance of oxygen/emery/locke [otto] (Ongoing)
 * Migrate Reportcard off Labs onto stat1001 -- reportcard.wikimedia.org [otto + dsc] (Jan)

Data Releases

 * Blogpost about what awesome stuff you can do with the AWS datasets [diederik] (Jan) (Ongoing)
 * Deploy alpha of prototype pageview metrics API on stat1001 [diederik/dario/andrew] (Jan)
 * Pageview Metrics API work [Andrew] - DONE
 * Create Data Release Practices Task Force [diederik/dario -- waiting for feedback from Dario] (Jan)
 * Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dan] (Ongoing)
 * Get Amazon to take back their instance! [diederik]

Engineering Community Team

 * Bug Wrangling [Andre + volunteers]
 * working especially on reducing backlog of unprioritized issues
 * mentoring Valerie in bug triage
 * Mentorship programs - ongoing [Quim, Sumana, Andre]
 * Outreach Program for Women kickoff -- DONE
 * First bug day, upcoming in January [Andre, Valerie]
 * Creation of useful list of MediaWiki enterprise users/vendors: DONE [Mariya, Sumana]
 * LevelUp kickoff
 * Matchmaking target: Jan 11 -- in process [Sumana]
 * Volunteer coordination and outreach
 * Volunteer product management program
 * Discuss and agree on the direction and implementation of the program [ECT, led by Guillaume] DONE
 * Consolidate announcements for events, meetings, IRC office hours, etc. (i.e. all events related to volunteer involvement) into a central Calendar page [Guillaume] DONE
 * Start a discussion with the Product team to enlist their help in mentoring volunteers (possibly as part of LevelUp) [Guillaume]
 * Technical communications
 * Assemble and publish the Wikimedia engineering report for December 2012 [Guillaume] DONE
 * Prepare and execute communications around the data center migration [Guillaume] DONE
 * Tech ambassadors program
 * Finish to write up Tech/Ambassadors [Guillaume] DONE
 * Kick off the program [Guillaume] DONE
 * Use the data center migration to advertise the ambassadors program [Guillaume] DONE
 * Misc Engineering community proposals' followup [Sumana]
 * Code of Conduct [Sumana] -- in process - target 31 Jan

Language Engineering (i18n/l10n)

 * Translate UI (TUX) enhancements: Development in progress [Niklas, Santhosh, Amir, Pau]
 * Language Selector API
 * Translate UI/MT integration of Translate Aids
 * Solr engine setup and configuration for translate: In progress [Niklas]
 * Translate UI UX testing with experts and users: Completed [Pau, Red Hat L10n team, translator community]
 * Publish UX Testing results and feedback: Completed [Pau]
 * Central Notice/Translate integration: Deployment in progress [Siebrand, Niklas, FR team]
 * Adam from FR team has merged the CentralNotice/Translate Extension changes into master. These changes will be going live on production during the next scheduled deployment (Feb 6 is the likely date for it to hit meta).
 * Universal Language Selector (ULS)
 * Feature complete [Done]
 * Deployment to Wikimedia sites: On hold until site caching architecture change occurs
 * Milkshake
 * jQuery.IME - added more input methods - continues [Santhosh, Amir, community]
 * jQuery.ULS - minor bug fixes [Amir, Santhosh]
 * Open Source language tools development collaboration
 * Work with Red Hat i18n team on i18n tools evaluation, IME integration into jQuery.IME - Completed[Alolita]
 * Work with Google i18n team on Webfonts integration: In progress [Alolita]
 * Work sprint planning: In progress [Alolita]
 * Standards
 * CLDR, Unicode discussions participation [Santhosh, Amir, Alolita]
 * Communications
 * Blog posts on sprint progress, latest features for completed sprint - In progress[Amir]
 * i18n CR, Bugfixes, Maintenance - in progress [All]

Features

 * VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
 * experimentation into C/C++ version
 * Echo_(Notifications)
 * Global preferences for echo delivery [Deferred to 2013]
 * centralization and cross wiki notification delivery [Deferred to 2013]
 * first experimental release on the English Wikipedia (February 2013)
 * Article feedback (Matthias)
 * Full deployment to English Wikipedia [2013]

Technical Operations

 * Deploy Ceph@Eqiad
 * Deploy 'test / beta 'OTRS
 * Labs database @ tampa

MediaWiki Core

 * git-deploy migration (43338)
 * Git/Conversion (Chad)
 * MediaWiki 1.21/Roadmap
 * MediaWiki 1.21/wmf9
 * MediaWiki 1.21/wmf10
 * Admin tools development [Chris]
 * Lua scripting [Tim, Chad, Brad]
 * Install Lua (Scribunto) on other Wikimedia wikis ( see Goals)
 * Detailed deployment plan [TBD] [volunteer product management task shepherded by Sumana]
 * Wikidata client deployments (wikis TBD)

Multimedia

 * TimedMediaHandler [Jan Gerber]
 * Ramping up new engineers

QA

 * Ongoing test automation coverage [Chris, Zeljko]
 * Echo testing [Chris, Quim, Fabrice]
 * Visual Editor testing (possible) [James F., Chris]
 * More community testing! (format TBA) [Quim, Chris]

Analytics
See also: Analytics/Roadmap

Page views

 * Add a new report to stats.wikimedia.org that replaces the report at http://stats.wikimedia.org/EN/TablesPageViewsMonthlyMobile.htm and uses the squid log data files as input. [Stefan + Dvl]
 * Migrate mobile stats reporting to Kraken. [Otto + Diederik + (Stefan)]
 * Create monthly mobile target pageview numbers to use in the ReportCard. [Howie + Diederik]

Funnel Analysis

 * Simple funnel analysis tool [Diederik + Dan]
 * Automate funnel analysis results distribution [Diederik + Otto]

Tracking usage alpha and beta mobile site

 * Provide storage of this data in Kraken. [Otto + Dsc]
 * Convert EventLogging data to Avro for use in Kraken (using EventLogging schemas) [Dsc]
 * Provide raw data to data analysts in timely and easily accessible manner. [Otto + Dvl + Dsc]

Wikipedia Zero

 * Automatic and recurring (hourly / daily / weekly) reporting of pageviews per carrier. [Otto + Evan + Diederik]
 * Provide pageview counts which accurately reflect requests from Opera browsers. [Otto + Diederik]
 * Deploy new Wikipedia Zero filters on per request basis.[Otto]

Engineering Community Team

 * Bug Wrangling [Andre + volunteers]
 * reducing backlog of unprioritized issues
 * improving Bugzilla application
 * mentoring Valerie in bug triage
 * Mentorship programs - ongoing [Quim, Sumana, Andre]
 * Outreach Program for Women
 * Bug Days (triage) [Andre, Valerie]
 * Outreach to enterprise users & vendors [Mariya, Sumana]
 * Product management re data dumps (or other target product TBA) [Mariya, Sumana]
 * LevelUp metamentoring -- Sumana, ongoing
 * Volunteer coordination and outreach
 * QA outreach & coordination [Quim]
 * User group coordination [Quim]
 * Volunteer product management program
 * Create a landing page for Product management and link to it from How to contribute [Guillaume & Quim, in collaboration with Product team]
 * Figure out a better, less corporate-jargon-y way of explaining "product management" activities [Guillaume]
 * Create a list of activities/projects where PM resources are lacking [Guillaume] [in progress]
 * Create, and later help maintain, a list of tasks or microtasks to perform by interested volunteers, with clear and direct calls to action [Guillaume & Quim, in collaboration with Product team]
 * Technical communications
 * Assemble and publish the Wikimedia engineering report for January 2013 [Guillaume]
 * Tech ambassadors program
 * Set up collaboration with Signost & other volunteers to join efforts in identifying deployment changes likely to impact editors and communicating about them across wikis [Guillaume]
 * Expand the Glossary [everyone, led by Guillaume]
 * Set up a process to translate the Glossary [Guillaume]

Language Engineering (i18n/l10n)

 * Translate UI enhancements: Development in progress [Niklas, Santhosh, Amir]
 * Language Selector API
 * Translate UI/MT integration
 * Solr engine schema tuning
 * Integration of Translate Aids (part of translate editor)
 * Translate UI UX testing with experts and users - Iterative, In progress [Pau, Red Hat L10n team, community]
 * ULS
 * Feature complete [Done]
 * Deployment to Wikimedia sites - on hold until site caching architecture change occurs
 * Milkshake
 * jQuery.IME - added more input methods - continues [Santhosh, Amir]
 * jQuery.IME port to Java for mobile - done [Yuvi, Santhosh]
 * Open Source project collaboration
 * Work with Red Hat i18n team on i18n tools evaluation, IME integration into jQuery.IME [Alolita]
 * Work with Google i18n team on Webfonts integration [Alolita]
 * W3C Standards
 * CLDR, Unicode discussions participation [Santhosh, Amir, Alolita]
 * Communications
 * Blog posts on sprint progress, latest features - done [Amir]
 * i18n CR, Bugfixes, Maintenance - in progress [All]

Parking lot and todos

 * Configuration management overhaul [Chad]
 * Fundraising flesh-out
 * HipHop deployment (deferred because Facebook has HipHopVM)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * BlameMaps (WMDE/Luca de Alfaro; see proof of concept)
 * Various HTTPS support fixes
 * Wikimedia server error page
 * Disk-backed object cache
 * Relicense our Lucene contributions [future Lucene search engineer]
 * OAuth [ChrisS]
 * related to potential API redesign (Wikia)
 * Changes to visual design to make ACW consistent with Design Toolkit [TBD]
 * Features parking
 * Queuing infrastructure work for Echo (probably using Aarons JobQueue abstraction)
 * Article creation workflow deferred
 * Gadgets 2.0?
 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Git/Gerrit
 * Have Lucene work underway upstream for 2.6 release cycle [Chad]

Archive
Previous months:
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January
 * /2012/February
 * /2012/March
 * /2012/April
 * /2012/May
 * /2012/June
 * /2012/July
 * /2012/August
 * /2012/September
 * /2012/October
 * /2012/November
 * /2012/December

zh:Template:Percentage bar