Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation.

See also:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF  goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software

Previous months:
 * /2011/September
 * /2011/October
 * /2011/November
 * /2011/December
 * /2012/January

Item syntax:
 * Project name
 * [Qualifier keyword] Action item [Owner(s)] [Completion keyword]

Examples:
 * New unicorn engagement
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Operations

 * Data Centers
 * Text Squid deployment @ Eqiad (100% of traffic) [DONE]
 * API Squid deployment @ EQIAD [DONE]
 * EQIAD datacenter new row of racks [ordered, in progress]
 * db refresh (s1 to s7) [Asher] [DONE]
 * Test effectiveness of Flash Drive [Asher]
 * Set up new server for Torrus (Manutius) [Mark] [DONE]
 * prep Stat1 to replace Bayes [wip]


 * Swift
 * Thumbnail server replacement - Back-end (ms5) [Ben]
 * wikitech:Swift/Deploy Plan - Thumbnails
 * 100% thumbs deployment [DONE]


 * Wikimedia Labs
 * Deploy Labs Gluster File storage cluster [Ryan/Sara]
 * Labs features rollout
 * Build out LVS in Labs [Ryan/Mark]
 * Create squid instance in Labs [Ryan/Sara]
 * Add resilience to Lab Controller service by building redundant Controller services in EQIAD which includes DNS, LDAP, MySQL [Sara]


 * HTTPS
 * Move internal wikis to HTTPS-only
 * OfficeWiki [DONE]
 * less Httpsless domains [DanielZ]


 * Data Dumps
 * Set up XML Dump infrastructure @ EQIAD [Ariel]
 * Dataset1001 [DONE]
 * Snapshot[1001..1004] [DONE]
 * Put Dataset1001 into production

Editing tools

 * Visual editor
 * Editable Surface feature additions ongoing
 * First iteration of minimal round trip functionality [2/2012]
 * Content Editable feature parity with ES feature set [2/2012]
 * IME functionality development and integration [2/2012]


 * Parsoid


 * Internationalization and localization tools
 * Additional translate extension improvements [2/2012]

Participation and editor engagement

 * Article feedback
 * Decide on "winner": Preliminary discussion with community [DONE]
 * Design and development of first version of Feedback Page, using data from test versions of Feedback Form [DONE]
 * Additional features: hiding/oversight; dismiss in UI [2/22]


 * Feedback Dashboard
 * Feature Evaluation [Dario]
 * Concurrency notification: Delayed, pending release of ACW


 * Article Creation Workflow
 * Article Creation Interstitial:
 * Deploy first release to labs/prototype [2/10] [DONE]
 * Deployment of A/B of new version [tentative: 2/22]
 * New Page Triage: Scrum training with Thoughtworks

Multimedia tools

 * UploadWizard
 * Enable chunked uploading support


 * TimedMediaHandler
 * deployment (after Rel 1.19 is released)

Feature support

 * Wikipedia Education Program

Reader tools

 * Personal image filter
 * development presumed to begin, may be done via vendor development. Support third party block lists instead of categories?

Mobile
(heavy traveling for mobile team) [DONE]


 * MobileFrontend
 * Full screen search to production [holding]
 * Product plan (UI, use cases) for Mobile Uploads: Initial workflow
 * Product plan (UI, use cases) for Mobile Nav: Enumerate use cases, develop first pass of navigational constructs
 * Login, User pref, etc (features TBD)
 * UI mockups for full screen search [DONE]
 * UI mockups for opt in beta [DONE]
 * MobileFrontend Refactor
 * De-Wikimedify MobileFrontend [Arthur] [CODING DONE]
 * Cleanup of JS/CSS in MF [Jon] [CODING DONE]


 * Wikipedia Android app
 * V1.1beta1 [DONE]
 * V1.1beta2 [IMMINENT]


 * Wikipedia iOS app
 * V3.0beta1 [IMMINENT]


 * WikipediaZero
 * [TESTING]


 * Wikipedia over SMS/USSD
 * [LABS TESTING]


 * GPS Storage/Retrieval
 * [TESTING]

Special projects

 * Kiwix on Android
 * http://sourceforge.net/mailarchive/forum.php?thread_name=4F426983.2080908%40engelhart.org&forum_name=kiwix-developer

MediaWiki Core

 * MediaWiki 1.19/Roadmap
 * Finish code review for 1.19 [DONE]
 * Cut 1.19 release branch [DONE]
 * 1.19 deploy (from SVN)
 * Postmortem 1.19 after the 1.19 deployment [RobLa]


 * Git/Conversion
 * Final test conversion of trunk + extensions
 * Documentation tasks (blog post on Git)
 * Finish up specific Git management scripts / changes (staging on Fenari (bug 34138), i18n (bug 34137))
 * Make Gerrit behave like we want it to (hooks, permissions, gitweb redirects)


 * SwiftMedia
 * 100% deployment for thumbnails [Aaron/Ben/Mark B] [IN PROGRESS]
 * BEFORE 1.19 deployment.
 * Thumbnail response to end-users will work by hitting squids, falling back to swift, falling back to ms5, falling back to image scalers.
 * Thumbnail purging will work as it does now except swift will be purged via a MediaWiki hook.


 * Lua scripting
 * Design work begins - significant work won't begin until March

Analytics

 * Analytics/Reportcard
 * Prototype first release: late February; Scope TBD, but will probably include active editors data, some page view data


 * Log collection pipeline
 * (udp2log) Deploy new consolidated filtering code for test filters to emery [Diederik/Tim/Roan] [DONE]
 * (mobile) refactor Nimish's code, incorporate fixes causing discrepancy between Nimish and Erik's code [Diederik/Andre]

TL;DR

 * Wikimedia blog maintenance
 * Rewrite the layout so it can be centered [Guillaume] [In progress]
 * Deploy recent Wikimedia-specific theme and plugin updates (RT# created) [Ops]
 * Upgrade core to WordPress 3.3.1 and third-party plugins to their latest version (RT# created) [Ops]

Operations

 * Data Centers
 * upload.wikimedia.org@eqiad using varnish (replacing squid)
 * ESAM router replacement (from Foundary to Juniper)
 * /home move to NetApp [Mark] [in progress]
 * Lucene Search deployment@EQIAD
 * New Nagios server@EQIAD


 * Swift
 * Add more SWIFT nodes for thumbnails
 * Prep for SWIFT to replace 'Originals' (ms7)


 * Wikimedia Labs


 * HTTPS


 * Data Dumps

Editing tools

 * Visual editor


 * Parsoid


 * Internationalization and localization tools
 * Start work on universal language selection UI

Participation and editor engagement

 * Article feedback
 * Noise reduction/management for Feedback Page
 * Potential deployment of AFT to broader list of articles on enwp


 * Feedback Dashboard


 * Article Creation Workflow
 * First deployment of revised workflow for page triage

Multimedia tools

 * UploadWizard


 * TimedMediaHandler

MediaWiki infrastructure

 * ResourceLoader
 * Gadgets 2.0 deployed after MW 1.19 (dependent on REL1_19 schedule) [in progress]

Feature support

 * Wikipedia Education Program

Reader tools

 * Personal image filter

Mobile

 * MobileFrontend


 * Wikipedia Android app
 * Release v2.0
 * Add Photo Upload
 * Add Nav Menu
 * Add AFT support


 * WikipediaZero


 * Wikipedia over SMS/USSD


 * GPS Storage/Retrieval


 * Wikipedia iOS app
 * Release v2.0

Special projects

 * 2011 Wikimedia fundraiser


 * Kiwix on Android

MediaWiki Core

 * MediaWiki 1.19/Roadmap


 * Continuous integration
 * Begin developing framework for integration testing [Antoine, Chris McMahon]
 * possibly Selenium


 * Git/Conversion
 * First release from git mainline development branch
 * SVN import, post-release/deployment make read-only switch


 * SwiftMedia
 * originals work


 * Lua scripting


 * Interwiki transclusion
 * ... [Sam(, Roan?)]


 * IPv6 support
 * Prepare for IPv6 day (June 6th) [Aaron,Mark?]
 * rc_ip schema change to be lumped into 1.19 schema changes


 * Disk-backed object cache
 * DB sharding


 * Superprotection (notes to be copied to wiki somewhere by RobLa or Tim) [Sam?]

Analytics

 * Analytics/Reportcard


 * Log collection pipeline

TL;DR

 * Wikimedia blog maintenance


 * Wikimedia operations history and architecture document
 * Start collecting information [Guillaume]


 * Wikimedia server error page
 * (currently "Wikimedia Foundation error") Decide on the format (internal project vs. contest) & timeline [Guillaume]

Operations

 * Data Centers
 * Convert Upload cache from using squid to varnish
 * upload.wikimedia.org @ EQIAD
 * Fundraising infrastructure buildup@ EQIAD
 * own firewall
 * own payment clusters
 * OTRS Upgrade
 * db9/db10 upgrade (db for etherpad, RT, Bugzilla, ..)
 * Apache & Memcache Deployment @ EQIAD
 * Phase 1
 * new UDP2log infrastructure @ EQIAD


 * Swift
 * Prep Swift infrastructure to be ready for 'Originals' (Commons) deployment


 * Wikimedia Labs


 * HTTPS


 * Data Dumps
 * EQIAD test/parallel runs
 * Test suites for XML Dumps
 * own MQ server

Editing tools

 * Visual editor
 * Small scale production/default deployment


 * Parsoid


 * Internationalization and localization tools

Participation and editor engagement

 * Article feedback


 * Feedback Dashboard


 * Article Creation Workflow

Multimedia tools

 * UploadWizard


 * TimedMediaHandler

MediaWiki infrastructure

 * Gadgets 3.0
 * Includes review of Salvatore's "Gadgets user preferences" work

Feature support

 * Wikipedia Education Program

Reader tools

 * Personal image filter

Mobile

 * MobileFrontend


 * Wikipedia Android app


 * WikipediaZero


 * Wikipedia over SMS/USSD


 * GPS Storage/Retrieval


 * Wikipedia iOS app

Special projects

 * 2011 Wikimedia fundraiser


 * Kiwix

MediaWiki Core

 * MediaWiki 1.19/Roadmap


 * Continuous integration


 * Git/Conversion


 * SwiftMedia


 * Lua scripting


 * Interwiki transclusion


 * IPv6 support
 * Prepare for IPv6 day (June 6th) [Aaron,Mark?]


 * Disk-backed object cache


 * Superprotection (notes to be copied to wiki somewhere by RobLa or Tim) [Sam?]

Analytics

 * Analytics/Reportcard


 * Log collection pipeline

TL;DR

 * Wikimedia blog maintenance


 * Wikimedia operations history and architecture document


 * Wikimedia server error page

May 2012

 * General engineering process checkin (Berlin?)
 * Config backend [Chad]
 * Evaluate WikiFactory from Wikia

Mobile / Offline

 * First MobileFrontend feature related to mobile uploads (e.g. media upload beta).

Platform

 * Prepare IPv6 support for IPv6 day (June 6th)
 * Revisit Gerrit, git-review, and other code review/workflow tools (3 months after the git switch) to see whether to switch to Phabricator or another code review tool

Mobile / Offline

 * USSD gateway rollout
 * SMS gateway rollout
 * Varnish as default cache for Mobile
 * New article creation
 * Watchlists
 * Diffs
 * Search improvements
 * Kiwix on Android
 * Partner integration (various)
 * Lazy loading articles
 * reduce bandwidth requirements for some users
 * WikipediaZero

Features

 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Re-energize LiquidThreads (rename it?!) (may need refactoring for notification streams)
 * Respond to open design questions
 * Photo/media moderation tools; additional patrolling and moderation tooling
 * Help features
 * Mentoring features
 * Other real-time features, chat
 * Dependent on MediaWiki 1.20 deployment date, ResourceLoader 2 -- mostly the Gadgets-extension work which is (as of Feb 2012) in a branch.
 * Gadgets 2.0?

Platform / Dev-Ops

 * Extension management -- depends on config mgmt revamp

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)
 * Labs maintenance
 * Test infrastructure maintenance

Parking lot and todos

 * Details of git migration (extension mgmt, permissions, code review tool, Gerrit-Jenkins integration, etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf Wikimedia Platform Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles


 * Database-backed session handling
 * HipHop deployment (deferred because Facebook has a different shiny new thing)
 * Extension:Configure (merge with config management?)
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * audit users of OAIPMH and check how search indexing uses it
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)
 * Wikidata (WMDE)
 * BlameMaps (WMDE)
 * graphical UI on top of improved backend for configuration management
 * Various HTTPS support fixes