Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation.

See also:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * Software deployments schedule

Previous months:
 * /2011/September
 * /2011/October
 * /2011/November

Item syntax: Examples:
 * Action item
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Platform / Dev-Ops

 * Continuous integration improvements
 * TestSwarm integration [Antoine, Timo, Chad]
 * Move towards continuous integration via git, goalpost: weekly deployment


 * Jenkins (Testswarm/PHPUnit tests) on git branches


 * Git conversion
 * prelim test conversions early in month [Chad/Brion] [DONE]
 * Git workflow architecture review [Chad/Brion]
 * Agree on implementation strategies regarding remaining development process questions, e.g. how to handle multi-repo commits


 * Code review management
 * Under 500 revs to code review on trunk


 * Real-time visualization of all MediaWiki profiler data [Asher]
 * include visualization timepoints of all code deploys [DONE]
 * announce and communicate about new visualization tools


 * Engineering project documentation
 * Publish December report [Guillaume]


 * Wikimedia blog maintenance
 * Implement Wikimedia blog theme & features tweaks [Guillaume] [partial]


 * Swift
 * Initial performance tests
 * Purchase additional thumbs storage hardware [Ben/Rob]
 * Performance Stress Tests

http://wikitech.wikimedia.org/view/Swift/Tasks#Overall_Schedule
 * Thumbnail deployment [Aaron/Ben/Mark B]
 * Merge FileBackend refactoring work to trunk [Aaron]

Mobile / Offline

 * Feature iteration
 * User login [DONE]
 * JavaScript tiering/ MW Views [DONE]
 * Small bug fixes
 * Unlocking viewport [DONE]
 * Updated image description pages '[DONE]
 * HTTPS for Wikipedia Mobile [DONE]
 * WikipediaZero features [in progress, escalating priority]
 * Wikimedia Mobile (Android) RC1/2/3 [RC1 - done]
 * Daily Featured Articles RSS [12/19 Max starts]
 * Begin implementation of GPS Storage/Retrieval
 * Other features started at India Hackathon, such as USSD/SMS
 * Publish high-level mobile roadmap [in progress]
 * Kiwix release [in progress]
 * Default .m for all sibling projects [in progress]
 * Publish the mobile research documentation sent by Parul [Guillaume & Rebecca]

Features

 * AFT
 * Test versions of Feedback Form in production [Scheduled for Dec 15]
 * Design and development of Feedback Page, using data from test versions of Feedback Form [deployment scheduled for January]


 * Moodbar/Feedback Dashboard:
 * Email notification and other features[scheduled for Wed 12/14]


 * Visual Editor: Test deployment of visual editor for limited use case, possibly only experienced editors; collect feedback from community and evaluate for adding to product backlog for visual editor features [DONE]


 * Multimedia
 * Upload Wizard in maintenance mode
 * Deploy accumulated fixes
 * TMH review completed by Ian and Neil; pending review by Brion or Tim for security; MDale has completed chunked uploading changes; [testing in December and January to be merged into Rel 1.19]
 * TMH deployment (contingent on getting fixes completed based on code review comments)


 * I18n/L10n
 * WebFonts: Full scale deployment [DONE]
 * Deploy new Narayam input methods and fixes [DONE]

Operations

 * Squid deployment @ Eqiad
 * Deploy puppetmaster dashboard [DONE]
 * Lily [Mailman] server replacement [PY/DZahn]
 * DB servers refresh [in progress]
 * starts with db9 (OTRS)
 * /home move to NetApp [Mark] [in progress]
 * Swift integration for media storage [Ben/Aaron] (see above)
 * https support for mobile (https:// *.m.wikipedia.org) [partial]

Platform / Dev-Ops

 * MediaWiki 1.19/Roadmap
 * Cut 1.19 release branch
 * Finish code review


 * Begin developing framework for integration testing [Antoine, new QA Lead]]
 * possibly Selenium


 * Make decision regarding project parameters for inline template scripting [discuss in SF Hackathon]
 * Lua or Javascript V8 or something else?


 * Git conversion
 * Finish up specific Git management scripts
 * SVN import, post-release/deployment make r/o switch
 * Deployment scripts
 * CI tests get run when a developer chooses to push to the stage between their branch and the mainline branch


 * Volunteer coordination and outreach
 * Consolidate and integrate the content from the training materials [Sumana, Guillaume] [in progress]


 * Log collection (udp2log) improvement [Tim]
 * Deployment to locke in January


 * Superprotection feature (notes to be copied to wiki somewhere by RobLa or Tim)


 * Swift
 * Full deployment [Aaron/Ben/Mark B]

Mobile / Offline

 * Selection Sifter to production - Yuvi Panda's GSOC Project [Yuvi]
 * Deploy mobile web photo uploading to mobile beta [Patrick/Yuvi]
 * RSS feeds for article snippets like "Featured Articles" [Max]
 * Deploy prototype of Wikipedia on SMS/USSD for India audience [Praekelt Foundation / Tomasz]
 * Full screen search to production [holding]
 * Photo upload [DELAYED]

Features

 * New Page Patrolling: Limited test release (stretch goal: release to production)
 * [development scheduled to start Jan 2012]
 * [Needs review] Global Profile: continue design work, begin development
 * Addition moderation tools: TBD; moderation tools in support of contribution features that are not captured elsewhere
 * Personal Image Filter development presumed to begin, may be done via vendor development. Support third party block lists instead of categories?
 * Deploy AFT improvement iteration
 * TMH testing
 * I18n WebFonts update
 * Visual Editor

Operations
Labs (Ryan) Created a SAL for every project, and a combined SAL (https://labsconsole.wikimedia.org/wiki/Server_Admin_Log) Updated adminbot (morebots) to be configurable Packaged and in the process of puppetizing adminbot with hyperon (volunteer) petan brought up nagios in labs petan, hyperon, and damianz have been working on a bots infrastructure Cluebot is now running in labs; it was previously not running due to its infrastructure being dead krinkle brought up testswarm in labs Helped diederik and nimish bring up reportcard project and move the service over Made a bunch of projects and added a bunch of users Did an architecture overview of production and labs infrastructure with Andrew and Sara Put in for a quote for volume storage using glusterfs A ton of minor fixes
 * db refresh (s1 to s7)
 * EQIAD datacenter new row of racks
 * new network provider for TPA and additonal provider for EQIAD
 * Security updates (key management)
 * Thumbnail server replacement (ms5)
 * External storage recompression [Ben/Tim]

Platform / Dev-Ops

 * HipHop decision point and development schedule
 * Wikipedia operations history+architecture document [Guillaume]
 * New error page [Guillaume]
 * Git conversion
 * First release from git mainline development branch
 * MediaWiki roadmap/1.19
 * 1.19 deploy (from SVN)
 * Postmortem 1.19 after the 1.19 deployment
 * ResourceLoader 2


 * Database-backed session handling [New Security Eng]

Features

 * GlobalProfile development to kick off in January, provided development capacity is available
 * Finalize development of iteration 2 on NPP / aticle creation flow
 * Deploy AFT improvement iteration
 * RL2/Gadgets improvements deployed with MW 1.19 (dependent on Rel 1.19 schedule)
 * [in progress; to be deployed with Rel 1.19]
 * TMH deployment

Platform

 * Interwiki transclusion
 * HipHop deployment preparation
 * Create HipHop instance that includes a full production image
 * run CI tests on HipHop as well as Zend
 * push further necessary fixes into the detail plan
 * possibly break out parsing from rest of MediaWiki

Mobile / Offline

 * First MobileFrontend feature related to mobile editing (e.g. section editing or annotation)

Features

 * Prototype deployment of GlobalProfile
 * Deploy iteration 2 on NPP / article creation flow
 * Deploy AFT improvement iteration

Features

 * Visual Editor: Small scale production/default deployment of visual editor

Mobile / Offline

 * First MobileFrontend feature related to mobile uploads (e.g. media upload beta).

Mobile / Offline

 * USSD gateway rollout
 * SMS gateway rollout
 * Varnish as default cache for Mobile
 * New article creation
 * Watchlists
 * Diffs
 * Search improvements
 * Kiwix on Android
 * Partner integration (various)
 * Lazy loading articles
 * reduce bandwidth requirements for some users
 * WikipediaZero

Features

 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Re-energize LiquidThreads (rename it?!) (may need refactoring for notification streams)
 * Photo/media moderation tools; additional patrolling and moderation tooling
 * Help features
 * Mentoring features
 * Other real-time features, chat

Platform / Dev-Ops

 * Config management overhaul
 * Extension management?

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)

Parking lot and todos

 * Details of git migration (extension mgmt, permissions, code review tool etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf http://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles


 * Database-backed session handling
 * HipHop deployment
 * Extension:Configure
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * UDP based logging (Add RFC 3164 (BSD syslog protocol) support. See Brion's post : Wikipedia downtime 2x today)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)