Roadmap

Background: In recent years, Wikimedia Foundation's engineering planning has primarily taken place in a combination of internal goals documents and public project-level documentation. The purpose of this page is to establish a practice of sharing holistic engineering goals, initially primarily with a short term horizon, as a public document with open participation.

See also:
 * Product Whitepaper for Wikimedia Foundation product priority justifications; Strategic Plan for long-term WMF goals, some of which are engineering goals
 * Roadmap/Archive for an earlier effort to create a roadmap more focused on point releases of the MediaWiki software
 * Software deployments schedule

Previous months:
 * /2011/September
 * /2011/October
 * /2011/November

Item syntax: Examples:
 * Action item
 * [Needs consensus] Deploy unicorns [Brion/Tim]
 * Deploy unicorns [Brion/Tim] [DONE]

Platform / Dev-Ops

 * Continuous integration improvements
 * TestSwarm integration [Antoine, Timo, Chad]
 * Move towards continuous integration via git, goalpost: weekly deployment


 * Jenkins (Testswarm/PHPUnit tests) on git branches


 * Git conversion
 * prelim test conversions early in month [Chad/Brion] [DONE]
 * Git workflow architecture review [Chad/Brion]
 * Agree on implementation strategies regarding remaining development process questions, e.g. how to handle multi-repo commits


 * Code review management
 * Under 500 revs to code review on trunk


 * Real-time visualization of all MediaWiki profiler data [Asher]
 * include visualization timepoints of all code deploys [DONE]
 * announce and communicate about new visualization tools


 * Engineering project documentation
 * Publish December report [Guillaume]


 * Wikimedia blog maintenance
 * Implement Wikimedia blog theme & features tweaks [Guillaume] [partial]


 * Swift (http://wikitech.wikimedia.org/view/Swift#Documentation_Organization)
 * Initial performance tests
 * Purchase additional thumbs storage hardware [Ben/Rob]
 * Performance Stress Tests

http://wikitech.wikimedia.org/view/Swift/Tasks#Overall_Schedule
 * Thumbnail deployment [Aaron/Ben/Mark B]
 * Merge FileBackend refactoring work to trunk [Aaron]

Mobile / Offline

 * Feature iteration
 * User login [DONE]
 * JavaScript tiering/ MW Views [DONE]
 * Small bug fixes
 * Unlocking viewport [DONE]
 * Updated image description pages '[DONE]
 * HTTPS for Wikipedia Mobile [DONE]
 * WikipediaZero features [in progress, escalating priority]
 * Wikimedia Mobile (Android) RC1/2/3 [RC1 - done]
 * Daily Featured Articles RSS [12/19 Max starts]
 * Begin implementation of GPS Storage/Retrieval
 * Other features started at India Hackathon, such as USSD/SMS
 * Publish high-level mobile roadmap [in progress]
 * Kiwix release [in progress]
 * Default .m for all sibling projects [in progress]
 * Publish the mobile research documentation sent by Parul [Guillaume & Rebecca]

Features

 * AFT
 * Test versions of Feedback Form in production [Scheduled for Dec 15]
 * Design and development of Feedback Page, using data from test versions of Feedback Form [deployment scheduled for January]


 * Moodbar/Feedback Dashboard:
 * Email notification and other features[scheduled for Wed 12/14]


 * Visual Editor: Test deployment of visual editor for limited use case, possibly only experienced editors; collect feedback from community and evaluate for adding to product backlog for visual editor features [DONE]


 * Multimedia
 * Upload Wizard in maintenance mode
 * Deploy accumulated fixes
 * TMH review completed by Ian and Neil; pending review by Brion or Tim for security; MDale has completed chunked uploading changes; [testing in December and January to be merged into Rel 1.19]
 * TMH deployment (contingent on getting fixes completed based on code review comments)


 * I18n/L10n
 * WebFonts: Full scale deployment [DONE]
 * Deploy new Narayam input methods and fixes [DONE]

Operations

 * Squid deployment @ Eqiad
 * Deploy puppetmaster dashboard [DONE]
 * Lily [Mailman] server replacement [PY/DZahn]
 * DB servers refresh [in progress]
 * starts with db9 (OTRS) {DONE]
 * /home move to NetApp [Mark] [in progress]
 * Swift integration for media storage [Ben/Aaron] (see above)
 * https support for mobile (https:// *.m.wikipedia.org) [partial]
 * Labs (Ryan)
 * Create a SAL for every project, and a combined SAL (https://labsconsole.wikimedia.org/wiki/Server_Admin_Log) [Done]
 * Update adminbot (morebots) to be configurable [Done]
 * Package and  puppetize adminbot with hyperon (volunteer) [Done]
 * nagios in labs [Done]
 * deploy Cluebot [Done]
 * Bring up testswarm in labs [Done]
 * bring up reportcard project and move the service over [Done]

Platform / Dev-Ops

 * MediaWiki 1.19/Roadmap
 * Cut 1.19 release branch
 * Finish code review


 * Begin developing framework for integration testing [Antoine, new QA Lead]]
 * possibly Selenium


 * Make decision regarding project parameters for inline template scripting [discuss in SF Hackathon]
 * Lua or Javascript V8 or something else?


 * Git conversion
 * Finish up specific Git management scripts
 * SVN import, post-release/deployment make r/o switch
 * Deployment scripts
 * CI tests get run when a developer chooses to push to the stage between their branch and the mainline branch


 * Volunteer coordination and outreach
 * Consolidate and integrate the content from the training materials [Sumana, Guillaume] [in progress]


 * Log collection (udp2log) improvement [Tim]
 * Deployment to locke in January


 * Superprotection feature (notes to be copied to wiki somewhere by RobLa or Tim)


 * Swift
 * Full deployment [Aaron/Ben/Mark B]

Mobile / Offline

 * Selection Sifter to production - Yuvi Panda's GSOC Project [Yuvi]
 * Deploy mobile web photo uploading to mobile beta [Patrick/Yuvi]
 * RSS feeds for article snippets like "Featured Articles" [Max]
 * Deploy prototype of Wikipedia on SMS/USSD for India audience [Praekelt Foundation / Tomasz]
 * Full screen search to production [holding]
 * Photo upload [DELAYED]

Features

 * New Page Patrolling: Limited test release (stretch goal: release to production)
 * [development scheduled to start Jan 2012]
 * [Needs review] Global Profile: continue design work, begin development
 * Addition moderation tools: TBD; moderation tools in support of contribution features that are not captured elsewhere
 * Personal Image Filter development presumed to begin, may be done via vendor development. Support third party block lists instead of categories?
 * Deploy AFT improvement iteration
 * TMH testing
 * I18n WebFonts update
 * Visual Editor

Operations

 * db refresh (s1 to s7)
 * EQIAD datacenter new row of racks
 * new network provider for TPA and additonal provider for EQIAD
 * Security updates (key management)
 * Thumbnail server replacement (ms5)
 * External storage recompression [Ben/Tim]

Platform / Dev-Ops

 * HipHop decision point and development schedule
 * Wikipedia operations history+architecture document [Guillaume]
 * New error page [Guillaume]
 * Git conversion
 * First release from git mainline development branch
 * MediaWiki roadmap/1.19
 * 1.19 deploy (from SVN)
 * Postmortem 1.19 after the 1.19 deployment
 * ResourceLoader 2


 * Database-backed session handling [New Security Eng]

Features

 * GlobalProfile development to kick off in January, provided development capacity is available
 * Finalize development of iteration 2 on NPP / aticle creation flow
 * Deploy AFT improvement iteration
 * RL2/Gadgets improvements deployed with MW 1.19 (dependent on Rel 1.19 schedule)
 * [in progress; to be deployed with Rel 1.19]
 * TMH deployment

Platform

 * Interwiki transclusion
 * HipHop deployment preparation
 * Create HipHop instance that includes a full production image
 * run CI tests on HipHop as well as Zend
 * push further necessary fixes into the detail plan
 * possibly break out parsing from rest of MediaWiki

Mobile / Offline

 * First MobileFrontend feature related to mobile editing (e.g. section editing or annotation)

Features

 * Prototype deployment of GlobalProfile
 * Deploy iteration 2 on NPP / article creation flow
 * Deploy AFT improvement iteration

Features

 * Visual Editor: Small scale production/default deployment of visual editor

Mobile / Offline

 * First MobileFrontend feature related to mobile uploads (e.g. media upload beta).

Mobile / Offline

 * USSD gateway rollout
 * SMS gateway rollout
 * Varnish as default cache for Mobile
 * New article creation
 * Watchlists
 * Diffs
 * Search improvements
 * Kiwix on Android
 * Partner integration (various)
 * Lazy loading articles
 * reduce bandwidth requirements for some users
 * WikipediaZero

Features

 * Affiliation tools for WikiProjects [build on GlobalProfile]
 * Re-energize LiquidThreads (rename it?!) (may need refactoring for notification streams)
 * Photo/media moderation tools; additional patrolling and moderation tooling
 * Help features
 * Mentoring features
 * Other real-time features, chat

Platform / Dev-Ops

 * Config management overhaul
 * Extension management?

Continuous Responsibilities

 * Code review
 * RT review / ops tasks
 * Community extension review and deployment
 * Bug triages and general bug fixes
 * New extension deployment (Review queue)

Parking lot and todos

 * Details of git migration (extension mgmt, permissions, code review tool etc.)
 * More work on product backlogs for features and mobile
 * Ops section
 * Fundraising flesh-out
 * cf http://www.mediawiki.org/wiki/Wikimedia_Platform_Engineering and merge as appropriate
 * Namespace hacks for development of simple language wikipedias in languages other than English without server, database, or major wiki installation provisioning
 * Most popular related articles


 * Database-backed session handling
 * HipHop deployment
 * Extension:Configure
 * Unicode normalization considerations
 * Schema handling improvements (Abstracted database schema description, schema updaters to make the multiple schema variants for MySQL 4.0 and 5/UTF-8 more maintainable)
 * Parallelizable, restartable dump runner (notes; Clean up the currently fragile externaldump process, make our internal backups more reliable
 * OAI cleanup (Clean up OAI exporter and importer, add time-delay features
 * Deletion queue system (notes; Human-helpful process improvements
 * Integrated heuristic checks, improved patrolling (human-helpful process improvements)
 * UDP based logging (Add RFC 3164 (BSD syslog protocol) support. See Brion's post : Wikipedia downtime 2x today)
 * On-wiki namespace manager (Special:Namespaces) (original wikidata branch specs and see notes)