Roadmap/2012/November

Technical Operations

 * Deploy Varnish on upload@esams to replace Squid [done]
 * Full Memcached deployment@Tampa [Done]
 * Swift upgrade from 1.5 to 1.74 [Faidon/Ariel] [on-going rolling upgrade]
 * Ceph evaluation [Asher / Mark / Faidon] [on-going]
 * Multiwrite thumbnails to nas01 [Aaron/Faidon/Mark] [done]
 * Upgrade MW AppServers/ImageScalers to Precise [done]
 * Complete Fundraising infrastructure buildout@Eqiad and lockdown [done]
 * Complete Wikivoyage infrastructure buildout and deploy [Daniel/Asher] [Done]
 * Provision 2 extra parser cache servers (pc2, pc3) @ Tampa [done]

Features

 * VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
 * API finalization/debugging [IN PROCESS]
 * Iteration on Visual Editor for Dec release [IN PROCESS]
 * Parsoid testing framework for semantic/superficial regressions [IN PROCESS]
 * Node.js parsoid to completion [ONGOING]


 * New Pages Feed (fka Page Triage) (Kaldari, Benny, Nischay)
 * Descriptive statistics on NPF (revised dashboard on toolserver) [Dario] [IN PROCESS]
 * Bug fixes (no more feature development) [ONGOING]


 * Echo_(Notifications) (Kaldari, Benny, Werdna, Luke Welling, Preilly, Aaron S)
 * Continued development of prototype for Mediawiki.org [Werdna/Vibha/Kaldari/Benny] [ONGOING] (note: prototype has been reverted on Mediawik.org. Redeployment is pending further feature development)
 * First community feedback via IRC Office Hours (Nov 14) [DONE]
 * UI and bugfixing of Echo [ONGOING]
 * Global user changes [JamesF] [IN PROCESS]
 * (see Performance engineering for infrastructure prep work:)
 * queueing & Echo
 * Database sharding & Flow


 * Article feedback (Matthias)
 * performance/db related changes to AFTv5 queries before scaling, + sharding (see Database sharding) [Matthias] [IN PROCESS]


 * E3 (Ori, S, Luke)
 * Clicktracking/Analytics
 * Event tracking live [DONE]
 * EventTracking schema filtering [DONE]
 * Clicktracking sunset [DONE]
 * Funnel tracking of Community Portal [ONGOING]


 * Post-edit Feedback 2
 * Data analysis once experiment is complete [DUE 10/31]
 * Continued productization [DONE]


 * Account Creation User Experience
 * Minor UI tweaks (ACUX 2, no benefits) [DONE]
 * Real-time validation test (ACUX 3)[DONE]
 * Deploy ACUX 3 to 100% on English Wikipedia during productization [DONE]
 * Productization of ACUX [IN ANALYSIS]
 * Data analysis once experiment is complete [IN PROCESS]


 * Onboarding
 * Develop wireframes and workflow [DONE]
 * Work with SuggestBot author to develop list of tasks [IN PROCESS]
 * Extensionize guided tour [IN PROCESS]
 * Deploy first version of onboarding to production [Dec 6 target]


 * MicroDesign
 * anon user template [DONE]
 * other edit window templates [IN PROCESS]
 * (see Agora)

Fundraising

 * Data discrepancy error [DONE]
 * Start the Fundraiser in early November [DONE-NOV 26]

Mobile

 * DevCamp [DONE]
 * Mobile Web
 * Beta editing [DONE]
 * Beta watchlist (add and view) [DONE]
 * Beta random [DONE]
 * Better table formatting [DONE]
 * Apps
 * iOS Bugfixes [DONE]
 * landscape [DONE]
 * Release to market
 * Android
 * Offline issue
 * Solr for GeoData [CODE REVIEWED]
 * CR [DONE]

MediaWiki Core

 * Datacenter migration (see TechOps section above)
 * Git/Conversion (Chad)
 * LDAP fixes for our Gerrit use [Chad] [partial]
 * Usability improvements for Gerrit (collaboration at Gerrit Summit) [Chad] [partial]
 * Swift
 * Copying thumbnail files from Swift into NFS [Aaron]
 * MediaWiki 1.20/Roadmap
 * Bi-weekly deployment cycle continues
 * MediaWiki 1.21/wmf4 [DONE]
 * MediaWiki 1.21/wmf5 [partial]
 * Continuous integration
 * Integrate zuul into Jenkins pipeline [Antoine] -- [DONE]
 * Move Jenkins job setup out of web interface into Git using Jenkins Job Builder [Antoine] [DONE]
 * Wikidata deployment
 * Make ULS support for logged-in users only [Wikidata team] [DONE]
 * Site performance
 * One-off template profiling [Tim] [DONE]
 * Lua scripting [Tim, Chad, Brad]
 * Deploy TemplateSandbox [Brad] [ongoing. deployed to test2, mediawiki.org]
 * Shared Lua repo [Chad] [partial, postponed]
 * Additional APIs for multi-lingual support [Tim] [DONE]
 * Wikivoyage migration [Chris, Sam]
 * Soft launch -- DONE
 * Public launch -- DONE
 * Search
 * Added Timeouts to lsearchd [Tim] [DONE]

Multimedia

 * TimedMediaHandler
 * Bugfixing urgent deployment related issues DONE (Michael Dale)
 * Deployment of transcoding boxes [DONE]
 * Create production deployment plan [DONE]
 * Create new async upload API for large file uploads [Aaron] [partial]
 * Add new async upload API support to UploadWizard [Jan] [DONE]
 * Multi-cache support in MediaWiki (Varnish/Squid/etc) [Aaron] [DONE]

QA

 * Beta labs
 * AFTv5 and E3 hosted reliably on beta, prototype host is decommissioned [DONE]
 * PageTriage code is hosted reliably on beta, state of db contents may not be up to date [IN PROCESS] [Antoine]
 * Automated testing
 * Ported reporting frameworks from raw RSpec to Cucumber (see below) [ONGOING]
 * Tests are running under Jenkins in Cloudbees against test2 and beta. [DONE]
 * Creating new tests for AFTv5 [ONGOING]
 * Created [|test backlog page] . Cucumber format and backlog should enable wider participation in ATDD (Acceptance Test Driven Development) [ONGOING]
 * Coverage goals: regression tests from backlog page, extensions managed on beta. [ONGOING]
 * Feature testing
 * E2
 * Significant AFTv5 testing on hold pending sharding, creating browser regression tests meanwhile [ONGOING]
 * PageTriage development moving slowly, focusing on support in beta labs [ONGOING]

Analytics
See also: Analytics/Roadmap

Kraken

 * Pixel Service
 * First Prototype: udp2log -> product code topic filters -> Kafka -> hadoop [otto] (Nov 9) [DONE]
 * Pixel Service documentation [dsc] (Nov) [DONE]
 * Work out Event Data conventions for proxied fields / normal web request components [dsc, e3] (Nov) [DONE]
 * Product Code wiki [dsc] (Nov) [DONE]
 * Email stakeholders and gather feedback [dsc] (Dec)
 * Reinstall Dells [otto] (Nov) [DONE]
 * Fix Cisco machines (an02, an07) [otto] (Nov) (Awaits ops help)
 * Puppetize Kraken [otto] (Ongoing)
 * Monitoring [otto] (Nov)
 * Ganglia Monitoring [otto] (Nov) [DONE]
 * Set up JMX monitoring -- needs to be our LAN [otto + dsc] (Nov)
 * Solution Research page [dsc] (Nov) [DONE]
 * Core Jobs
 * Data Formats wiki page [dsc + diederik] (Nov) [DONE]
 * Avro Schemas for Request & Event Data records [dsc + diederik] (Dec) [DONE]
 * Set up full pixel/firehose (w/o ETL) via  [otto] (Dec) (Ready to go, holding off on flipping switch)
 * Figure out varnishncsa log format for event stream [otto + dsc] (Nov) [DONE]
 * Get to consensus with Ops regarding logging of the firehose [dsc + otto] [DONE]
 * Research needed: test running cli JVM producers uses acceptable resources [otto] (Oct) [not doing]

Limn

 * Refactor charting to use d3 [dan + dsc]
 * Feature Parity with Dygraphs (plus bugfixes, etc) (Dec) [DONE]
 * Support the Global Dev dashboard [evan] (Ongoing)
 * Support the Gerrit Stats dashboard [diederik] (Ongoing)

Legacy Log Collection

 * Add support for new domain names in webstatscollector (blog, etc) [diederik] (Sept) [Status: Made a Baby Instead???]
 * udp2log filters
 * Update filters for Wikipedia Zero [otto] (Ongoing)
 * Filter by X-Carrier headers. [otto + asher + diederik] (Oct) [Status: At Least We Think It's a Baby?]
 * udp-filter to filter by http status. [otto] (Oct) [Status: Maybe It's a T-Rex!!!]

WikiStats
(Stale -- will consult Stefan)
 * Reduce backlog regarding Wikistats traffic (squid etc) scripts [stefan] (Oct)
 * Repair data errors in wikistats, and add process for checking data integrity [ezachte] (Sept)
 * Make wikistats more robust (MoM validations) [ezachte] (Oct)
 * Add Blackbox testing to WikiStats [diederik + ezachte] (Oct)

Infrastructure

 * Access/support requests for stat1, stat1001 [otto] (Ongoing)
 * Maintenance of oxygen/emery/locke [otto] (Ongoing)
 * Kraken
 * Reinstall Dells [otto] (Nov) [DONE]
 * Fix Cisco machines (an02, an07) [otto] (Nov) (Awaits ops help)

Data Releases

 * Start pushing datasets to AWS [diederik] (Nov) [DONE]
 * Blogpost about what awesome stuff you can do with this [diederik + ?] (Oct) [Status: BABYPOCALYPSE 2k12]
 * Finalize scripts to massively compact dammit.lt data [erik] (Oct) [???]
 * Create Data Release Practices Task Force [diederik] (Sept) [Status: Baaaaaby!]
 * Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dsc] (Ongoing)

Engineering Community Team

 * Volunteer coordination and outreach
 * Event planning & followup [Sumana, Quim]
 * Planning online or in-person QA trainings for the community [Quim]
 * Planning community testing events [Quim]
 * Bug Wrangling [Andre + volunteers]
 * Updating BZ maintenance documentation [Andre] -- ongoing
 * Improving workflow for highest-priority bugs escalation -- ongoing [Andre]
 * Summer of Code 2012
 * Postmortem [Sumana] - DONE
 * Pushing on merges -- ongoing [Sumana]
 * Outreach Program for Women - ongoing [Quim, Sumana]
 * Tech Ambassadors Network [Guillaume] -- ongoing on meta
 * Volunteer product management program [Guillaume] -- being integrated into Tech/Ambassadors
 * Misc Engineering community proposals
 * 20% time/LevelUp [Sumana] -- ongoing - socialized internally, soon to launch for real
 * CoC [Sumana] -- in process - target 15 Dec

i18n/L10n
No roadmap updates provided, see Wikimedia engineering report/2012/October for October updates.