Roadmap/2012/November

From mediawiki.org

November 2012[edit]

Technical Operations[edit]

  • Deploy Varnish on upload@esams to replace Squid [done]
  • Full Memcached deployment@Tampa [Done]
  • Swift upgrade from 1.5 to 1.74 [Faidon/Ariel] [on-going rolling upgrade]
  • Ceph evaluation [Asher / Mark / Faidon] [on-going]
  • Multiwrite thumbnails to nas01 [Aaron/Faidon/Mark] [done]
  • Upgrade MW AppServers/ImageScalers to Precise [done]
  • Complete Fundraising infrastructure buildout@Eqiad and lockdown [done]
  • Complete Wikivoyage infrastructure buildout and deploy [Daniel/Asher] [Done]
  • Provision 2 extra parser cache servers (pc2, pc3) @ Tampa [done]


Features[edit]

  • VisualEditor/Parsoid (Trevor, Roan, Rob, Gabriel, Subbu, MarkTraceur, + Wikia)
    • API finalization/debugging [IN PROCESS]
    • Iteration on Visual Editor for Dec release [IN PROCESS]
    • Parsoid testing framework for semantic/superficial regressions [IN PROCESS]
    • Node.js parsoid to completion [ONGOING]
  • New Pages Feed (fka Page Triage) (Kaldari, Benny, Nischay)
    • Descriptive statistics on NPF (revised dashboard on toolserver) [Dario] [IN PROCESS]
    • Bug fixes (no more feature development) [ONGOING]
  • Echo_(Notifications) (Kaldari, Benny, Werdna, Luke Welling, Preilly, Aaron S)
    • Continued development of prototype for Mediawiki.org [Werdna/Vibha/Kaldari/Benny] [ONGOING] (note: prototype has been reverted on Mediawik.org. Redeployment is pending further feature development)
    • First community feedback via IRC Office Hours (Nov 14) [DONE]
    • UI and bugfixing of Echo [ONGOING]
    • Global user changes [JamesF] [IN PROCESS]
    • (see Performance engineering for infrastructure prep work:)
  • E3 (Ori, S, Luke)
    • Clicktracking/Analytics
      • Event tracking live [DONE]
      • EventTracking schema filtering [DONE]
      • Clicktracking sunset [DONE]
      • Funnel tracking of Community Portal [ONGOING]
  • Post-edit Feedback 2
    • Data analysis once experiment is complete [DUE 10/31]
    • Continued productization [DONE]
  • Account Creation User Experience
    • Minor UI tweaks (ACUX 2, no benefits) [DONE]
    • Real-time validation test (ACUX 3)[DONE]
    • Deploy ACUX 3 to 100% on English Wikipedia during productization [DONE]
    • Productization of ACUX [IN ANALYSIS]
      • Data analysis once experiment is complete [IN PROCESS]
  • Onboarding
    • Develop wireframes and workflow [DONE]
    • Work with SuggestBot author to develop list of tasks [IN PROCESS]
    • Extensionize guided tour [IN PROCESS]
    • Deploy first version of onboarding to production [Dec 6 target]
    • MicroDesign
      • anon user template [DONE]
      • other edit window templates [IN PROCESS]
      • (see Agora)

Fundraising[edit]

  • Data discrepancy error [DONE]
  • Start the Fundraiser in early November [DONE-NOV 26]

Mobile[edit]

  • DevCamp [DONE]
  • Mobile Web
    • Beta editing [DONE]
    • Beta watchlist (add and view) [DONE]
    • Beta random [DONE]
    • Better table formatting [DONE]
  • Apps
    • iOS Bugfixes [DONE]
      • landscape [DONE]
      • Release to market
    • Android
      • Offline issue
  • Solr for GeoData [CODE REVIEWED]
    • CR [DONE]

Platform / Dev-Ops[edit]

MediaWiki Core[edit]

  • Datacenter migration (see TechOps section above)
  • Git/Conversion (Chad)
    • LDAP fixes for our Gerrit use [Chad] [partial]
    • Usability improvements for Gerrit (collaboration at Gerrit Summit) [Chad] [partial]
  • Swift
    • Copying thumbnail files from Swift into NFS [Aaron]
  • MediaWiki 1.20/Roadmap
  • Continuous integration
    • Integrate zuul into Jenkins pipeline [Antoine] (bug 40681) -- [DONE]
    • Move Jenkins job setup out of web interface into Git using Jenkins Job Builder [Antoine] [DONE]
  • Wikidata deployment
    • Make ULS support for logged-in users only [Wikidata team] [DONE]
  • Site performance
    • One-off template profiling [Tim] [DONE]
  • Lua scripting [Tim, Chad, Brad]
    • Deploy TemplateSandbox [Brad] [ongoing. deployed to test2, mediawiki.org]
    • Shared Lua repo [Chad] [partial, postponed]
    • Additional APIs for multi-lingual support [Tim] [DONE]
  • Wikivoyage migration [Chris, Sam]
    • Soft launch -- DONE
    • Public launch -- DONE
  • Search
    • Added Timeouts to lsearchd [Tim] [DONE]

Multimedia[edit]

  • TimedMediaHandler
    • Bugfixing urgent deployment related issues DONE (Michael Dale)
    • Deployment of transcoding boxes [DONE]
    • Create production deployment plan [DONE]
    • Create new async upload API for large file uploads [Aaron] (bug 36587) [partial]
    • Add new async upload API support to UploadWizard [Jan] [DONE]
    • Multi-cache support in MediaWiki (Varnish/Squid/etc) [Aaron] (bug 40779) [DONE]

QA[edit]

  • Beta labs
    • AFTv5 and E3 hosted reliably on beta, prototype host is decommissioned [DONE]
    • PageTriage code is hosted reliably on beta, state of db contents may not be up to date (bug 36228) [IN PROCESS] [Antoine]
  • Automated testing
    • Ported reporting frameworks from raw RSpec to Cucumber (see below) [ONGOING]
    • Tests are running under Jenkins in Cloudbees against test2 and beta. [DONE]
    • Creating new tests for AFTv5 [ONGOING]
    • Created [backlog page] . Cucumber format and backlog should enable wider participation in ATDD (Acceptance Test Driven Development) [ONGOING]
    • Coverage goals: regression tests from backlog page, extensions managed on beta. [ONGOING]
  • Feature testing
    • E2
      • Significant AFTv5 testing on hold pending sharding, creating browser regression tests meanwhile [ONGOING]
      • PageTriage development moving slowly, focusing on support in beta labs [ONGOING]

Analytics[edit]

See also: Analytics/Roadmap

Kraken[edit]
  • Pixel Service
    • First Prototype: udp2log -> product code topic filters -> Kafka -> hadoop [otto] (Nov 9) [DONE]
    • Pixel Service documentation [dsc] (Nov) [DONE]
    • Work out Event Data conventions for proxied fields / normal web request components [dsc, e3] (Nov) [DONE]
  • Product Code wiki [dsc] (Nov) [DONE]
    • Email stakeholders and gather feedback [dsc] (Dec)
  • Reinstall Dells [otto] (Nov) [DONE]
  • Fix Cisco machines (an02, an07) [otto] (Nov) (Awaits ops help)
  • Puppetize Kraken [otto] (Ongoing)
  • Monitoring [otto] (Nov)
    • Ganglia Monitoring [otto] (Nov) [DONE]
    • Set up JMX monitoring -- needs to be our LAN [otto + dsc] (Nov)
  • Core Jobs
  • Set up full pixel/firehose (w/o ETL) via udp2log [otto] (Dec) (Ready to go, holding off on flipping switch)
    • Figure out varnishncsa log format for event stream [otto + dsc] (Nov) [DONE]
  • Get to consensus with Ops regarding logging of the firehose [dsc + otto] [DONE]
    • Research needed: test running cli JVM producers uses acceptable resources [otto] (Oct) [not doing]


Limn[edit]
  • Refactor charting to use d3 [dan + dsc]
    • Feature Parity with Dygraphs (plus bugfixes, etc) (Dec) [DONE]
  • Support the Global Dev dashboard [evan] (Ongoing)
  • Support the Gerrit Stats dashboard [diederik] (Ongoing)
Legacy Log Collection[edit]
  • Add support for new domain names in webstatscollector (blog, etc) [diederik] (Sept) [Status: Made a Baby Instead???]
  • udp2log filters
    • Update filters for Wikipedia Zero [otto] (Ongoing)
    • Filter by X-Carrier headers. [otto + asher + diederik] (Oct) [Status: At Least We Think It's a Baby?]
    • udp-filter to filter by http status. [otto] (Oct) [Status: Maybe It's a T-Rex!!!]
WikiStats[edit]

(Stale -- will consult Stefan)

  • Reduce backlog regarding Wikistats traffic (squid etc) scripts [stefan] (Oct)
  • Repair data errors in wikistats, and add process for checking data integrity [ezachte] (Sept)
  • Make wikistats more robust (MoM validations) [ezachte] (Oct)
  • Add Blackbox testing to WikiStats [diederik + ezachte] (Oct)
Infrastructure[edit]
  • Access/support requests for stat1, stat1001 [otto] (Ongoing)
  • Maintenance of oxygen/emery/locke [otto] (Ongoing)
  • Kraken
    • Reinstall Dells [otto] (Nov) [DONE]
    • Fix Cisco machines (an02, an07) [otto] (Nov) (Awaits ops help)
Data Releases[edit]
  • Start pushing datasets to AWS [diederik] (Nov) [DONE]
    • Blogpost about what awesome stuff you can do with this [diederik + ?] (Oct) [Status: BABYPOCALYPSE 2k12]
  • Finalize scripts to massively compact dammit.lt data [erik] (Oct) [???]
  • Create Data Release Practices Task Force [diederik] (Sept) [Status: Baaaaaby!]
  • Publish Monthly Report Card -- deal with monthly data processing irregularities, perform correction/validation [ezachte + diederik + dsc] (Ongoing)


Engineering Community Team[edit]

  • Volunteer coordination and outreach
    • Event planning & followup [Sumana, Quim]
    • Planning online or in-person QA trainings for the community [Quim]
    • Planning community testing events [Quim]
  • Bug Wrangling [Andre + volunteers]
    • Updating BZ maintenance documentation [Andre] -- ongoing
    • Improving workflow for highest-priority bugs escalation -- ongoing [Andre]
  • Summer of Code 2012
    • Postmortem [Sumana] - DONE
    • Pushing on merges -- ongoing [Sumana]
  • Outreach Program for Women - ongoing [Quim, Sumana]
  • Tech Ambassadors Network [Guillaume] -- ongoing on meta
    • Volunteer product management program [Guillaume] -- being integrated into Tech/Ambassadors
  • Misc Engineering community proposals
    • 20% time/LevelUp [Sumana] -- ongoing - socialized internally, soon to launch for real
    • CoC [Sumana] -- in process - target 15 Dec

i18n/L10n[edit]

No roadmap updates provided, see Wikimedia engineering report/2012/October for October updates.

Retrieved from "https://www.mediawiki.org/w/index.php?title=Roadmap/2012/November&oldid=5092344"