Help
Book creator (disable)

Wikimedia Platform Engineering

From MediaWiki.org
(Redirected from WMF General Engineering)
Jump to: navigation, search

The Wikimedia Foundation Platform Engineering team is responsible for the MediaWiki core, managing work from volunteer developers, as well as for providing services that are used by multiple technology teams. The Platform Engineering group is managed by Rob Lanphier. It is part of the Wikimedia Foundation's engineering department.

See also a series of articles presenting the Platform engineering group:

Contents

[edit] MediaWiki Core

[edit] Team

[edit] Responsibilities

  • Manage the MediaWiki release cycle
  • Ensure that MediaWiki core is meeting the evolving needs of the website
  • Make quality MediaWiki releases available for others outside of the Wikimedia Foundation
  • Develop and document a clear set of APIs so that external developers can create applications that easily interface with MediaWiki

[edit] Current activities

[edit] Main projects

Project Description Project lead & team Status
MediaWiki 1.20
[edit]
Deployment of MediaWiki 1.20wmf versions and release of MediaWiki 1.20
[edit]
Sam Reed
[edit]
[add new]
Status: (2012-05-02) MediaWiki 1.20wmf2 was just rolled out to all non-Wikipedia wikis in the past couple of hours. -- RobLa-WMF (talk) 20:14, 2 May 2012 (UTC)
Git conversion
[edit]
Conversion of the MediaWiki source repository from Subversion to Git
[edit]
Chad Horohoe, Antoine Musso, Sumana Harihareswara
[edit]
[add new]
Status: (2012-04-monthly) MediaWiki core and all extensions deployed on Wikimedia sites are getting into regular development cycles. The second group of extensions were migrated on April 27. We've also branched and released 1.20wmf1 to most Wikimedia sites. The priorities for Gerrit are currently fixing our UTF-8 problem, improving Gerrit integration with IRC and e-mail, making project/user information more discoverable, and upgrading Gerrit to 2.3 in the next week or so.
SwiftMedia
[edit]
Replacement of NFS with Swift on the Wikimedia cluster
[edit]
Ben Hartshorne, Aaron Schulz
[edit]
[add new]
Status: (2012-05-18) Aaron and Ben deployed a new version of the thumbnail handler to commons, test, test2, and mediawiki.org that uses our Swift FileBackend code, which should provide us useful production testing prior to using Swift FileBackend for handling originals. Cleanup of corrupted thumbnails is done now. Next up: Aaron plans to deploy a SiteStats fix today that should make uploads much faster and fix some timeout problems. Also, Ben and Aaron will roll out the FileBackend-based thumbnail handler to the rest of the wikis.
Lua scripting
[edit]
Support for Lua as a scripting language for MediaWiki end-users
[edit]
Tim Starling
[edit]
[add new]
Status: (2012-05-10) Tim Starling started the implementation of a replacement for MediaWiki markup-based templates, using the Lua scripting language, embedded via the Scribunto extension. You can try it now at the Labs prototyping site. Patrick Reilly spoke about the Lua scripting work in a presentation on 3 May 2012 (scroll to about 22:00 or 27:00 to see it). The current roadmap aims for a deployment to mediawiki.org in a few months; full deployment to Wikimedia sites is scheduled for 2013. Tim will lead tutorial sessions at the Berlin Hackathon 2012.
OAuth
[edit]
OAuth integration into MediaWiki
[edit]
Chris Steipp		 Add your first status update about this activity.

[edit] Continuous responsibilities

Project Description Project lead & team Status
Code review management
[edit]
Review of changes made to the MediaWiki core code and extensions used on Wikimedia wikis.
[edit]
Rob Lanphier, Tim Starling, Chad Horohoe, Sam Reed, Aaron Schulz
[edit]
[add new]
Status: (2012-04-monthly) The Wikimedia engineering 20% policy is the current approach of the Wikimedia Foundation to improving the code review situation. With the move to Git, we no longer have a code review backlog in trunk, but we are still facing a backlog of patches to review (in Gerrit and Bugzilla), RFCs to comment on, and extensions to review.
API maintenance
[edit]
Perennial maintenance of the MediaWiki API
[edit]
Sam Reed
[edit]
[add new]
Status: (2011-10-31) This activity was mostly on hold in October.
Shell requests
[edit]
Ongoing processing of Bugzilla requests that require shell access to fix
[edit]
Sam Reed
[edit]
[add new]
Status: (2012-04-30) Currently at 77 open (query for latest number), 3187 total resolved since the beginning of tracking (query for latest number), 2717 total fixed (query for latest number), and 46 resolved in April so far (query for April)
Security auditing and response
[edit]
Security auditing and response
[edit]
', 		 Add your first status update about this activity.

[edit] Planned or backburner projects

[edit] Recently completed projects

[edit] Quality Assurance

[edit] Responsibilities

  • Ensure that all new code that is released meets quality standards
  • Triage, manage and resolve all bug reports

[edit] Team

  • Chris McMahon

[edit] Current activities

Project Description Project lead & team Status
QA/testing
[edit]
Software testing and quality assurance for WMF projects
[edit]
Chris McMahon, Chris McMahon, TBD
[edit]
[add new]
Status: (2012-05-monthly) Testing for MediaWiki 1.20wmf3 is completed, including a community QA project with Weekend Testing on May 5. Tauhida Parveen was brought in to help with Timed Media Handler testing, but blocking bugs are preventing substantive testing. Testing is also being performed on Article Feedback Tool (AFT) and Editor Engagement projects, with an eye toward the community testing project for AFT in collaboration with OpenHatch on 9 June. Chris McMahon prepared a plan for automated UI testing, utilizing Selenium, RSpec, and PageObject. Work continues on improving the Beta cluster, with a sprint in San Francisco completed for the week of May 7.
Beta cluster
[edit]
Establish a functional, production-like environment in Wikimedia Labs suitable for final-stage testing of new features.
[edit]
Chris McMahon, Antoine Musso, Sam Reed, Faidon Liambotis
[edit]
[add new]
Status: (2012-05-20) Project is now a bit more on par with production status.
  • A job runner has been setup, currently catching up with all the pending jobs. Apparently, that includes some video resizing for TimeMediaHandler.
  • All code has been updated to a recent version and all databases have been upgraded.
  • Uploading file should work again (as of May 17th)
Continuous integration
[edit]
A testing platform to provide continuous quality control for MediaWiki
[edit]
Antoine Musso, Chad Horohoe, Timo Tijhof
[edit]
[add new]
Status: (2012-04-25) Jenkins has been upgraded, providing a nicer GUI and the jobs rewrite deployed.

Progress was made on implementing a universal linter for all gerrit changes (so far it lints all (modified) PHP files as part of the Jenkins job for each mediawiki/core changeset in gerrit).

The TestSwarm connection with Jenkins has been established - TestSwarm is now running MediaWiki's QUnit test suite again. The TestSwarm installation became idle after the migration to Gerrit+Git because it was configured for SVN. The old configuration is now disabled, and everything is now handled by Jenkins instead.

Christian Aistleitner created a test suite for the (rewritten) MWDumper system and are also monitored live on Jenkins.

[edit] Analytics

[edit] Team

  • Erik Zachte
  • Diederik van Liere
  • Andrew Otto
  • David Schoonover

[edit] Responsibilities

  • Implement and manage Web analytics tool to provide data for other foundation departments and the movement as a whole
  • Monitor and analyze key metrics on Wikimedia Foundation websites

[edit] Current activities

Project Description Project lead & team Status
Report card
[edit]
Streamlining and modularization of the creation process of the monthly reportcard
[edit]
Diederik van Liere, Andrew Otto, Fabian Kaelin, David Schoonover
[edit]
[add new]
Status: (2012-04-monthly) David Schoonover and Fabian Kaelin continued work on the new Report card, whose prototype is available on Wikimedia Labs. Andrew Otto has been working with the Operations team to puppetize existing services, and to add a third server (Oxygen) to run filters; we are in the process of migrating bayes to stat1. Andre Engels, Erik Zachte and Diederik van Liere have worked on new mobile reports that will be integrated into the new report card.
Page view logging
[edit]
Work surrounding udp2log filters
[edit]
Andrew Otto, Diederik van Liere, Erik Zachte
[edit]
[add new]
Status: (2012-05-22) Will soon deploy new version of udp-filter that accepts a variable number of fields. This will allow us to migrate more custom C filters to udp-filter. udp-filter can now filter by HTTP response status, and geocode along side of IP address.
Kraken - The WMF Analytics Cluster
[edit]
Setting up new full stack analytics cluster.
[edit]
Andrew Otto, David Schoonover, Diederik van Liere,
[edit]
[add new]
Status: (2012-05-22) First 10 Cisco boxes are available. Just puppetized a generic Java installation.

[edit] Engineering Community Team

Formerly "TL;DR" (Technical Liaisons & Developer Relations)

Archived notes from weekly meetings (caution: messy)

[edit] Team

[edit] Responsibilities

  • Attract and support volunteer developers so that they can become an even more powerful force in driving improvement on the website
  • Manage volunteer-developed extensions and widgets
  • Provide communications support for all groups in the engineering department
  • Develop clear documentation for MediaWiki so that new staff, volunteer developers and external users have a smooth ramp-up process to becoming MediaWiki developers

[edit] Current activities

Project Description Project lead & team Status
Bug management
[edit]
Management of Wikimedia's bug tracker.
[edit]
Mark Hershberger
[edit]
[add new]
Status: (2012-04-monthly) Mark Hershberger used the deployment of MediaWiki 1.20wmf1 to gather volunteers for the Bug Squad. He has begun working with volunteers on IRC to coordinate bug-grooming activities.
Summer of Code 2011
[edit]
A sponsored community program allowing students to join the community as developers.
[edit]
Sumana Harihareswara
[edit]
[add new]
Status: (2012-02-29) After contact from Sumana Harihareswara, GSoC students Kevin Brown and Akshay Agarwal have returned to their projects, working towards the goal of getting them deployed and in use on Wikimedia wikis.
Summer of Code 2012
[edit]
A sponsored community program allowing students to join the community as developers.
[edit]
Sumana Harihareswara, Gregory Varnum as backup organizational administrator
[edit]
[add new]
Status: (2012-04-monthly) Wikimedia engineers have chosen nine students for this year's program. For the next few weeks, until May 21st, the students and their mentors are working together to train the students in MediaWiki development, so that they'll have all the basic domain knowledge they'll need to succeed during the summer.
Engineering project documentation
[edit]
An activity to ensure that project documentation of Wikimedia engineering activities is complete and up-to-date.
[edit]
Guillaume Paumier
[edit]
[add new]
Status: (2012-05-monthly) Rob Moen implemented the first version of a JavaScript Status helper tool that makes it easier to add new status udpates to activity pages. Guillaume Paumier initiated a project to clean up the Wikitech wiki, possibly facilitated by a JavaScript tool derived from one used by WikiProjects to assess articles.
Volunteer coordination and outreach
[edit]
Coordination of volunteer efforts, technical outreach and recruitment of new volunteers
[edit]
Sumana Harihareswara
[edit]
[add new]
Status: (2012-04-monthly) Sumana Harihareswara and other members of the Engineering Community Team continued to follow up on contacts, recruit new contributors to the Wikimedia tech community, and mentor new contributors. They granted developer access and planned upcoming events.
Wikimedia blog maintenance
[edit]
Maintenance and new features for the Wikimedia blog
[edit]
Guillaume Paumier, 
[edit]
[add new]
Status: (2012-04-monthly) Guillaume Paumier made minor changes to the theme, like not displaying trackbacks and pings in the comments list. He investigated plugins to support multiple authors, photo galleries and multilingual posts, and is testing Piwik as a possible solution to record metrics. Chad Horohoe created git repositories for the WP-Victor theme and the WMBlog plugin in Gerrit, and imported their history from GitHub.
Wikimedia Foundation engineering 20% policy
[edit]
A policy and practice whereby WMF engineering staff spend at least 20% of their work time on tasks that directly serve the Wikimedia developer and user community
[edit]
Sumana Harihareswara, Rob Lanphier, All engineering staff
[edit]
[add new]
Status: (2012-05-16) Sumana Harihareswara is coordinating 20% time effort. The highest priority is the Gerrit merge queue, especially for backlogged components such as UploadWizard and ProofreadPage, and secondarily patches awaiting review in Bugzilla for MediaWiki or WMF-deployed extensions. Some participants are concentrating on the extensions awaiting review for deployment.

[edit] Planned or backburner projects

[edit] Recently completed projects

Retrieved from "http://www.mediawiki.org/w/index.php?title=Wikimedia_Platform_Engineering&oldid=541824"
Personal tools
Namespaces

Variants
Actions
Navigation
Support
Download
Development
Communication
Print/export
Toolbox