Wikimedia Engineering/Report/2014/June

''Note: We're also providing a shorter, simpler and translatable version of this report that does not assume specialized technical knowledge.

Engineering metrics in June:
 * 151 unique committers contributed patchsets of code to MediaWiki.
 * The total number of unresolved commits went from around 1440 to about 1575.
 * About 14 shell requests were processed.

Upcoming events
There are many opportunities for you to get involved and contribute to MediaWiki and technical activities to improve Wikimedia sites, both for coders and contributors with other talents.

For a more complete and up-to-date list, check out the Project:Calendar.

Work with us
Are you looking to work for Wikimedia? We have a lot of hiring coming up, and we really love talking to active community members about these roles.

*

Announcements

 * Elliot Eggleston joined the Wikimedia Foundation as a Features Engineer in the Fundraising-Tech team (announcement).

Technical Operations
New Dallas data center
 * On-site work has started in our new Dallas (Carrollton) data-center (codfw). Racks have been installed, the equipment we moved from Tampa has been racked and cabling work has been mostly completed over the course of the month. We are now awaiting the installation of connectivity to the rest of our network as well as the arrival of the first newly-ordered server equipment, so server & network configuration can commence.

Puppet 3 migration
 * In July we migrated from Puppet 2 to Puppet 3 on all production servers. Thanks to the hard work of both volunteers and Operations staff on our Puppet repository in the months leading up to this, this migration went very smoothly.

Labs metrics in June:
 * Number of projects: 173
 * Number of instances: 424
 * Amount of RAM in use (in MBs): 1,741,312
 * Amount of allocated storage (in GBs): 19,045
 * Number of virtual CPUs in use: 855
 * Number of users: 3,356

 Wikimedia Labs
 * Last month we switched the Labs puppetmaster to Puppet 3; this month all instances switched over as well. Some cleanup work was needed in our puppet manifests to handle Trusty and Puppet 3 properly; everything is fairly stable now but a bit of mopping up remains.

Editor retention: Editing tools
VisualEditor  edit In June, the VisualEditor team provided a new way to see the context of links and other items when you edit to make this easier, worked on the performance and stability of the editor so that users could more swiftly and reliably make changes to articles, and made some improvements to features focussed on increasing their simplicity and understandability, fixing 94 bugs and tickets. The editor now shows with a highlight where dragging-and-dropping content will put it, and works for any content, not just for images. The citation and reference tools had some minor adjustments to guide the user on how they operate, based on feedback and user testing. A lot of fixes to issues with windows opening and closing, and especially the link editing tool, were made, alongside the save dialog, categories, the language editing tool, table styling, template display and highlights on selected items. The mobile version of VisualEditor, currently available for alpha testers, moved towards release, fixing a number of bugs and improving performance. Work to support languages made some significant gains, and work to support Internet Explorer continued. The new visual interface for writing TemplateData was enabled on the Catalan and Hebrew Wikipedias. The deployed version of the code was updated four times in the regular release cycle (1.24-wmf8, 1.24-wmf9, 1.24-wmf10 and 1.24-wmf11). Parsoid  edit In June, the Parsoid team continued with ongoing bug fixes and bi-weekly deployments; the selective serializer, improving our parsing support for some table-handling edge case, nowiki handling, and parsing performance are some of the areas that saw ongoing work. We began work on supporting language converter markup.

We added CSS styling to the HTML to ensure that Parsoid HTML renders like PHP parser output. We continued to tweak the CSS based on rendering differences we found. We also started work on computing visual diffs based on taking screenshots of rendered output of Parsoid and PHP HTML. This initial proof-of-concept will serve as the basis of more larger scale automated testing and identification of rendering diffs.

The GSoC 2014 LintTrap project saw good progress and a demo LintBridge application was made available on wmflabs with the wikitext issues detected by LintTrap.

We also had our quarterly review this month and contributed to the annual engineering planning process.

Core Features
Flow/Project information  edit  In June, the Flow team finished an architectural re-write for the front-end, so Flow will be easier to keep updating in the future. This will be released to mediawiki.org the first week of July, and Wikipedia the following week.

The new feature in this release is the ability to sort topics on a Flow board. There are now two options for the order that topics appear on the board: you can see the most recently created threads at the top (the default), or the most recently updated threads. This new sorting option makes it easier to find the active conversations on the board.

We've also made a few changes to make Flow discussions easier to read, including: a font size now consistent with other pages; dropdown menus now easier to read; the use of the new button style, and the WikiGlyphs webfont.

Growth
Growth  edit In June, the Growth team completed analysis of its first round of A/B testing of signup invitations for anonymous editors on English, French, German, and Italian Wikipedias. Based on these results, the team prepared a second version to be A/B tested.

Additionally, the team released a major refactor of the GuidedTour extension's API, as well as design enhancements like animations, a new CSS-based way of drawing guider elements, updated button styles, and more. The team also launched GuidedTours on three new Wikipedias: Arabic, Norwegian, and Bengali.

Support
Wikipedia Education Program  edit This month, the Education Program extension again received incremental improvements and bugfixes. Sage Ross of the Wiki Education Foundation submitted two patches: one that adds information to the API for listing students, and another that lets anonymous users compare course versions. Also, a student from Facebook Open Academy fixed a usability issue in the article assignment feature.

Minutes: WMF & Wiki Education Foundation Education Program Extension Sync Up, January 29, 2015
Attendees:

Anna Koval (WMF)

Andrew Russell Green (WMF)

Adam Wight (WMF)

Erica Litrenta (WMF)

Tighe Flanagan (WMF)

Sage Ross (Wiki Ed)

Agenda:

Let's discuss the current status and the future of the education program extension.

Etherpad:

https://etherpad.wikimedia.org/p/education_extension_sync_up_jan_2015

Discusion:

Sage/WikiEd: Just asked for money (how much? from whom?) to turn what they have not into a full-fledged usable replacement for the extension so that in the next fall term WikiEd would not use the extension at all. It calls the API and it's an OAuth app. It would make posts on wiki pages but would not require any changes to mediawiki. http://dashboard-testing.wikiedu.org/ The intention is that others could set up their own instances of it. It runs on Ruby on Rails. And they're getting it set up on wmflabs. So it can be used on other languages and other projects. The hope is to have it in August. Will know more in a couple of weeks if that aggressive ask will be granted.

Andy: At karaoke, Erik proposed a 1-week sprint...

Erica: Will Wiki Ed and WMF coordinate? Or will Andy and Adam be focusing on other things?

Sage: Wiki Ed is not managing the extension. They are moving away from the extension because of all the problems that it has. They are trying to create a system that won't require any development resources from WMF. Campaigns is much broader and more useful than just the education extension and the education use case and it's still a good idea for that work to move forward, in their opinion. Wiki Ed won't likely use it. Because we don't know when it will be done.

Erica: Will what Wiki Ed is building be exported to other education programs?

Sage: In his volunteer time, will document it as much as possible so that program leaders with very little technical skills can implement it on their project. But he cannot manage the deployments.

Andy: Internationalization? i18n contributions to WikiEd's RoR app are welcome. Probably quite easy to do, but I don't know how hard it will be to tie into translatewiki.

Sage: Going to try to make it happen but can't make any promises while wearing his Wiki Ed hat.

Andy: Replacing EPE has always been the plan. I would support replacing EPE with something better. That is the point of Campaigns. <3

Sage is trying not to depend on anything that doesn't exist yet. For example, their app is going to use oauth to make edits on behalf of the users, adding a course badge to their user page, adding the user's name to the course page, etc. Even something like Campaigns could be done this way.

Adam's other notes:

* Getting WMF to install a RoR app is sadly not gonna happen. :(    -- but deploying to wmflabs is fine   :) Is it very impossible? It's fine, but means that we won't have much official support from Operations. Let's ask Dan Duvall, cos he loves Ruby and has introduced some Ruby-based stuff for browser testing.

* If we're wily, the RoR app would tie into the Campaigns API, etc... and we could still benefit from parallel development.

* Killing (no longer maintaining or developing & soon, replacing) EducationProgram extension seems to be on the table. ???     :(

* Question: WikiEducation is using a contractor, WINTR, to do the development, so coordination would probably be coordinated through Sage, via pull requests.

Andy:

WikiEd and WMF alignment:

- programs

- software dev goals

- servers

- collaborations

Andy: The idea that something runs off-wiki doesn't make it not a wmf thing. The fact that Wiki Ed is doing something doesn't mean that WMF can't also use it. It's a laudable goal...

Andy: Per Erik, we can make a proposal for a short sprint, after which we'll reevaluate the outcome. He offhandedly said something like 3 devs, a PM, and insisted that someone from the "community" be involved.

Erica: What he said is if we can make a strong case for it, then he will buy it. We need a team, a MVP and interest!! If we show all that, he doesn't see any reason why he would say no.

Sage said let's do an i18n hackathon on the (hopefully existing by then) off-wiki course page system at Wikimania! Adam said count me in! :)

Adam: Sage? What can we help with? :)

Tighe: My concern is for groups that don't have a support system like Wiki Ed in place. Maybe my team can host an instance of this, it would need to support multiple projects. It's great that Wiki Ed is innovating, but that can't result in a dead extension with no replacement.

Sage: One of the reasons we started building this off-wiki system is because of a conversation with Erik. It would be great if a 1-week spring with a 4-person team could get far enough to get Campaigns to keep track of people in a Campaign.

Andy to Adam: Is it really totally impossible to get WMF Ops to work with RoR??

Adam to Andy: There are people we can talk to... mentioned above: Dan Duvall will be interested...

Erica: Note that for some reason Dan Duvall currently owes me a favor. Bear this in mind if you contact him about Ruby. ;)

Sage: from an ops perspective, Ruby's versioning mess sounds like a bit of a nightmare.

yeah, making it easy for Ruby devs to dive into the ecosystem = WIN

Adam: @sage yeah we would have to host a mirror of the rubygems repos...

+1 to andrew--I want to see a roadmap for WikiEd software as well

https://github.com/WikiEducationFoundation/WikiEduDashboard

https://trello.com/b/dVXtznOh/wiki-edu-roadmapping

http://dashboard-testing.wikiedu.org/

Sage is doing a planning sprint next week with WINTR next week: one outcome if which will be a more specific plan for what we'll be doing in the next year or so.

-- This is the current high level roadmap for Wiki Ed tech plans: https://trello.com/b/dVXtznOh/wiki-edu-roadmapping

Erica: Concerns about internationalization. My team supports lots of languages. language variants and RTL are not trivial. +1 to RTL! <3 Next steps:

So... what do we do when we leave this room?? What are our next steps and who is going to do what?? :)

* WMF talks internally about what we want to do next (what we want to accomplish) and loop back with Sage. 

* WMF takes a look at the internationalization. Try to draft a language engineer, write a blurb describing why we need this person.

* Adam talks to DDuvall about deploying and supporting a RoR app on labs.

* Another WMF meeting in the next week.

* Another meeting in 2 weeks or a month to touch base with Sage on his plans.

-- Create, share and study design docs for upcoming work?

-- Make a design proposal and study frameworks to use (OOjs UI?)

Instructions:

Ed team was told in their QR, if we have any asks for WMF tech support, Lila/Erik/Damon need to know:

1. If we were to put technical resources behind this, what will we be able to do with it (or not do if we don't have it)

2. A rough scope and an estimate of the cost from engineering or product

See also:

BTW: Sage has a proposal for a Wikimania session related to this:

https://wikimania2015.wikimedia.org/wiki/Submissions/Building_wiki_tools_outside_of_Wikimedia_Foundation

2015-02
Status: We're looking at existing work that overlaps with our goals, and trying to integrate with the Wiki Education Foundation's recent work.

Minutes from WMF Education Program Extension Sync Up, Feb 3
Attendees:

Anna Koval (WMF)

Andrew Russell Green (WMF)

Adam Wight (WMF)

Amir Aharoni (WMF)

Erica Litrenta (WMF)

Agenda:

To discuss what we might be able to build in a week, vis a vis a replacement for the education program extension, something that has a much broader application.

To loop in Language Engineering.

Etherpad:

https://etherpad.wikimedia.org/p/education_extension_sync_up_feb_2015

Notes:

AA: Why Ruby on Rails?

AG: The Wiki Education Foundation is developing tools off wiki with an external contractor and their application is on Ruby on Rails, but it is not internationalized. So the idea is to coordinate with what they're doing since we share the same goals. But to do so, we need to see how their app can be internationalized. Maybe it could be set up on labs?

AA: RoR should have some internationalization capability. If the developers are storing the strings in English, they could be put on translatewiki.net for our amazing community of volunteer translators to translate.

EL: Do the extension or whatever replaces it need particular language support?

AA: The extension doesn't need any special internationalization. Don't know about the RoR app yet. Need to see it first. To move forward, need the strings on translatewiki.net. Does it have a user interface? (Yes.) That may need adaptation to RTL.

http://dashboard-testing.wikiedu.org/

AA: It's likely that Translatewiki already can deal with RoR... Siebrand will need full permissions to write on the github...

AW: If there's no glue currently Translatewiki-RoR it should be easy to make. AA also says so.

AA: In addition to translating strings, we'd need maybe to flip some icons and maybe ake some CSS adaptations, generally doesn't seem like a lot of work even so.

EL: On enwiki now, programs from the US/Canada use the Wiki Ed dashboard, but other programs use the old education program extension. Should the Foundation host an instance of their code?

AW: Someone needs to support it and we're probably the best candidate for that. Dan Duvall might be interested in giving us official support.

AG: Another option is that Wiki Ed and WMF share server space and there could be a switcher for which project and which language you want to see  dashboard data for?

WEF is attempting to replace the EducationProgram extension.

Note that the color schemes are different on the resulting course pages.

A larger tech roadmap will help.

Also knowing the education program's non-technical needs for the development of the product.

Need to get the users involved.

Need to know if Wiki Ed has done any user research already.

And was there any user research done by Steven Walling and the Growth group for Editor Campaigns?

Brainstorming about requirements
 * * Pau, Abby (and friends) could do user experience research to gather real needs.


 * * An RfC for community input on the extension?


 * * Need to discuss more internally :)


 * * Bring Campaigns back to life.


 * * Integrate with WikiMetrics to make pulling reports less terrible (AG has done a lot here, for Campaigns).

Steps to add i18n to the WEF app:

* Choose a Ruby i18n library.

* Wrap all strings in the dashboard app, and document the strings for context (like the "qqq" language in MW).

* (WEF) Give Siebrand Mazeland permissions to the dashboard github repo.

* (WEF) Give Amir a link to the repository of the icon and image files used.

* (AA) Look into RTL issues.

* (SM) Create a project on translatewiki.net for the dashboard.

* (SM) Set up synchronization to pull new translations into the dashboard

Other steps:

* (AW) Ask Dan Duvall to support the education dashboard deployment

* (AW) Mirror the Wiki Ed dashboard source repo

* (AW) Reach out to the Gather folks and see if they want to collaborate

* (AW) Hunt down Steven Walling's road map for Campaigns
 * ** https://www.mediawiki.org/wiki/Special:Contributions/Steven_(WMF)

* (??) Find a replacement for Steven Walling
 * ** https://en.wikipedia.org/w/index.php?title=Special%3ACentralAuth&target=Steven+%28WMF%29

Questions:
 * How to keep Campaigns alive
 * Make sure dashboards works on mobile
 * Make sure there's synergy between Campaigns and Gather
 * Do we need to talk with Dan Duvall about Ruby?
 * Do we actually need a language engineer? Was the team contacted?
 * AK: Yes. I've invited Amir Aharoni with messaging input from AW. <3 He will be joining our call. <3
 * AW: my suggestion was that we should ask for a language engineer to help with the Ruby on Rails i18n, but they probably don't need to be involved in our planning process.  We would contact them before doing our hackathon to add i18n to WEF's dashboard application, they can at least point us in the most sane direction.
 * The question is, are there parts of the code or of the program itself which  are right now inadequate because of lack of support for a given language? Or is this something which needs to be resolved at a higher, MediaWiki level? For example, the extension is being used at a couple of  RTL wikis, need to check if it works there or they have demands which weren't met yet.
 * Do we need to look for volunteer devs? (Anna already needs one for a different project what project? - Someone to bring https://www.mediawiki.org/wiki/Automatic_cross-language_screenshots forward, adapting it so that when Edu documentation gets screenshots, they can be updated automatically when needed and taken in multiple languages). AW: My unprofessional opinion is that we should elaborate the most ambitious possible tech roadmap for WEP, then pare it back to a few first steps.  If there is a tech roadmap already, can someone link here?
 * Sage has been updating this page, last edit was April 2014.  Does it still reflect priorities for WEP at WMF?  http://www.mediawiki.org/wiki/Wikipedia_Education_Program See https://trello.com/b/dVXtznOh/wiki-edu-roadmapping and https://github.com/WikiEducationFoundation/WikiEduDashboard instead?
 * The last status on mediawiki is https://www.mediawiki.org/wiki/Wikipedia_Education_Program/status#2014-06-monthly . We should update it?

See also:
 * Related Wikimania 2015 submission: https://wikimania2015.wikimedia.org/wiki/Submissions/Tour_of_the_education_extension

Minutes from Gather - Education Program coordination meeting, Feb 6
Attendees:

AKoval, AGreen, JKatz, AWight

Agenda:

To determine what the overlap might be between our projects!

Notes:

We shared the following material:

Overview of Education Program Extension:

https://outreach.wikimedia.org/wiki/Education/Extension

Overview of the "Campaigns" project:

https://www.mediawiki.org/wiki/Editor_campaigns

An example course with articles, associated editors and reviewers:

http://en.wikipedia.org/wiki/Education_Program:University_of_Mississippi/Mammalogy_%28BISC_350%29_%28Fall_2014%29#Summary_and_students

Metrics dump of coursework results:

http://outreach.wikimedia.org/wiki/Education/Dashboard

List of courses:

http://en.wikipedia.org/wiki/Special:Courses

Staging site for Ruby on Rails dashboard, meant to replace Extension:EducationProgram

http://educationdashboard.wmflabs.org/

Wikimania session submission to discuss using the education extension across the wider movement for non-education outreach:

https://wikimania2015.wikimedia.org/wiki/Submissions/Tour_of_the_education_extension

Gather mockups (unknown URL)

Discussion:
 * It looks like there is great potential for synergy here.  Jon has added the Education Program to his list of stakeholders for Gather work.  We're talking about leaning on Gather as a standalone component that would provide the associations between students and articles, reviewers and articles, and then even higher-order groupings such as course, institution, and country.


 * Gather will be prototyped on mobile web first, future direction is unknown.
 * Education will interface with Gather via the API, and would like to participate in design.
 * Gather collections have a title, which is almost perfect for our workflows.  We might consider asking for concrete collection "type" however, such as  "Reviewing" which would plug into an actual workflow.


 * Education is asking for collections of collections, to allow higher-order views such as course-wide article work.
 * Gather is already thinking about potential for watchlists based on these collections, which would substitute for our activity feed.  Integrating with the MediaWiki core watchlists might be uphill, it might have to be done entirely as an extension.


 * Gather is already planning to include ordering between articles.  I don't know what that would look like for higher-order collections.


 * I accidentally mentioned a possible use case for "automatically maintained" collections, such as grouping everything being worked on by people belonging to a campaign.


 * Another far-fetched idea is to feed calculated metrics back into the collections, e.g. "your class expanded these articles by 50%".

Mobile
Wikimedia Apps  edit The Mobile Apps team released the new Android Wikipedia app and it is now available to be downloaded through the Google Play store on Android devices.

Core features of the app include the ability to save pages for offline reading, a record of your browsing history, and the ability to edit either as a logged in user or anonymously. Therefore the app is the first mobile platform that allows anonymous editing! The app also supports Wikipedia Zero for participating mobile carriers.

Additional work done this month includes the start of implementing night mode for the Android app (by popular demand), creating an onboarding experience which is to be refined and deployed in July, and numerous improvements to the edit workflow. Mobile web projects  edit This month, the mobile web team finished work on styling the mobile site to provide a better experience for tablet users. We began redirecting users on tablets, who had previously been sent to the desktop version of all Wikimedia projects, to the new tablet-optimized mobile site on June 17. Our early data suggests that this change had a positive impact on new user signup and new editor activation numbers. We also continued work on VisualEditor features (the linking and citation dialogs) in preparation for releasing the option to edit via VisualEditor to tablet users in the next three months. Wikipedia Zero <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Wikipedia Zero/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Wikipedia Zero" data-statuspage="Wikipedia Zero/status" data-entrydate="2014-06-monthly">During the last month, the team deployed the refactored Wikipedia Zero codebase that replaces one monolithic extension with multiple extensions. The JsonConfig extension, which allows a wiki-driven JSON configuration system with data validation and a tiered configuration management architecture, had significant enhancements to make it more general for other use cases.

Additionally, the team enabled downsampled thumbnails for a live in-house Wikipedia Zero operator configuration, and finished Wikipedia Zero minimum viable product design and logging polish for the Android and iOS Wikipedia apps. The team also supported the Wikipedia apps development with network connection management enhancements in Android and iOS, with Find in page functionality for Android, and response to Wikipedia for Android Google Play reviews.

The team facilitated discussions on proxy and small screen device optimization, and examined the HTML5 app landscape for the upcoming fiscal year's development roadmap. The team also created documentation for operators for enabling zero-rating with different connection scenarios. Bugfixes were issued for the mobile web Wikipedia Zero and the Wikipedia for Firefox OS app user experience.

Routine pre- and post-launch configuration changes were made to support operator zero-rating, with routine technical assistance provided to operators and the partner management team to help add zero-rating and address anomalies. Finally, the team participated in recruitment for a third Partners engineering teammate.

Wikipedia Zero (partnerships)


 * We launched Wikipedia Zero with Airtel in Bangladesh, our third partner in Bangladesh, and our 34th launched partner overall. We participated in the Wiki Indaba conference, the first event of its kind to be held in Africa. The event, organized by Wikimedia South Africa, brought together community members from Tunisia, Egypt, Ghana, Kenya, Namibia, Nigeria, Ethiopia, Malawi and South Africa. The attendees shared experiences and challenges to work in the region and formulated strategies to support and strengthen the movement's efforts across the continent. While in South Africa, Adele Vrana also met with local operators. Meanwhile, Carolynne Schloeder met with numerous operators and handset manufacturers in India. Carolynne joined Wikimedian RadhaKrishna Arvapally for a presentation at C-DOT, and both participated a blogger event hosted by our partner Aircel, along with other members of Wikimedia India in Bangalore. Smriti Gupta joined the group as Mobile Partnerships Manager, Asia.

Language Engineering
Language tools <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Language tools/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Language tools" data-statuspage="Language tools/status" data-entrydate="2014-06-monthly"> Content translation <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Content translation/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Content translation" data-statuspage="Content translation/status" data-entrydate="2014-06-monthly">The team added support for link adaptation, worked on the infrastructure for machine translation support using Apertium and on hiding templates, images and references that cannot be easily translated. They also prepared for deployment on beta wikis and made multiple bug fixes and design tweaks.
 * The Translate extension received numerous bug fixes, including fixing workflow states transitions for fundraising banners.
 * Changes in input methods: Ludic (lud) transliteration and Tibetian (bo) EWTS layouts added.

MediaWiki Core
HHVM <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="HHVM/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="HHVM" data-statuspage="HHVM/status" data-entrydate="2014-06-monthly">The team has been running HHVM on a single test machine ("osmium") for the purpose of testing the job queue in production. The machine is only put into production on a very limited basis, while enough bugs are found to keep the team busy for a while, and then it's disabled again as the team fixes those bugs. We're planning on having HHVM running on a few job runner machines (continually) in July, then turning our focus toward running HHVM on the main application servers, taking a similar strategy. Wikimedia Release and QA Team <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Wikimedia Release and QA Team/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Wikimedia Release and QA Team" data-statuspage="Wikimedia Release and QA Team/status" data-entrydate="2014-06-monthly">The Release and QA Team had their mid-quarter check-in on June 27. Phabricator work is progressing nicely. The latest MediaWiki tarball release (1.23) was made and the second RFP started and is close to completion. We are moving to only WMF-hosted Jenkins for all jobs, and we are working with the MediaWiki Core and the Operations teams on HHVM-related integration (both for deployment and for the Beta Cluster). Admin tools development <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Admin tools development/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Admin tools development" data-statuspage="Admin tools development/status" data-entrydate="2014-06-monthly">Work on this project is currently being completed along with the SUL finalisation project, including the global rename tool (bug 14862) and cleaning up the CentralAuth database (bug 66535). Search <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Search/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Search" data-statuspage="Search/status" data-entrydate="2014-06-monthly">CirrusSearch is running as the default search engine on all but the highest traffic wikis at this point. Nik Everett and Chad Horohoe plan to migrate most of the remaining wikis in July, leaving only the German and English Wikipedia to migrate in August. Auth systems <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Auth systems/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Auth systems" data-statuspage="Auth systems/status" data-entrydate="2014-06-monthly">Continued work on the SOA Authentication RFC and Phabricator OAuth integration. We made OAuth compatible with HHVM and made other minor bug fixes. SUL finalisation <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="SUL finalisation/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="SUL finalisation" data-statuspage="SUL finalisation/status" data-entrydate="2014-06-monthly">The MediaWiki Core team has committed to having the following work completed by the end of September 2014: Security auditing and response <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Security auditing and response/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Security auditing and response" data-statuspage="Security auditing and response/status" data-entrydate="2014-06-monthly">We released MediaWiki 1.23.1 to prevent multiple issues caused by loading external SVG resources. We also performed security reviews of the Wikidata property suggester, Extension:Mantle for mobile/Flow, and Flow's templating rewrite.
 * Completing the necessary engineering work to carry out the finalisation.
 * Setting a date on which the finalisation will occur (Note: this date may be after September).
 * Have a communications strategy in place, and community liaisons to carry that out, for the time period between the announcement of the date of the finalisation and the finalisation proper.

Quality assurance
Quality Assurance <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Quality Assurance/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Quality Assurance" data-statuspage="Quality Assurance/status" data-entrydate="2014-06-monthly">This month saw significant improvements to the MediaWiki-Vagrant development environments from new WMF staff member Dan Duvall. We have completed support for running the full suite of browser tests on a Vagrant instance under the VisualEditor role. In the near future, we will extend that support to the MobileFrontend and Flow Vagrant roles, as well as making general improvements to Vagrant overall. Another great QA project is from Google Summer of Code intern Vikas Yaligar, who is using the browser test framework to automate taking screen captures of aspects of VisualEditor (or any other feature) in many different languages, for the purpose of documentation and translation. Quality Assurance/Browser testing <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Quality Assurance/Browser testing/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Quality Assurance/Browser testing" data-statuspage="Quality Assurance/Browser testing/status" data-entrydate="2014-06-monthly">After two years of using a third-party host to run browser test builds in Jenkins, this month we have completed the migration of those builds to Jenkins hosted by the Wikimedia Foundation. Hosting our browser test builds ourselves gives us more control over every aspect of running the browser tests, as well as the potential to run them faster than previously possible. Particular thanks to Antoine Musso, whose work made it possible. Simultaneously, we have also ported all of the remaining tests from the  repository either to   or to their relevant extension. This gives us the ability to package browser-based acceptance tests with the release of MediaWiki itself. After more than two years evolving the browser testing framework across WMF, the  repository is retired, and all if its functions now reside in the repositories of the features being tested.

Multimedia
Multimedia <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Multimedia/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Multimedia" data-statuspage="Multimedia/status" data-entrydate="2014-06-monthly">In June, the multimedia team released Media Viewer v0.2 on all Wikimedia wikis, with over 20 million image views per day on sites we track. Global feedback was generally positive and helped surface a range of issues, many of which were addressed quickly. Based on this feedback, Gilles Dubuc, Mark Holmquist, and Gergő Tisza developed a number of new features, with designs by Pau Giner: view images in full resolution, view images in different sizes, show more image information, edit image file pages, as well as easy disable tools for anonymous users and editors.

This month, we started working on the Structured Data project with the Wikidata team, to implement machine-readable data on Wikimedia Commons. We are now in a planning phase and aim to start development in Fall. We ramped up our work on UploadWizard, reviewed user feedback, collected metrics, fixed bugs and started code refactoring, with the help of contract engineer Neil Kandalgaonkar. We also kept working on technical debt and bug fixes for other multimedia tools, such as image scalers, GWToolset and TimedMediaHandler, with the help of Summer contractor Brian Wolff.

As product manager, Fabrice Florin helped plan our next steps, hosting a planning meeting and other discussions of our development goals, and led an extensive review of user feedback for Media Viewer and UploadWizard with new researcher Abbey Ripstra. Community liaison Keegan Peterzell introduced Media Viewer and responded to user comments throughout the product's worldwide release. To learn more about our work, we invite you to join our discussions on the multimedia mailing list.

Engineering Community Team
Bug management <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Bug management/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Bug management" data-statuspage="Bug management/status" data-entrydate="2014-06-monthly">Apart from gruntwork (handling new tickets; prioritizing tickets; pinging on older tickets) and Andre's main focus on Phabricator, Parent5446, Krinkle and Andre created several requested Bugzilla components, plus moved 'MediaWiki skins' to a Bugzilla product of their own. In Bugzilla's codebase, Tony and TTO styled Bugzilla's Alias field differently, Tony removed the padlock icons for https links in Bugzilla and cleaned up the codebase, and Odder fixed a small glitch in Bugzilla's Weekly Summary and rendering of custom queries on the Bugzilla frontpage. Numerous older tickets with high priority were triaged on a bugday. Phabricator/Migration <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Phabricator/Migration/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Phabricator/Migration" data-statuspage="Phabricator/Migration/status" data-entrydate="2014-06-monthly">Apart from discussions on how to implement certain functionality and settings in Phabricator among team members and stakeholders, Mukunda implemented a MediaWiki OAuth provider in Phabricator (Gerrit changes: 1, 2; related ticket) and Chase created a Puppet module for Phabricator. Mentorship programs <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Mentorship programs/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Mentorship programs" data-statuspage="Mentorship programs/status" data-entrydate="2014-06-monthly">Google Summer of Code and FOSS Outreach Program for Women interns and mentors evaluated each other as part of the mid-term evaluations. Reports are available for all projects: Technical communications <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Technical communications/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Technical communications" data-statuspage="Technical communications/status" data-entrydate="2014-06-monthly">In addition to ongoing communications support for the engineering staff, Guillaume Paumier focused on information architecture of Wikimedia engineering activities. This notably involved reorganizing the Wikimedia Engineering portal (now linked from mediawiki.org's sidebar) and creating a status dashboard that lists the status of all current activities hosted on mediawiki.org. The portal is now also cross-linked with the other main tech spaces (like Tech and Tech News) and team hubs. Volunteer coordination and outreach <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Volunteer coordination and outreach/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Volunteer coordination and outreach" data-statuspage="Volunteer coordination and outreach/status" data-entrydate="2014-06-monthly">Volunteers and staff are beginning to add or express interest in topics for the 2014 Wikimania Hackathon in London. The WMUK team is working hard to finalize venue logistics so that we can schedule talks and sessions in specific rooms. Everything is on track for a successful (and very large!) Hackathon. Tech Talks held in June: How, What, Why of WikiFont on June 12 and A Few Python Tips on June 19. A new process has been set up for volunteers needing to sign an NDA in order to be granted special permissions in Wikimedia servers. On a similar note, we have started a project to implement a Trusted User Tool in Phabricator, in order to register editors of Wikimedia projects that have been granted special permissions after signing a community agreement. Architecture process <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Architecture process/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Architecture process" data-statuspage="Architecture process/status" data-entrydate="2014-06-monthly">Developers had several meetings on IRC about architectural issues or Requests for comment:
 * Tools for mass migration of legacy translated wiki content
 * Wikidata annotation tool
 * Email bounce handling to MediaWiki with VERP
 * Google Books, Internet Archive, Commons upload cycle
 * UniversalLanguageSelector fonts for Chinese wikis
 * MassMessage page input list improvements
 * Book management in Wikibooks/Wikisource
 * Parsoid-based online-detection of broken wikitext
 * Usability improvements for the Translate extension
 * A modern, scalable and attractive skin for MediaWiki
 * Automatic cross-language screenshots for user documentation
 * Separating skins from core MediaWiki
 * Chemical Markup support for Wikimedia Commons
 * Improving URL citations on Wikimedia
 * Historical OpenStreetMap
 * Welcoming new contributors to Wikimedia Labs and Tool Labs
 * Evaluating, documenting, and improving MediaWiki web API client libraries
 * Feed the Gnomes - Wikidata Outreach
 * Template Matching for RDFIO
 * Switching Semantic Forms Autocompletion to Select2
 * Catalogue for Mediawiki Extensions
 * Generic, efficient localisation update service.


 * 2014-06-02 — Discussion of Requests for comment/Grid system;
 * 2014-06-11 — Discussion of several RfCs (lightning round);
 * 2014-06-13 — Discussion of security guidelines draft;
 * 2014-06-20 — Discussion of revamping MediaWiki's skin systems: Trevor Parscal's "Redo skin framework" and Bartosz Dziewoński's "Separating skins from core MediaWiki" work;
 * 2014-06-25 — Discussion of front-end and UX standardization.

Analytics
Analytics/Wikimetrics <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Analytics/Wikimetrics/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Analytics/Wikimetrics" data-statuspage="Analytics/Wikimetrics/status" data-entrydate="2014-06-monthly">To support Editor Engagement Vital Signs, the team has implemented a new metric: Newly Registered User. There is also a new backup system to preserve user's reports on cohorts as well as the ability to tag cohorts. A number of bugs have been fixed, including fixing the first run of a recurrent report and preventing the creation of reports with invalid cohorts. Analytics/Data Processing <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Analytics/Data Processing/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Analytics/Data Processing" data-statuspage="Analytics/Data Processing/status" data-entrydate="2014-06-monthly">The team has now integrated Data Processing as part of its Development Process. New Stories/Features have been identified and tasked. Also, experimentation with Cloudera Hadoop 5 is complete and we are ready to upgrade the cluster in July. Analytics/Editor Engagement Vital Signs <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Analytics/Editor Engagement Vital Signs/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Analytics/Editor Engagement Vital Signs" data-statuspage="Analytics/Editor Engagement Vital Signs/status" data-entrydate="2014-06-monthly">The ability to run a metric over an entire project (wiki) in Wikimetrics drives us closer to producing data daily for our first Vital Sign. The team has also iterated on the design of the dashboard and navigation. We added a requirement from executives to have a default view when EEVS is loaded. This view would display metrics for the 7 largest Wikipedias. Analytics/EventLogging <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Analytics/EventLogging/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Analytics/EventLogging" data-statuspage="Analytics/EventLogging/status" data-entrydate="2014-06-monthly">We fixed a serious bug where cookie data was getting captured in the country column. Saved data was scrubbed of the unwanted information and some old and unused tables were dropped. The team also implemented Throughput Monitoring to help catch potential issues in EventLogging. Analytics/Research and Data <span class="plainlinks noprint mw-statushelper-editlink" style="margin: 0 0 0 1em; font-size:80%; background:#e4e4e4;" data-statuspage="Analytics/Research and Data/status" data-entrydate="2014-06-monthly"> edit <div style="margin: 0 0 0 2em;" class="mw-statushelper-entry" id="Analytics/Research and Data" data-statuspage="Analytics/Research and Data/status" data-entrydate="2014-06-monthly">This month we refined the Editor Model – a proposal to model the main drivers of monthly active editors – and expanded the documentation of the corresponding metric definitions. We applied this model to teams designing editor engagement features (Growth, Mobile) and supported them in setting targets for the next fiscal year.

We analyzed the early impact of the tablet desktop-to-mobile switchover on traffic, |total edit volume, unique editors, and new editor activation.

We hosted the June 2014 edition of the research showcase with two presentations on the effect of early socialization strategies and on predictive modeling of editor retention.

We released wikiclass, a library for performing automated quality assessment of Wikipedia articles.

We released longitudinal data on the daily edit volume for all wikis with VisualEditor enabled, since the original rollout.

We continued work on an updated definition for PageViews.

Finally, we held our quarterly review (Q4-2014) and presented our goals for the next quarter (Q1-2015).

Wikidata
The Wikidata project is funded and executed by Wikimedia Deutschland.


 * The team worked on fixing bugs as well as a number of features. These include data access for Wikiquote, support for redirects, the monolingual text datatype as well as further work on queries. Interface messages where reworked to make them easier to understand. First mockups of the new interface design have been published for comments. The entity suggester a team of students worked on over the last months has been deployed. This makes it easier to add new statements by suggesting what kind of statements are missing on an item. Wikidata the Game has been extended by Magnus by 2 games to add date of birth and date of death to people as well as to add missing images.

Future

 * The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.