The team continued to heavily refactor the Puppet configuration: manifests for MySQL, nginx, SSH, puppetmaster, and several others are now properly organized as modules. Alexandros Kosiaris has made considerable progress towards supporting multiple puppetmasters on our cluster, which will greatly improve Puppet performance.
Andrew Bogott has been working with Yuvaraj Pandian to get the new proxy system properly deployed; this will greatly reduce our need to hand out public IPs to labs users. We're close to hiring a contractor to help with the upcoming migration of Labs from Tampa to Ashburn.
In October, the VisualEditor team continued to improve the stability and performance of the system, and add new features. The deployed version of the code was updated five times (1.22-wmf20, 1.22-wmf21, 1.22-wmf22, 1.23-wmf1 and 1.23-wmf2). Beyond fixing bugs, the focus of the team's work this month was to make a number of large structural changes to make the system more dependable and extensible, and continue to make some usability improvements. For example, you now need to press the "delete" key twice to delete a template, reference or image; the first time, they only become selected, to avoid accidental deletion of infoboxes and similar content. A new feature, empowering users to switch from editing in VisualEditor to editing wikitext directly without having to save the page, was also implemented.
In October, the Parsoid team continued to refine the parser behavior in edge cases. Performance was improved by increasing the parallelism of API requests and separating page updates from dependency-triggered updates in the job queue. The round-trip testing server performance was improved so that we can now run round-trip tests on 160,000 pages over night. Support for private wikis was also added this month.
We also made additional progress on Rashomon, the revision storage service based on Apache Cassandra. Rashomon is initially going to be used for implementing HTML and metadata storage for Parsoid output. Rashomon was deployed on a test cluster and import/write tests were performed.
In October, we released Notifications worldwide on about 800 different sites, including most of the remaining Wikipedia wikis and 'sister projects'. Fabrice Florin and Keegan Peterzell managed the community outreach for these final releases, while Benny Situ proceeded with the technical deployments. Community members we've spoken to generally find this tool helpful, across languages and world regions. Notifications are also available on mobile devices, and seem to be adding value on these platforms as well. We now plan a final release of Notifications on the German and Italian editions of Wikipedia at the end of November. To learn more, visit our project hub, read the help page and join the discussion on the talk page.
In October, the Flow team implemented a new visual design treatment on the Flow prototype (currently hosted on WMF Labs), and we continued to work through the set of features needed for a minimum viable product (MVP) on Wikipedia. We held an in-office User Experience workshop, primarily with users new to Wikipedia, to get feedback on the usability of the new design. We're planning to demo the Flow MVP to interested WikiProjects in November to get more feedback on what's needed for a first on-Wikipedia trial.
The fundraising team has been steadily ramping up to the big English fundraiser, and so the fundraising tech team has been transitioning into their usual year-end fundraising monitoring and support role. At the end of October, we went in to a feature freeze for CentralNotice, and have been concentrating on bugfixes with the intent of increasing stability on the payments cluster and in the rest of the donation pipeline.
We have also started planning for the international fundraising campaigns targeted for early 2014.
This month, we fixed several bugs in the Education Program extension and added one much-needed UI feature. More improvements are in the pipeline. For now, we're focusing on issues that are not linked to the extension's lack of integration with ContentHandler, and on features that might carry over to a new version of the extension. We've also begun the groundwork for redesigning the UX and broadening the extension's scope.
Many changes were made to the iOS Commons app in October. Users can now select the license among the most common ones, and swipe to delete categories on yet-to-be-uploaded images. A number of visual and interface improvements have been made, including on the settings page and the image details slider, notably for iOS 7 compatibility. Icon consistency has been improved throughout the app. The app now also has better internationalization support (via Autolayout), as well as better landscape and iPad support.
We started redirecting ineligible access attempts against subdomains zero.wikipedia.org to a unified warning page; this will reduce residual content in search engine caches, and reduce charges for users accidentally visiting zero.wikipedia.org subdomains while off participating networks.
We added support for partner self-management of configuration for JSON-oriented users; this will be enhanced with stage gating before any actual push to production.
Last, we nearly finalized IP address zero-rating lists (with the support of the Operations team), and we've also started working on Wikipedia Zero support for all Wikimedia projects.
The mobile web team has been focusing on a variety of new features, as well as running tests and gathering metrics on new and new-ish mobile editors. The team has also been working on general design enhancements to improve the user experience and make the look and feel consistent across all aspects of the mobile experience.
Feature highlights from the past month include: a better AbuseFilter support for mobile editing, early experimentation around integrating VisualEditor with MobileFrontend for tablets, client-side performance enhancements, the ability to 'thank' from the watchlist, and Captcha support for mobile editing.
In beta, 'Near this page' is now available, as well as user profiles and 'Keep Going' (a series of calls to action for new editors).
The Language Engineering team worked on building a new Main Page for translatewiki.net, the primary translation platform for MediaWiki projects. This includes a new sign-in screen and project displays. A special feature is also being introduced for new translators through which they can provide a limited set of translations for evaluation.
As part of the enhancements towards Universal Language Selector (ULS), a new font has been developed for displaying Language Names in their own scripts. This is known as the Autonym font and contains only the characters needed for the language names. Significantly small in size, this font will soon be deployed on Wikimedia projects. The team also worked on fixing a font inheritance issue for the edit area (bug 53734).
The monthly release of the MediaWiki Language Extension Bundle (MLEB) was also completed.
The team prepared for the open-source language summit to be held in Pune, India on November 18−19 in collaboration with Red Hat and other open-source internationalization developers.
The last deployment from the 1.22 MediaWiki release was on October 24. To see all the changes that occurred during the 1.22 development cycle, see the main 1.22 release page or the RELEASE NOTES file. Final release of MediaWiki 1.22 is expected to be November 29.
We implemented logging, aggregation & graphing of the VisualEditor DOM load & save timing. We also rolled out mw.inspect, a library for inspecting static asset metrics. We configured stable URLs and improved cache headers for font resources, and rolled out a localStorage module caching to test wikis and the beta cluster.
In October, CirrusSearch was deployed as a secondary search engine to Wikidata, all Wikivoyage wikis, and Wikipedia in Bengali. It became the primary search engine on Wiktionary in Italian, Wikipedia in Catalan and Wikisource in English. In November, we plan to deploy many more wikis including some larger than the Catalan Wikipedia. To expand to those larger wikis, we've negotiated some new hardware that should be deployed mid month.
We continued to refine the OAuth UX with the design team, and completed all major development tasks for the initial OAuth product. The first third-party application approved to use OAuth, "Gerrit Patch Uploader", was successfully used by several end users. We plan to finish the OAuth deployment in November.
In October the Quality Assurance project brought on a new software tester for VisualEditor, Rummana Yasmeen, who became productive quickly, reporting a number of issues and validating a VE deployment. QA spent a significant amount of time with the WMF Language team, using automated browser tests to find issues with the UniversalLanguageSelector and other Language software. QA continues to expand the utility of the beta labs test environment, supporting the new Flow extension there, and increasing the role of beta for deployments. Membership in the QA mailing list continues to grow, and October saw more contributions to the automated tests from volunteers. QA will participate in the Google Code In project as well.
October has been dedicated to consolidating the Jenkins configuration to make it easier to edit. Most actions are now handled by shell scripts under integration/jenkins.git:/bin; editing the scripts doesn't require updating Jenkins jobs. The second slave server has been added to production and is successfully running PHPUnit tests. The packaging of dependencies required to upgrade Zuul has been completed, and Antoine Musso now has a version working in Labs. Finally, we investigated the possibility of running the browser tests whenever a change is submitted in Gerrit; that work is still in progress.
Thanks to Carl Fürstenberg's work during the Summer, we are now able to build some Debian packages straight into Jenkins using a dedicated instance and the Jenkins Debian glue scripts. The jobs are listed in Jenkins under the Ops-DebGlue view.
October saw a major change to the architecture of the browser tests. Since they are now in many different repositories (VisualEditor, UniversalLanguageSelector, MobileFrontend, Flow, etc.), we consolidated the code shared among the various tests in all the repositories into a single library. This allows all the tests, present and future, to use the best and most up-to-date infrastructure. As for the tests themselves, we expanded coverage and identified issues for VisualEditor, Flow, gadgets, Language features, MobileFrontend, and more.
Bugzilla now offers a new guided bug entry form which will make creating good bug reports easier for newcomers. Bugzilla now also displays metadata changes of a bug report inline for all logged-in users, so they can see in the comments who changed a value of a field (without clicking on "History"). Daniel Zahn upgraded Wikimedia Bugzilla to latest version 4.2.7. Legoktm mass-imported about 400 Pywikibot tickets from Sourceforge to Bugzilla. On a related note, Amir ran a PyWikibot Bug Triage resulting in nearly 100 tickets receiving updates. Furthermore, Andre Klapper investigated Wikimedia Bugzilla's customizations in CSS and code in order to clean up and sync with the upstream code base, to simplify current maintenance and also make a potential future upgrade of Wikimedia Bugzilla from version 4.2 to 4.4 easier.
In October, we continued to expand our multimedia team and hired Gergő Tisza as software engineer. Mark Holmquist worked with Gergő to develop a first beta version of the Media Viewer, which displays images in larger size or full screen; this work was based on designs by Pau Giner, May Tee-Galloway and Jared Zimmerman.
We also completed development on the Beta Features program, which invites users to try out new features before they are released for everyone. A first version of both products is now ready for testing by logged in users on MediaWiki.org (to try new features, click on the small 'Beta' link next to your 'Preferences').
We plan to release the Beta Features program in coming days to Wikimedia Commons and Meta-Wiki, then to all wikis worldwide at the end of November. Fabrice Florin managed the development of both projects, and updated our multimedia plans to prepare for roundtable discussions with community members next month.
Bryan Davis started work on improving the thumbnail pipeline and guided the development of an upcoming GLAM Toolset for batch uploads by museum curators. To discuss these features and keep up with our work, we invite you to join the new multimedia mailing list. We are also recruiting for a senior software engineer position on our team.
The team designed and implemented multiple data center configuration support for Kafka (message bus). A bug involving buffer space allocation was exposed and fixed in the Varnish module, and infrastructure work done on automated data ingestion and partitioning. Product Analyst Oliver Keyes did some exploratory work with Hive and provided feedback to the Development team on ease of use and use cases.
This month, we continued to support Growth and Mobile as the team's focus areas for this quarter. We published the results of the latest GettingStarted test run by the Growth team, we completed the cohort analysis for Mobile user acquisition and we worked with the Mobile team to prepare the launch a new test for new user activation, currently underway.
We analyzed active editor trends to determine whether the Wikipedia total active editors data for September 2013 (~67k) represented an anomalous change from seasonality and the long-term trend, and concluded that this was not the case. The results of this analysis call for the need to apply time series analysis and forecasting methods to other key performance indicators that the Foundation publishes on a daily or monthly basis.
We continued to work with the analytics engineers to provide requirements for Wikimetrics (with a particular focus on UserMetrics feature parity), and to perform data QA and validate the output of the application for metrics that were recently implemented.
We completed a round of consultations with internal stakeholders to identify research needs of each team in the organization and determine their priority. We presented a review of our activities for Q1 and plans for Q2 at the Analytics Quarterly meeting. We identified "metric standardization" as one of the goals the team will focus on in this quarter.
We organized and announced the inaugural Wiki Research Hackathon, a global event hosted in 8 locations in 5 countries, bringing together Wikimedia researchers, academics and community members to work on wiki research projects. The hackathon — the first event organized in the context of the Labs2 initiative — will take place on November 9, 2013.
We have released a new version (1.4) of Kiwix for Android; besides other improvements, we have introduced a localized user interface. We continue our efforts to speed-up the ZIM throughput; Wikipedia is now available offline in more than 60 languages. The last flash drives from our Deaddrops & Geocaching pilot were sent; we hope to get them walled up before the end of the year. We have also submitted a few proposals about Kiwix for Android for the Google Code-In program.
The engineering management team continues to update the Deployments page weekly, providing up-to-date information on the upcoming deployments to Wikimedia sites, as well as the annual goals, listing ongoing and future Wikimedia engineering efforts.