Wikimedia Features engineering
- 1 Current projects
- 2 Future and backburner projects
- 3 2015-01
- 4 2015-02
- 5 Recently completed projects
- 6 Team meetings
§Current projects[edit | edit source]
|VisualEditor||Creating a visual editor for MediaWiki, other platforms and the Web at large||Team James Forrester, Trevor Parscal, Editing:|
In November, the team working on VisualEditor introduced table structure editing, improved some existing features, and fixed over 100 tasks, bugs and requests.
You can now edit the structure of a table, adding or deleting rows and columns and various other common tasks like merging cells and using captions. VisualEditor now support keyboard shortcuts like entering "
The toolbar's menus in VisualEditor now open in a collapsed, short form with uncommon tools only shown when requested. You can now create and edit simple "blockquoted" paragraphs for indenting. You can now use a basic editor for gallery and hieroglyphic blocks on the page. Category editing was enhanced in a few ways, including adding a redirect category now adds its target, and making categories without a description page show as red. We improved compatibility with some variations of how wikis use the Flagged Revisions system. Armenian language users now get customised bold and italic icons in the toolbar; if your language would benefit from customised icons, please contact us.1.25-wmf7, 1.25-wmf8, 1.25-wmf9 and 1.25-wmf10).
|Editing||Extending and improving the editing tools used at Wikimedia||Team James Forrester, Roan Kattouw, Trevor Parscal, Across various teams:
|Parsoid||A bidirectional runtime wikitext parser. Converts back and forth between wikitext and HTML/XML DOM with RDFa.||Team Subramanya Sastry, C. Scott Ananian, Arlo Breault, Marc Ordinas i Llopis:|
In December, we finished work supporting templates that generate attributes of a table
as well as content of the table and do not fit well within the DOM-based model that Parsoid works with. Besides that, we improved error reporting when handling images that lets clients like Flow and VE handle them better. With a view to reducing the HTML size that needs to be loaded and parsed by clients, we stripped the private data-parsoid attribute from templated content since it is unnecessary. We also continued with code cleanup and pay back technical debt. Specifically, we did a bunch of fixes in our nowiki handling when HTML is serialized to wikitext. We improved robustness, correctness, and reduced the number of scenarios where nowikis are needed for quotes. We also made it simpler to detect nowiki scenarios for other wikitext constructs and specifically appliedit to links of all flavors.
|Services and REST API||Team implementing backend services and a REST-style content API as a step towards a service-oriented architecture.||Team Gabriel Wicke, Hardik Juneja, James Earl Douglas, Marko Obrovac:|
September saw a lot of activity on the RESTBase storage and API service. A new 'pagecontent' composite bucket type using revisioned blob buckets was introduced. This uses the by-now fairly rich table storage backend to provide functionality similar to MediaWiki's revision table, and supports any number of revisioned types of content (like HTML, wikitext, JSON metadata) associated with each revision.
Work on secondary index updates continued at full steam, and is now close to being merged.
|Flow||User-to-user communication and collaboration.||Team Danny Horn, Erik Bernhardson, Matthias Mullie, S Page, Matt Flaschen, Quiddity:|
In December, the Collaboration team completed work on the first iteration of Flow's Table of Contents feature, for release in early January. Catalan Wikipedia now uses Flow on their new Village Pump/Technical page, and we're working on feature requests that will get Flow ready for further rollout on both Catalan and French WP.
|Fundraising tech||Technical work supporting the Wikimedia Foundation's fundraising efforts||Team Katie Horn:|
New activity page was started
§Future and backburner projects[edit | edit source]
|Article Creation Workflow||A feature to improve the article creation workflow for new editors.||Team Fabrice Florin, Benny Situ, Ryan Kaldari, Ian Baker, Brandon Harris, Oliver Keyes, Howie Fung, Dario Taraborelli, Terry Chay:|
Fabrice Florin, Benny Situ, Ryan Kaldari, Oliver Keyes and Brandon Harris have put this Article Creation feature on hold, in order to make more progress on the New Pages Feed project (code-named Page Triage). We are not comfortable deploying this feature until the New Pages Feed is released, because it may create more work for page patrollers. The current ACW prototype is available for testing on Wikimedia Labs.
|GlobalProfile||A set of features to help users provide information about themselves and associate by interests.
|Team Alolita Sharma, Brandon Harris:|
This project was on hold in September, because of a lack of development resources.
|Community prototyping||Features development to support community research and community change experiments.
|Team Alolita Sharma, Katie Horn:|
In December, active development has begun on the user metrics API project in E3. The effort is threefold: to standardize user metrics , to build infrastructure to efficiently compute metrics for a large set of users [2,3], and finally to expose those results via an API. The technical portion of this project is written in Python and Flask, and the assigned resources are Dario Taraborelli and Ryan Faulkner. Development and documentation efforts will be continuing through January. https://gerrit.wikimedia.org/r/gitweb?p=analytics/E3Analysis.git;a=summary
|LiquidThreads 3.0||A rewrite of the LiquidThreads extension to improve its back-end and UI||Team James Forrester, Andrew Garrett, Brandon Harris,:|
"Further deployments of LiquidThreads are indefinitely on hold because we don't have the resources to support it." – Andrew Garrett (2012-03-19) there is no further development expected.
|Etherpad lite integration||Create a project page for this activity (how-to)|
|MoodBar||A feature to encourage new users to provide feedback.
|Team Alolita Sharma, Brandon Harris, Howie Fung, Ian Baker), Rob Moen, Benny Situ:|
Benny Situ and Rob Moen familiarized themselves with the existing code, and the requirements for phase 2. Benny also added server-side validation for empty comment, and Rob fixed Internet Explorer bugs and added front end validation. On November 30th, there was a follow up deployment that added a preview mode to feedback response and also allowed the use of wikitext. Additionally, there was an update to the response template used in the user talk page. Dario Taraborelli uploaded a series of real-time dashboards for Moodbar data to the Toolserver.
|Mobile/Athena||Create a project page for this activity (how-to)|
|FlaggedRevs maintenance||A feature to allow changes made by logged-out and new users to be reviewed before they appear as the primary version of an article.||Team James Forrester, Matthias Mullie, Aaron Schulz:|
No update this month.
|MicroDesign||Low hanging fruit||Team Howie Fung, Oliver Keyes, Munaf Assaf, Vibha Bamba, Rob Moen, Benny Situ:|
The Agora extension moves ever-closer to completion, with help from Munaf Assaf, Trevor Parscal, Rob Moen and Vibha Bamba. Several templates on the English-language Wikipedia have been redesigned to reduce interface clutter, with some already implemented.
|Wikipedia Education Program||A MediaWiki extension to support the Wikipedia Education Program||Team Terry Chay, Andrew Russell Green, Sage Ross, Jeroen De Dauw (original author):|
This month, the Education Program extension again received incremental improvements and bugfixes. Sage Ross of the Wiki Education Foundation submitted two patches: one that adds information to the API for listing students, and another that lets anonymous users compare course versions. Also, a student from Facebook Open Academy fixed a usability issue in the article assignment feature.
§2015-01[edit | edit source]
§Minutes: WMF & Wiki Education Foundation Education Program Extension Sync Up, January 29, 2015[edit | edit source]
Anna Koval (WMF)
Andrew Russell Green (WMF)
Adam Wight (WMF)
Erica Litrenta (WMF)
Tighe Flanagan (WMF)
Sage Ross (Wiki Ed)
Let's discuss the current status and the future of the education program extension.
Sage/WikiEd: Just asked for money (how much? from whom?) to turn what they have not into a full-fledged usable replacement for the extension so that in the next fall term WikiEd would not use the extension at all. It calls the API and it's an OAuth app. It would make posts on wiki pages but would not require any changes to mediawiki. http://dashboard-testing.wikiedu.org/ The intention is that others could set up their own instances of it. It runs on Ruby on Rails. And they're getting it set up on wmflabs. So it can be used on other languages and other projects. The hope is to have it in August. Will know more in a couple of weeks if that aggressive ask will be granted.
Andy: At karaoke, Erik proposed a 1-week sprint...
Erica: Will Wiki Ed and WMF coordinate? Or will Andy and Adam be focusing on other things?
Sage: Wiki Ed is not managing the extension. They are moving away from the extension because of all the problems that it has. They are trying to create a system that won't require any development resources from WMF. Campaigns is much broader and more useful than just the education extension and the education use case and it's still a good idea for that work to move forward, in their opinion. Wiki Ed won't likely use it. Because we don't know when it will be done.
Erica: Will what Wiki Ed is building be exported to other education programs?
Sage: In his volunteer time, will document it as much as possible so that program leaders with very little technical skills can implement it on their project. But he cannot manage the deployments.
Andy: Internationalization? i18n contributions to WikiEd's RoR app are welcome. Probably quite easy to do, but I don't know how hard it will be to tie into translatewiki.
Sage: Going to try to make it happen but can't make any promises while wearing his Wiki Ed hat.
Andy: Replacing EPE has always been the plan. I would support replacing EPE with something better. That is the point of Campaigns. <3
Sage is trying not to depend on anything that doesn't exist yet. For example, their app is going to use oauth to make edits on behalf of the users, adding a course badge to their user page, adding the user's name to the course page, etc. Even something like Campaigns could be done this way.
Adam's other notes:
* Getting WMF to install a RoR app is sadly not gonna happen. :( -- but deploying to wmflabs is fine :) Is it very impossible? It's fine, but means that we won't have much official support from Operations. Let's ask Dan Duvall, cos he loves Ruby and has introduced some Ruby-based stuff for browser testing.
* If we're wily, the RoR app would tie into the Campaigns API, etc... and we could still benefit from parallel development.
* Killing (no longer maintaining or developing & soon, replacing) EducationProgram extension seems to be on the table. ??? :(
* Question: WikiEducation is using a contractor, WINTR, to do the development, so coordination would probably be coordinated through Sage, via pull requests.
WikiEd and WMF alignment:
- software dev goals
Andy: The idea that something runs off-wiki doesn't make it not a wmf thing. The fact that Wiki Ed is doing something doesn't mean that WMF can't also use it. It's a laudable goal...
Andy: Per Erik, we can make a proposal for a short sprint, after which we'll reevaluate the outcome. He offhandedly said something like 3 devs, a PM, and insisted that someone from the "community" be involved.
Erica: What he said is if we can make a strong case for it, then he will buy it. We need a team, a MVP and interest!! If we show all that, he doesn't see any reason why he would say no.
Sage said let's do an i18n hackathon on the (hopefully existing by then) off-wiki course page system at Wikimania! Adam said count me in! :)
Adam: Sage? What can we help with? :)
Tighe: My concern is for groups that don't have a support system like Wiki Ed in place. Maybe my team can host an instance of this, it would need to support multiple projects. It's great that Wiki Ed is innovating, but that can't result in a dead extension with no replacement.
Sage: One of the reasons we started building this off-wiki system is because of a conversation with Erik. It would be great if a 1-week spring with a 4-person team could get far enough to get Campaigns to keep track of people in a Campaign.
Andy to Adam: Is it really totally impossible to get WMF Ops to work with RoR??
Adam to Andy: There are people we can talk to... mentioned above: Dan Duvall will be interested...
Erica: Note that for some reason Dan Duvall currently owes me a favor. Bear this in mind if you contact him about Ruby. ;)
Sage: from an ops perspective, Ruby's versioning mess sounds like a bit of a nightmare.
yeah, making it easy for Ruby devs to dive into the ecosystem = WIN
Adam: @sage yeah we would have to host a mirror of the rubygems repos...
+1 to andrew--I want to see a roadmap for WikiEd software as well
Sage is doing a planning sprint next week with WINTR next week: one outcome if which will be a more specific plan for what we'll be doing in the next year or so.
-- This is the current high level roadmap for Wiki Ed tech plans: https://trello.com/b/dVXtznOh/wiki-edu-roadmapping
Erica: Concerns about internationalization. My team supports lots of languages. language variants and RTL are not trivial. +1 to RTL! <3 Next steps:
So... what do we do when we leave this room?? What are our next steps and who is going to do what?? :)
*WMF talks internally about what we want to do next (what we want to accomplish) and loop back with Sage.
*WMF takes a look at the internationalization. Try to draft a language engineer, write a blurb describing why we need this person.
* Adam talks to DDuvall about deploying and supporting a RoR app on labs.
*Another WMF meeting in the next week.
*Another meeting in 2 weeks or a month to touch base with Sage on his plans.
-- Create, share and study design docs for upcoming work?
-- Make a design proposal and study frameworks to use (OOjs UI?)
Ed team was told in their QR, if we have any asks for WMF tech support, Lila/Erik/Damon need to know:
1. If we were to put technical resources behind this, what will we be able to do with it (or not do if we don't have it)
2. A rough scope and an estimate of the cost from engineering or product
BTW: Sage has a proposal for a Wikimania session related to this:
§2015-02[edit | edit source]
Status: We're looking at existing work that overlaps with our goals, and trying to integrate with the Wiki Education Foundation's recent work.
§Minutes from WMF Education Program Extension Sync Up, Feb 3[edit | edit source]
Anna Koval (WMF)
Andrew Russell Green (WMF)
Adam Wight (WMF)
Amir Aharoni (WMF)
Erica Litrenta (WMF)
To discuss what we might be able to build in a week, vis a vis a replacement for the education program extension, something that has a much broader application.
To loop in Language Engineering.
AA: Why Ruby on Rails?
AG: The Wiki Education Foundation is developing tools off wiki with an external contractor and their application is on Ruby on Rails, but it is not internationalized. So the idea is to coordinate with what they're doing since we share the same goals. But to do so, we need to see how their app can be internationalized. Maybe it could be set up on labs?
AA: RoR should have some internationalization capability. If the developers are storing the strings in English, they could be put on translatewiki.net for our amazing community of volunteer translators to translate.
EL: Do the extension or whatever replaces it need particular language support?
AA: The extension doesn't need any special internationalization. Don't know about the RoR app yet. Need to see it first. To move forward, need the strings on translatewiki.net. Does it have a user interface? (Yes.) That may need adaptation to RTL.
AA: It's likely that Translatewiki already can deal with RoR... Siebrand will need full permissions to write on the github...
AW: If there's no glue currently Translatewiki-RoR it should be easy to make. AA also says so.
AA: In addition to translating strings, we'd need maybe to flip some icons and maybe ake some CSS adaptations, generally doesn't seem like a lot of work even so.
EL: On enwiki now, programs from the US/Canada use the Wiki Ed dashboard, but other programs use the old education program extension. Should the Foundation host an instance of their code?
AW: Someone needs to support it and we're probably the best candidate for that. Dan Duvall might be interested in giving us official support.
AG: Another option is that Wiki Ed and WMF share server space and there could be a switcher for which project and which language you want to see dashboard data for?
WEF is attempting to replace the EducationProgram extension.
Note that the color schemes are different on the resulting course pages.
A larger tech roadmap will help.
Also knowing the education program's non-technical needs for the development of the product.
Need to get the users involved.
Need to know if Wiki Ed has done any user research already.
And was there any user research done by Steven Walling and the Growth group for Editor Campaigns?
Brainstorming about requirements
Steps to add i18n to the WEF app:
* Choose a Ruby i18n library.
* Wrap all strings in the dashboard app, and document the strings for context (like the "qqq" language in MW).
* (WEF) Give Siebrand Mazeland permissions to the dashboard github repo.
* (WEF) Give Amir a link to the repository of the icon and image files used.
* (AA) Look into RTL issues.
* (SM) Create a project on translatewiki.net for the dashboard.
* (SM) Set up synchronization to pull new translations into the dashboard
* (AW) Ask Dan Duvall to support the education dashboard deployment
* (AW) Mirror the Wiki Ed dashboard source repo
* (AW) Reach out to the Gather folks and see if they want to collaborate
* (AW) Hunt down Steven Walling's road map for Campaigns
* (??) Find a replacement for Steven Walling
§Minutes from Gather - Education Program coordination meeting, Feb 6[edit | edit source]
AKoval, AGreen, JKatz, AWight
To determine what the overlap might be between our projects!
We shared the following material:
Overview of Education Program Extension:
Overview of the "Campaigns" project:
An example course with articles, associated editors and reviewers:
Metrics dump of coursework results:
List of courses:
Staging site for Ruby on Rails dashboard, meant to replace Extension:EducationProgram
Wikimania session submission to discuss using the education extension across the wider movement for non-education outreach:
Gather mockups (unknown URL)
§Recently completed projects[edit | edit source]
|Feedback Dashboard||A dashboard allowing experienced users to answer comments and concerns coming from MoodBar||Team Fabrice Florin, Brandon Harris, Ian Baker, Rob Moen, Benny Situ, Alolita Sharma:|
We implemented a leaderboard of recent top responders on the feedback dashboard. New editor feedback is now added to a dedicated log. When feedback is marked as helpful, that fact is displayed on the feedback dashboard itself. Other than a few other smaller changes, we're now moving the project into maintenance mode to focus on article creation workflow and New Page Triage.
<translate> An extension to encourage expressions of appreciation between users.</translate>
<translate> The code was completed, and the feature deployed to the English Wikipedia at the end of June. The Product research team published a [[<tvar|wlove-analysis>Special:MyLanguage/WikiLove/Analysis</>|basic analysis]] of its usage, and [[<tvar|wlove-stories>Special:MyLanguage/WikiLove/Stories</>|stories of its evolving usage and impact]]. This project is now considered to be completed.</translate>
|Page Curation||Interface for reviewing new pages||Team Fabrice Florin, Ryan Kaldari, Benny Situ, Brandon Harris, Vibha Bamba, Howie Fung, Terry Chay:|
Page Curation is now in 'maintenance mode', following its release on the English Wikipedia in September 2012. There was no significant development activity on this project this month. Oliver Keyes has completed a project to look at various ways of localizing Page Curation to any and all wikis that want it: it is currently being reviewed by Howie Fung to assess its feasibility.
Create infastructure for providing guided tours, as well as some initial tours
|Team Terry Chay, Matt Flaschen, Steven Walling:|
Matt Flaschen and Sam Smith met on GuidedTour refactoring, and decided next on steps, including testing out the proposed domain-specific language, reviewing existing tours, and refactoring the user state code.
|2012 Wikimedia fundraiser||Engineering support and development for the 2012 fundraiser of the Wikimedia Foundation||Team Katie Horn, Peter Gehres, Adam Wight, Matthew Walker, Jeff Green:|
In March, we wrapped up our 2012/13 non-English international fundraising efforts, making approximately 5 million USD over the course of the month. Originally, we had planned to run the non-English international fundraiser continuously until June, but were forced to accelerate our plans due to some potential instability at the beginning of April with one of our crucial payment gateways. At the very end of March, we started publishing aggregate public fundraising data to samarium.wikimedia.org.
|Notifications||Project hub for the Notifications infrastructure and UI (code-name: Echo)||Team Fabrice Florin, Terry Chay, Howie Fung, Ryan Kaldari, Benny Situ, Luke Welling, Vibha Bamba, Brandon Harris, Oliver Keyes, James Forrester, Andrew Garrett, Matthias Mullie:|
In August, the Core features team is improving Echo notifications of changes to Flow boards and topics that a user is watching. The list of notifications in the Echo flyout scrolls. If you have Flow notifications the flyout has a separate "Messages" tab for them, and these notifications remain in an "unread" state until you view the topic, click "mark all as read" or "x" them out individually. The rest of your notifications appear in an Alerts tab and keep the current behavior of being marked as read as soon as the flyout displays the notification. Plus miscellaneous improvements to Echo code and tests. See technical details.
|Article feedback||A new tool to engage readers to improve articles on Wikipedia.||Team Fabrice Florin, Matthias Mullie, Pau Giner, Oliver Keyes, Dario Taraborelli, Howie Fung, Erik Möller:|
There was no activity on Article Feedback in October, as we await the results of the French pilot to determine our next steps for this project.
|2013 Wikimedia fundraiser||Engineering support and development for the 2013 fundraiser of the Wikimedia Foundation||Team Katie Horn, Peter Gehres, Adam Wight, Matthew Walker, Jeff Green:|
The fundraising team has been steadily ramping up to the big English fundraiser, and so the fundraising tech team has been transitioning into their usual year-end fundraising monitoring and support role. At the end of October, we went in to a feature freeze for CentralNotice, and have been concentrating on bugfixes with the intent of increasing stability on the payments cluster and in the rest of the donation pipeline. We have also started planning for the international fundraising campaigns targeted for early 2014.
|Growth||To create sustainable growth in the community of editors on Wikimedia projects.||Team Terry Chay (EPM), Matt Flaschen, Aaron Halfaker, Kaity Hammerstein, Rob Moen, Sam Smith, Moiz Syed, Steven Walling:|
In September, the Growth team shut down, with workflows shifting into the mainstream of other teams.