Purpose of this document: Goals for the Wikimedia Engineering and Product Development department, fiscal year 2014–15 (July 1, 2014 – June 30, 2015). The goalsetting process owner in each section is the person responsible for coordinating completion of the section, in partnership with the team and relevant stakeholders.
These goals will be iterated on a quarterly basis. Each quarter, the top priorities for the department as a whole will be elucidated and highlighted.
Note: The top priorities live on a subpage, but are dynamically loaded below - the content is identical and automatically updated in both places.
This page is currently a draft.
More information and discussion about changes to this draft on the talk page.
Strategic context for this quarter:
We are renewing our focus on VisualEditor as a core commitment we must deliver at a high level of quality, to make it easier for anyone to edit our projects. After two quarters of foundational work, many key blockers have been addressed and we are confident that we can make a successful push toward wider usage.
On the mobile web, we continue to focus on contributions beyond editing. While we have seen success and growth of mobile editors, we recognize that the mobile web is not simply a smaller version of the desktop, and offers new opportunities for casual participation. This quarter, we are calling out specifically the work on the mobile web, but the work in apps continues, consolidating and building on the readership focused features we launched last quarter.
We have long committed to being a data-driven organization. In order to make data a part of every product team's day-to-day decision-making, we need to ensure that key metrics are reliably collected (instrumentation). This is still an organization-wide issue, and we are therefore calling it out as a top priority. Beyond merely counting individual user actions, we are especially interested in making visible the drop-off/conversion rates for key user actions like editing and uploading in a consistent manner (funnel analysis).
What we are consciously not yet calling out as a top priority:
The Wikidata Query Service. We are continuing work on this project, prototyping different technical solutions and specifically using the mobile web team's work and the existing WDQ-based tools to inform requirements. However, we recognize that we are still in the prototyping stage, and we're not yet staffed to make this a top priority for this quarter.
SOA Auth. The priorities of the services engineering team in particular will need to be fully aligned with shipping VisualEditor and mobile features, so we are reluctant to call out any specific service that may need to be de-prioritized.
Fundraising refactor / A/B testing. With the analytics team focused on instrumentation and fundraising-tech busy with post-fundraiser cleanup and payment processing related work, now is not the best time for a major cross-functional initiative to consolidate e.g. A/B testing functionality across teams.
Front-end standardization / UX standardization work. This will take a backseat to VisualEditor work this quarter (many of the same individuals are involved), and deliverables will be scoped accordingly.
Measures of success
Personnel and teams involved
Prepare for bringing VisualEditor to all new users on wikis where it is currently opt-in
New Editor, Casual Editor
We will get VisualEditor by the end of Q3 to be ready to be on by default ("opt-out") on the English Wikipedia and other wikis in desktop mode, subject to community happiness, for "new users" – anonymous users and logged in users who have never edited; doing nothing to existing accounts with edits.
We aim to run a trial for new and IP users getting VisualEditor on by default for a week or two to test the value before the end of Q3.
Prioritised functional and technical objectives:
Any identified major stability issues fixed – no regressions or data corruptions
Improved performance, especially in editor load
Existing features provided, plus in-flight UX and product changes:
Storage capacity & redundancy expansion In progress
Process for user backups In progress
Labs available in the new data centre (with Neutron/IPv6) NNot done
Distributing tools, deployment-prep to both data centers (availability/redundancy) NNot done
OpenStack manager extension improvements or replacement (Horizon)
Improvements to the Labs management web interface (MediaWiki OpenStack manager or possibly Horizon) will need involvement from the UX group, and potentially development assistent from a (MediaWiki?) development team.
Upload Wizard - Features - progress bar, drag and drop, better FAQ, replace infographic
Structured Data - planning with Wikidata, new metadata structure, design file page/editing tools, community discussions
Critical bug fixes - Examples: Image scalers, Core media handling, Media Viewer, TimedMediaHandler load on every page
Multimedia Metrics - upload funnel analysis, drop-off rate, UI clicks, # uploads, # uploaders, other upload tools, files used
Upload Wizard - Platform - fix more bugs, separate interface code, metadata-related infrastructure
Upload Wizard - Features - better licensing & category tools, add metadata anytime during load
Structured Data - implement structured data on Commons with Wikidata and community
Critical bug fixes - Examples: Image scalers, Core media handling, Media Viewer, TimedMediaHandler upgrade
Upload Wizard - refactoring and unit testing (and possibly stability and UX improvements enabled by that). Bugs encountered while undertaking this effort will be fixed, but might not be as high-profile as the ones we'll handle once we have JS error visibility. Tracking funnel metrics over time instead of only having a rolling snapshot.
JS error logging - getting it working on beta and preparing a plan for production. The goal for the team is to get visibility on which main errors/bugs are responsible for the Upload Wizard funnel drops.
Platform - investigating a better solution for thumbnail storage.
Note that these milestones and roadmap are preliminary and subject to change. They will be re-evaluated quarterly, based on previous quarter results, as well as community and team feedback.
To meet our goals for multimedia, we plan to partner with these other teams:
Wikidata: implement structured data on Commons and other sites
Visual Editor: upload media to articles, find and use existing media
Search: make it easier to find media based on structured data
Anonymous editor signup invitations and basic signup workflow improvements released on desktop version of all top 10 Wikipedias or more. Stretch goal: collaboration with Mobile Web and Apps teams on mobile versions.
Personalized task suggestions and notifications for new editors released as default on Wikipedias where onboarding (GettingStarted and GuidedTour) is present. Stretch goal: task suggestions and notification aimed at very active editors (100+ edits/month).
Product roadmap for Q3 and Q4 to be determined based on mid-year targets data.
End of year targets: by June 2015, we aim to have gained at least one of the following...
Acquisition: we will increase (compared to a control) new registrations across Wikimedia projects by 23%.
Activation: we will increase (compared to a control) the rate at which new registrations become active editors (5+ content edits/month) by 23%.
Individually, any of these conversion rate increases will lead to a full reversal of the year-over-year decline in total active editors. If all three targets are addressed, only a 7.6% increase in acquisition and activation are required, along with a 29% increase in retention.
Research & Data and User Experience: Both of these teams are core dependencies for Growth. Currently, the Research and Data embeds a full-time research scientist (Aaron Halfaker), and in order to support continued A/B testing capacity this will continue to be a key need on the team. Additionally, User Experience embeds a primary and secondary design on the team (currently Moiz Syed and Kaity Hammerstein).
Analytics: First and foremost, Growth depends on Analytics development for continued support of EventLogging, Wikimetrics, and analytics slave databases. In addition, Growth plans to use planned data products built by Analytics, such as an interest graph, to better attract new editors.
Community Engagement (Product): The Community Engagement group did not provide support for Growth during the 2013–14 fiscal year. We think providing some form of dedicated community liason will be critical if the team is to tackle more contentious areas of the user experience, such as article creation. Currently Product (Steven Walling) is filling this role on an as-needed basis, which puts the team primarily in a reactive mode, rather than having a proactive outreach plan.
Team Practices Group: The Growth team is small but is relatively inexperienced with Agile practices, and the new team practices group could provide extremely valuable support. Growth has implemented some Scrum practicies, but its capacity for process improvement is limited since the team has never had a dedicated scrum master.
Mobile Web: There is close alignment this coming year with the goals of the Mobile Web team and Growth. Mobile wants to spread its knowledge about mobile-first product development and engineering, while Growth wants to help develop mobile versions of its products.
Mobile Apps: As we introduce login, editing, and other contributory workflows to our native apps, Growth should work closely with the Apps team to make sure there is consistent experience, discover ways to integrate its findings in to the apps, and consider methods for attracting apps users to desktop products.
This fiscal year, the Mobile Apps team will be focused on getting data on feature usage of the new native mobile applications for iOS and Android in order to set goals and prioritize work on new apps features.
The Mobile Web team will split their time between two main focus areas:
Technical enablement – ensuring that other teams can work with MobileFrontend code, standardizing UI and code across desktop and mobile, and helping other teams design and build features that are responsive and work cross-platform.
Product work – this encompasses both a) porting select desktop experiences to tablets and handsets; and b) creating new features and contribution streams that help raise the number of new mobile users who hit the active editor threshold (5+ edits per month; below, referred to as new mobile active editors), so that we not only acquire but retain a healthy new editor population via mobile apps and web.
Our quantitative targets are based on the Growth team's editor model and focus on acquisition (getting more users to register for accounts via mobile), activation (getting more newly registered users to make 5+ edits in their first month), and retention (retaining new mobile active editors for 2+ months post registration). Note: targets for acquisition may change based on ongoing data collection around total edit numbers on tablets; if we observe a sustained drop in total contributions post tablet redirect, we may prioritize work on anonymous editing on mobile, which may change the number of signups on mobile.
25% increase in acquisition of new mobile signups from Q1
25% increase in activation of new mobile active editors from Q1
Stretch goal: 50% increase in acquisition & activation from Q1
Tablets: Special pages and feeds
Handsets (based on tablet data from Q1 & 2): article creation and/or talk page overlay
(Based on results from Q2): continue work on new mobile-specific contribution funnels
50% increase in acquisition of new mobile signups from Q1
50% increase in activation of new mobile active editors from Q1
Stretch goal: 100% increase in acquisition & activation from Q1
Handsets: Special pages and feeds
Handsets (based on tablet data from Q1 & 2): article creation and/or talk page overlay
Based on data from Q1 & 2: continue testing and refining existing mobile-specific contributions or start new activation/retention experiments
100% YoY increase in acquisition of new mobile signups
100% YoY increase in activation of new mobile active editors
Stretch goal: 200% YoY increase in acquisition & activation; measurable increase in retention
Mobile editing fully handed off to VE team
Editing team: We'll be working with the Editing team throughout the year to create a standardized editing experience (both wikitext and VisualEditor) across devices.
Flow: A MobileFrontend engineer with be embedded with the Flow team throughout Q1 to help build a mobile-first Flow experience.
Research & Data and User experience: We will be doing more in-person qualitative testing of new and existing mobile features to ensure good usability and a high-quality user experience. We will also run controlled tests and gather quantitative data on features to determine their effects on new and active editor numbers.
Analytics: We will need help from Analytics to continue monitoring and analyzing pageview traffic to determine the flow of readership to the desktop and mobile sites of all our projects.
Growth: Some of the mobile-specific contribution experiments planned for this year may dovetail with Growth experiments. Because we will be focusing on the same set of metrics/targets, the Growth and Mobile team should be working closely and sharing product thinking, experimental setup and overall strategy.
The Maps & Geo team will be a new team at the Wikimedia Foundation tasked with scaling our existing mapping resources to enable the world to visualize Wikipedia all around them. They will start the year forming the team, inventorying existing work, and scale our tile services and osm db to be production ready. Next they'll work with the Mobile Web and App team to empower our users to use mapping resources.
The Editing Team is working to make VisualEditor a great editor for new and experienced editors alike, focusing on improving the performance and usability whilst adding some more features to make VisualEditor more helpful, intuitive and practical for use for every content edit, alongside maintaining, improving, and extending the existing editor software. Below are a set of goals that we hope to achieve over the next financial year (July 2014 – June 2015), with a balance of optimistic and pessimistic assumptions about speed of delivery; not all goals may be achieved in the time period indicated, and some may be updated over time.
On-going work happening every quarter:
Stability and bug fixing, prioritising any bugs that cause wikitext corruption or any other form of disruption for our wikis’ communities (including in the wikitext editor);
Performance improvements for users, tracking load & save times, execution speed;
UX improvements tracked regularly and reported in a quarterly public user testing narrative;
Ensuring the success of VisualEditor on mobile, with a target of feature equivalence on tablet and at least some features on phone, with responsibility for both VisualEditor and wikitext editing pipelines on tablets and phones as well as desktop transitioning from Mobile to the team; and
Collaborating with volunteers and other Engineering teams like Parsoid, Services, Platform, & Core on related efforts like skin improvements, front-end performance, and other areas.
NPostponedDeployment: Engaging with English Wikipedia to discover pain points as part of agreeing the criteria for a gradual ramp-up of VisualEditor availability and usage to default, expected to happen in Q2, subject to community discussions (after this point, ongoing support)
YDoneCore: Internet Explorer 10+ browser support
Service deployed in production but client not createdFeature: Auto-filling citations from ISBN, DOI or URL
NPostponedFeature: Editing templates’ parameters as rich content, not wikitext, with helper tools for some types like image (searching Commons), link (searching wikis), date (date selector extended from Wikidata’s), and possibly others
Load performance: No significant changes expected – 3s 50%ile, 5s 75%ile, 25s 99%ile (same as baseline)
Save performance: Major improvement is compressing save data before submitted for save – 3s 50%ile, 6s 75%ile, 15s 99%ile (from 4s 50%ile, 8s 75%ile, 15s 99%ile baseline)
Per-edit adoption: No significant changes expected – 35% IPs, 20% post-default users, 4% pre-default users (same as baseline)
NNot yet completeCore: Initial improved support for IMEs, for key expanded if not all language groups
NNot yet completeFeature: Auto-filling citations from ISBN, DOI or URL
YDoneFeature: Table editing – inserting new and deleting existing rows and columns, and common "advanced" tools like sortable columns & table header
Not yet complete; some improvements madeExperience: Clearer link editing tools based on design research
NNot yet completeExperience: Simplified citation system based on design research
YDoneExperience: Improved media search based on design research
Deployment: Engaging with English Wikipedia to discover pain points as part of agreeing the criteria for a gradual ramp-up of VisualEditor availability and usage to default, expected to happen in Q3/4, subject to community discussions (after this point, ongoing support)
Core: Initial improved support for IMEs, for key languages – e.g. ja, ko, ar, fa, Indic languages – if not all language groups
Feature: Auto-filling citations from ISBN, DOI or URL
Experience: Improved special character inserter based on design research
Experience: Clearer link editing tools based on design research
Experience: Simplified citation system based on design research
Experience: Media editing tool redesign
Deployment: Engaging with German, Dutch and Spanish Wikipedias in a similar fashion to English Wikipedia, agreeing the ramp-up to default to happen from around 2015Q1 onwards, subject to community discussion
Core: Language variant editing support for Chinese and other languages
Core: Release a stand-alone VisualEditor for third party embed-ability
Feature: Editing templates’ field as rich content, not wikitext, including nested templates
Feature: Helper tools for some template field types like image (searching Commons), link (searching wikis), date (date selector extended from Wikidata’s), and possibly others
Feature: Wikidata invocation editing as a template parameter type
Feature: Auto-save local-only drafts if a browser crashes/disconnects mid-edit
Hiring for the new services team has just started. Eventually we'll be four engineers. The team's focus will be on modularizing our technical infrastructure and enabling other teams by developing internal services and a high-performance external REST API.
Implement first iteration of REST API front-end (restface)
Alpha deploy to api.wikimedia.org
Simple proof of concept implementation of page metadata end point for Parsoid HTML views (redlinks etc)
PDF render service deployment
First iteration on Rashomon storage service (with versioned blob and queue buckets & 'lame' auth)
Deploy & start using it with Parsoid
Wrap HTML load/save in restface
HTML templating: documentation, in particular KnockOff compiler & PHP implementation
Iterate once we have feedback from users & think more about use for content & messages
Facilitate the achievement of 80% of the goals by teams receiving dedicated resourcing and/or undergoing multi-day workshops with the Team Practices Group
Establish a mechanism to transparently and iteratively measure engineering team health
Ongoing work across all quarters: support teams in process/practice improvements, driven by health-check survey results
The success of this team will be determined by the success of the teams that leverage TPG's services. A guiding belief of TPG is that healthy teams build better products. It is the TPG's assumption that by supporting WMF engineering teams in becoming healthier and more sustainable, WMF engineering will be more successful as a whole. In addition to qualitative feedback, we can measure the TPG's impact by the success achieved by the individual teams engaged with the TPG - as measured by those teams meeting, or coming close to meeting, their goals. Further, the TPG can begin to validate its assumptions about team health impacting sustainability and product quality by first establishing a mechanism to measure and evaluate engineering team health.
Hire 1 scrummaster to work with the Mobile Web and Mobile Apps teams YDone
Iterate on and deliver updated health-check survey to Mobile Web, Mobile Apps, Mediawiki Core, Analytics Engineering, and Analytics Research, Fundraising, Release Engineering, Core Features, Language In progress
Conduct a structured team practices workshop with the Mediawiki Core team YDone
Hire an additional TPG resource to work with the MediaWiki Core team In progress