Growth/Growth team updates

From MediaWiki.org
< Growth(Redirected from Growth/Project 2018 Q1)
Jump to navigation Jump to search


This page contains updates on the Growth team's work to increase new editor retention in Czech and Korean Wikipedias. The planning for this work began in June 2018. The team works on multiple projects at a time, but they will all be discussed on this page. During July, August, September, and October 2018, the team was also simultaneously working on a project to improve the Articles for Creation process and New Pages Feed in English Wikipedia, which is discussed on its own page.

How to get involved[edit]

It is important that our work is grounded in the reality of the communities we hope to help. If you have thoughts or ideas around this project or our team's work, please comment on this project's talk page.

Project updates[edit]

Update 2019-01-11: help panel deployed in Czech and Korean Wikipedias[edit]

Focus on help desk (Help panel)

Yesterday, we deployed the help panel to Czech and Korean Wikipedias. This is the Growth team's first feature built specifically to increase new user activation (editing for their first time) and retention (coming back to edit again). Going forward, half of new accounts will see the help panel button in the lower right of their screen when they open any editor on any page -- both on desktop and mobile (but not mobile apps). The other half of users are in the control group of our experiments.

We carefully instrumented the feature so we would be able to tell exactly how users do and don't use it, and we're going to be watching the data closely to see how many people open the panel, click on links, and ask questions. After about 24 hours of the feature being live, about 25 users have seen the help panel, with only one question having been asked so far -- although several users have opened the panel and clicked on links inside. We'll post more detailed numbers next week. To follow along with the help panel activity, feel free to check out these links:

Personalized first day (Welcome survey)

The welcome survey was deployed such that half of newcomers would receive it, with the other half in a control group. This was so that we could detect whether receiving the survey caused newcomers not to edit -- the concern was that putting up another barrier to editing would deter them. We are happy to say that the experiment shows that the survey does not inhibit editing, and we will publish these results in the coming weeks. Because of this, we feel comfortable testing another version of the survey: Variation C. We're going to be showing Variation C to half of newcomers, and the original Variation A to the other half. We want to find out whether Variation C's more dynamic and streamlined interface increases participation. We'll know the initial results in about four weeks.

Update 2019-01-03: initial results from welcome survey[edit]

Right now, the team is mostly focused on our upcoming launch of the help panel next week, executing on the plans described in previous updates to test and launch on time. But the most exciting new thing we are looking at and thinking about is the initial report on the welcome survey results.

Personalized first day (Welcome survey)

  • The welcome survey launched in Czech and Korean Wikipedias on November 19th, and we collected hundreds of responses in each wiki in the following month. Our team had a goal of generating an initial report by the end of December.
  • The initial report is here, in which we can see the reasons people create accounts, the topic they're interested in editing, the Wikipedia experience they have already had, and their interest in getting help from experienced editors. Here are some toplines, with the rest in the report:
    • The survey has high response rates: 67% in Czech and 62% in Korean.
    • Many people create accounts just to read articles -- a potential opportunity to engage new editors.
    • High numbers of people indicate they are interested in being contacted for help: 36% in Czech and 53% in Korean.
  • We're still digesting and thinking about these responses, and discussing how to take action on them. We're pleased to see that many people respond to the survey and take it seriously. We're seeing a pretty wide spread of intentions and topics people are interested -- this makes us optimistic that we can successfully personalize the first day.
  • A high priority analysis that we're working on now is figuring out whether being shown the survey makes a newcomer leave the wiki, instead of continuing on to edit. This is the main concern raised by community members around this idea, and it's the reason we have only been displaying the survey to half of respondents, keeping the other half as a control group. Early results seem like the survey is not decreasing activation (our word for "making a first edit"), and we will publish those results in more depth.

Focus on help desk (Help panel)

  • The help panel is still on track to launch next week to Czech and Korean newcomers.
  • The text has been largely translated on Translate Wiki, and the team is currently testing and fixing bugs.
  • We measuring using the new HelpPanel EventLogging schema, and we received both engineering and privacy approvals on our plan. We'll be publishing that plan on wiki in the coming weeks.

Understanding first day (EditorJourney)

  • The data analysis for this project is underway, and we are close to an initial report to publish for community members to see.

Update 2018-12-21: upcoming help panel deployment[edit]

As the year wraps up and people head out on vacation, we've been primarily focused on two things: preparing for the initial deployment of the help panel and assembling initial reports on "Understanding first day" and "Personalized first day".

Focus on help desk (Help panel)

  • Over the past week, we have been steadily working on the remaining tickets for our initial release. Some of the most recent code to merge includes:
    • Support for help panel on mobile.
    • Ability to add an email address or verify an unverified address through the help panel.
    • Ability to easily turn the help panel off.
    • Support for the new "HelpPanel" schema, which is what we will use to track and measure interactions with the feature. In January, we'll post our "measurement plan" so that community members can see how we'll be evaluating success.
  • The wording in the help panel is now available for translation here on Translate Wiki.
  • The help panel is currently available on the beta cluster for any community members who want to try it out. To turn it on, go to the "Editing" section of Preferences and select "Enable the editor help panel".
  • Our plan is still to deploy this feature to Czech and Korean Wikipedias around January 10, and we'll be giving plenty of notification and recommendations to those communities on how they can be ready to receive and answer incoming questions at their help desks.

Initial reports on existing projects

  • We're nearing completion on initial drafts of reports for the findings from both "Understanding first day" and "Personalized first day". We're looking forward to hearing what community members think of the results.

Update 2018-12-13: working on help panel and preparing for next quarter[edit]

Thanks to everyone who weighed in on the open questions we posted last week. We got some helpful ideas and responses. Please keep discussing! The team has been spending most of our time coding and testing the help panel so that it can be deployed at the beginning of January. We also spent time in the past week planning our next quarter. And in the time remaining this month, we are working to complete initial reports on the "Understanding first day" and "Personalized first day" projects.

Focus on help desk (Help panel)

  • We've updated our release date based on when code can be deployed in January. The help panel is now slated to arrive in Czech and Korean Wikipedias on January 10. Our goal is to finish all the code by December 21, so that when team members come back from holidays, all that's left to do is final testing.
  • We have finalized how we want to instrument the feature so that we can test how users use it, and whether it has a positive impact on activation and retention. We'll be posting a page on wiki that describes our measurement strategy and planned experiments, and a summary is available here.

Planning our next three months

  • We posted here last week about how we were thinking about our future plans for "Personalized first day", and asked for some community thoughts here.
  • After some more team discussion, we are currently planning on pursuing the "engagement emails" idea to use email as a way to re-engage editors who haven't made their first edit, or haven't returned to edit again after their first edit. We are also thinking about whether this could go together with some work on a "newcomer homepage" or "landing place" that would help orient newcomers to the resources that could help them accomplish their goals. And in order to maximize the impact of an "engagement emails" project, we are thinking about how to encourage more newcomers to create accounts with email addresses.
  • We'll post more detailed thoughts on our plans next week.

Initial reports on existing projects

  • At the beginning of this quarter, we set milestones for our "Understanding first day" and "Personalized first day" projects. Since they're both data collection efforts, we wanted to be able to generate initial reports on the early results from those projects.
  • We are still on track to create those, and plan to post them by the end of the month so that communities can start thinking about the implications of the analysis.

Update 2018-12-05: Help Panel and Variation C[edit]

Now that two of our projects are deployed, we are primarily working on our third project, the help panel. We are also working on "Variation C" of the welcome survey, which we think could increase response rate. Data analysis for both the welcome survey and for the EditorJourney work is beginning now.

In this update, there are a few places where we hope for community members to look over our work and comment. Those spots are marked with a To do To do.

Focus on help desk (Help panel)

  • Engineering continues on this project, with the release in Czech and Korean Wikipedias still planned for January 7.
  • To do To do We are currently working from this interactive prototype, which reflects our planned workflow and wording. We encourage community members to click around the prototype and post any thoughts or reactions here.
  • To do To do We have also settled on some business rules for who will receive this feature and in what parts of the wiki, described here on the project page. Please comment here with any thoughts on whether those rules are on the right or wrong track.
  • We are in the process of planning out what we will measure and experiment on with this feature, and planning out the EventLogging schemas for that measurement.
  • We conducted eight user tests via usertesting.com, and posted the results here. Users generally liked the help panel and understood how it works, both on desktop and mobile.
  • Our Czech and Korean ambassadors are consulting with their communities to decide which help links are best to include in the help panel.

Personalized first day (Welcome survey)

  • After deploying Variation A on November 20, we have been steadily collecting data. In terms of data analysis, we only have some very early numbers, and we plan to post more stable numbers during the month of December.
  • As mentioned in the 2018-11-25 update below, we are also working on Variation C, which is an improved design for the welcome survey, meant to increase response rates. We are currently deciding whether to deploy that change before or after the Christmas holidays, and how to incorporate that into the controlled experiment we're currently running.
  • To do To do The "Personalized first day" project originally started because we want to be able to personalize the newcomer's initial experience to help them achieve their goals, and keep them coming back to editing. That's why we deployed the welcome survey: to gather some information from newcomers so that we can present them with content that will help them. Now that the survey is gathering data, the team is spending time thinking about what that next step will be, and our initial thoughts are posted here. We invite community members to join the conversation here.

Understanding first day (EditorJourney)

  • This project is continuing to run and collect data. No additional work has been done since deploying it and fixing some post-deployment bugs.
  • Data analysis will being next week.

Update 2018-11-25: two projects deployed[edit]

We're excited to have deployed two of this quarter's three projects! The EditorJourney schema is now collecting data on the activities of new users in Czech and Korean Wikipedias, and the Welcome Survey is being shown to new users in both those wikis (but only to half of those new users, because half are in the control group). These deployments are our team's biggest milestones to date, and we're excited to start learning from the incoming data. As that data comes in, we'll be turning our engineering attention to the Help Pane project, and to the next version of the Welcome Survey.

Personalized first day (Welcome survey)

  • We deployed this on November 20. This was two weeks later than our initial plan of November 6. The main reason for the delay was the translation of the survey privacy statement and the privacy policy into Czech and Korean. The external translation vendor took one week to translate the documents, and then our ambassadors needed an additional week to adjust and correct parts of the translations to use wiki language. For future projects, we will be mindful of finding out in advance all the legal documents that are needed, and planning for the two weeks it takes to finalize strong translations. We also now have a set of improved best practices for working with translation vendors.
  • The survey is now being shown to half of new users in both Czech and Korean Wikipedias, and responses are being recorded in the database (the other half of new users are the control group in our A/B test, where we are checking to make sure the survey does not depress the rate at which newcomers make edits). In the next week, we'll take an initial look at the data to identify any issues or trends.
  • There are two next steps for this project:
    • Variation C: during our design research for this project, we did a comparative review of how other platforms find out basic information about their users. From that, we created an ideal design of how the Welcome Survey would look, which we called "Variation C". We think that design will maximize the number of users who complete the survey and stay on the wiki. But instead of building that design first, we quickly deployed the more simple "Variation A", so that we could start gathering data. Now that Variation A is deployed, we have turned attention to upgrading to Variation C over the course of the next two to three weeks. We plan to A/B test the two variations to see if they cause different response rates or have different effects on whether new users stay on the wiki.
    • Personalizing: the original objective of this project was to gather some initial information on what new editors are trying to accomplish so that we can give them the help content, task recommendations, or mentorship they need to be successful. We're currently discussing the right first project to work on to take action on the data we're collecting. Some of the options we're thinking about are on the short list below. But each of those things would need a place to "live", and this is making us think about ideas for some kind of "newcomer portal" where newcomers would easily be able to find the tools and information they need. We'll develop this idea a little more, and propose some specifics so that experienced community members can react and tell us what they think.
      • Making it easy for newcomers to see editing activity around the topic areas in which they indicated that they're interested.
      • Connecting interested newcomers to experienced editors.
      • Surfacing the help content most relevant to the reason for which the newcomers created their accounts.

Understanding first day (EditorJourney)

  • We deployed this on November 15. This was nine days later than our initial plan of November 6. The main reasons for the delay were some additional security reviews done by engineers from outside our team, and on the rigorous testing and adjusting we did before deployment. We're glad that the additional security reviews took place, because it's critical that our software is safe, but we'll know for the future to incorporate our dependencies on other teams into our planning. The rigorous testing and adjusting took longer than expected because of the many idiosyncrasies and edge cases in the Mediawiki software: we kept finding ways that the event data wasn't being recorded correctly. We think we have now solved the vast majority of those cases.
  • Data is now being recorded for all new users in Czech and Korean Wikipedias, and we've been auditing the data and preparing to make initial reports.

Focus on help desk (Editor help pane)

  • Engineers are now working on building the initial iteration of the help pane, though most of our attention has been focused on completing the roll out of the two other projects listed above.
  • We are currently planning to deploy this during the week of January 7. We think our engineering work will take up until the holiday season, and we don't want to deploy something new when many engineers and support staff will be taking vacations.
  • In the coming week, we'll be drafting the written copy that will be part of the feature. This copy is important, because we want to make sure new users understand what will happen when they ask a question in the help pane. That work is happening in this task.
  • We're also going to be running some live user tests, like we did for the Welcome Survey.
  • In addition to giving the ability to ask a question, the help pane will also contain a set of links to existing help content. Our ambassadors are determining the right initial set of most helpful links in this task.
  • We have some open questions around the rules and design choices for this feature. Later this week, we'll post them and start a community discussion to get recommendations from editors who have experience working with newcomers.

Update 2018-11-13: final testing on two projects, work begins for "Editor help pane"[edit]

A quick note before the update is that we've given nicknames to our projects, since "Personalized first day", "Understanding first day", and "Focus on help desk" are difficult to say and type. While those phrases still represent the overall goals of the projects, we're now also referring to them on a day-to-day basis as "Welcome survey", "EditorJourney", and "Editor help pane" (respectively), which are more specific to what we'll be deploying in the near term around those three projects.

Personalized first day (Welcome survey)

  • Current planned release to Korean and Czech Wikipedias: Nov 15
  • The welcome survey is now deployed to our testing environment where we are making sure that all the questions are laid out as expected, that no words are being cut off, that data is being recorded correctly, and that all the business rules we planned out are being followed. We'll need to test and fix for a couple more days.
  • The translations of the survey privacy statement and full privacy policy have been returned from our translation vendor, and are currently being edited by our community ambassadors from Czech and Korean Wikipedias to make sure that the vocabulary used makes sense. Posting these is also a blocker for release.

Understanding first day (EditorJourney)

  • Current planned release to Korean and Czech Wikipedias: Nov 15
  • Several team members are using Test Wiki and Beta Cluster to make sure that this data is being recorded correctly for both the desktop and mobile experiences and that all the privacy safeguards we put in place are working right. For instance, we're making sure that when users read articles in the main article namespace, the actual title of the article is not saved. Testing EventLogging is generally tedious, because it involves clicking through as many different elements of the software as possible, and then checking the database to make sure the actions were recorded correctly. A few bugs and issues have been found so far and are in the process of being fixed. The specifics can be found on this Phabricator task.

Focus on help desk (Editor help pane)

  • The team completed an initial design process for this feature in which we dreamed big, and looked into the future of what this feature could be if it shows a lot of promise. To do this, our designer drew on the review of similar features in other software and on similar Mediawiki projects from the past.
  • After going through all the ideas that came out of the design process, we decided on what the initial version of this feature would contain. This Phabricator task lists all the subtasks that describe the various design ideas. These two are the ones we'll be doing for the initial version (T206717 and T209318), in the points below.
    • A button in the lower right for newcomers, giving them the opportunity to "Get help with editing".
    • Clicking the button opens up the "help pane".
    • The help pane contains a set of links to the most useful help pages in the wiki.
    • The help pane also contains an option to "Ask a question", in which newcomers will be able to type a question that will be posted automatically in their wiki's help desk.
  • We're starting to engineer on those initial requirements and hammering out many small details. Please check out the evolving project page to follow along and see mockups of what this will look like.

Update 2018-11-02: nearing deployment on two projects[edit]

For both the "Understanding first day" and "Personalized first day" projects, we are nearing deployment within the next two weeks. Both projects are getting their final approvals and are being tested. Next week, we'll post a detailed update on Czech and Korean Wikipedias explaining what will be changing in the wikis and when.

Since those two projects are on the verge of being deployed and beginning to collect data, our team is going to be turning attention to the "Focus on help desk" project. We'll be making a full project page for that within a couple days so that it is easier for the community to follow along with how we're thinking about it.

Personalized first day

  • We settled on paths forward for the open questions from last week's update:
    • Topics of interest: since we noticed that WikiProjects differ substantially from wiki to wiki, and since we are not completely sure how we plan to use the new user's topics of interest, we assembled a list of 27 topics that appear across many different lists of important topics, including Czech WikiProjects, Korean WikiProjects, English WikiProjects, categories, and Vital Articles. The hope is that many new users will find find a topic that interests them, and if not, they can type in their own topics. We will analyze the results to decide what to do next. The topics we chose are Arts, Science, Geography, History, Music, Sports, Literature, Religion, Popular culture, Entertainment, Food and drink, Biography, Military, Economics, Technology, Film, Philosophy, Business, Politics, Government, Engineering, Crafts and hobbies, Games, Health, Social science, Transportation, and Education.
    • Interest in mentorship contact: we decided on phrasing the question such that it's clear that we do not yet have an active mentorship program. This question is now phrased as, "We are considering starting a program for more experienced editors to help newer users with editing. Are you interested in being contacted to get help with editing?"
  • We incorporated results from human testing that occurred through usertesting.com. Details on this will be added to the project page.
  • The finalized survey copy is now in translatewiki.net, and is already translated into Czech and Korean (as well as many other languages).
  • We have finalized a survey privacy statement for the survey, and we are in the process of getting it and our full privacy policy translated into Czech and Korean. This will help new users understand how their responses to the survey will be used.
  • We have decided on some of the details for the A/B test we'll be running with the feature. One of the community's main concerns with this feature is the potential that a longer sign-up process would cause new uses to leave before editing. Our experiment will give the survey to some new users, and leave the experience of others unchanged. Essentially, if they continue to edit at the same rates, than we can conclude that the survey is not causing users to leave before editing.
  • Up next, we'll be deploying the survey to our testing environments and checking to make sure everything works as expected.

Understanding first day

  • We've completed privacy review, and we are confident that we'll be handling user data in a way that helps our team learn while respecting user privacy.
  • We are finalizing some of the engineering and security aspects of this work, after getting review from several engineers around WMF.
  • We are now testing that data is being recorded and obfuscated correctly by using our test environments.

Focus on help desk

Update 2018-10-25: new project pages and open questions[edit]

Over the last week, we have made a lot of progress on the "Understanding first day" and "Personalized first day" projects. We currently hope that we can deploy both of those projects to some extent during the week of November 6th, but there are still many details to resolve in the next two weeks. We are hoping for any community wisdom to help us resolve some of the open questions laid out below. To help make it easier to follow along with our work, we now have full project pages for each of those two projects:

Those pages contain things like designs, decisions, and implementation details. The project page for "Focus on help desk" is coming next.

Below are some updates on the two projects above.

Personalized first day

Our team has been iterating on the exact wording for the text of the questions and responses in the survey. The mockups now reflect more up-to-date question wording, but we're still finalizing it. We've asked some survey experts to weigh in, and we have also engaged some human testers to try out the surveys and give feedback. We have a couple of open questions that we're not sure how to resolve yet:

  • Topics of interest: we want to ask new users about the topics they're interested in editing. That's because one day we may want to recommend editing tasks that fit their interests, direct them to WikiProjects about their interests, or connect them with other editors who share their interests. We are trying to figure out what topics to offer as choices. One idea is to use WikiProjects, but we know that the majority of WikiProjects tend to be inactive. We are also concerned that presenting very general topics -- "Art", "Sports", or "Geography" --are not specific enough to align with someone's actual interests, which may be "Ceramics", "FC Barcelona", or "Sahara desert". We're still gathering ideas and experimenting with the right way to do this.
  • Interest in mentorship contact: one of the questions on the survey asks whether the user is interested in being contacted by an experienced editor for help with their editing. This is important because it will help us gauge whether many new users are interested in mentorship, which might encourage our team to develop ideas around promoting mentorship on-wiki. We are trying to figure out a good way to ask that question without making new users wait to receive contact that won't be coming immediately, since we haven't yet developed the way to actually connect them with experienced editors. We don't want our questions to be confusing or misleading.

We are also in the process of making sure that we have thought through any data privacy elements with the survey so that users can feel good about answering the questions.

Understanding first day

We have largely settled on exactly what data we will be collecting and the questions we plan to answer with it. The project page describes those questions, and the exact technical specifications for measurement can be found here on Meta Wiki, with the associated business rules on its talk page. In short, we'll be recording the pages that new users in Czech and Korean Wikipedias visit during the first 24 hours after creating their accounts -- but only pages in non-sensitive namespaces that are important for our work, such as the Help, Wikipedia, Special, and User namespaces. For pages in the Article, File, Portal, and Draft namespaces, however, we will only record that a visit to that namespace occurred -- not the specific page. The identifying data will be deleted or aggregated after 90 days.

This new data will allow us to see the path new users take as they learn and prepare to edit. We're also going to connect the new data to existing measurement efforts in the visual and wikitext editors so we can see how the new user's path leads them to succeed or not succeed with their first edits.

Most of the technical work is complete, but we are still in the process of security, engineering, and privacy checks on our work. We're testing out the instrumentation in testing environments, and making sure that the other measurement efforts we want to use have the data we'll need.

Update 2018-10-18: starting three projects[edit]

Our team has set goals for what we'll accomplish by the end of the quarter (we are currently in 2018 Q2, which goes from October 2018 to December 2018.) Our goals can be found with the rest of the Audiences department's goals here. In brief, our goals for this quarter are:

  • Collect initial "Understanding first day" data from instrumentation, and post early numbers on wiki.
  • Collect initial "Personalized first day" data from user input, and post early numbers on wiki.
  • Deploy a "Focus on help desk" intervention experiment in production.
  • Develop a relationship with a third community beyond Czech Wikipedia and Korean Wikipedia.

Since choosing those three priority projects at the beginning of October, the Growth team has started work on each one, with most of our activity taking place in Phabricator under an "epic" task for each project:

Over the last two weeks, the team has primarily been working on "Understanding first day" and "Personalized first day".

For "Understanding first day", our engineers and data analyst have been developing the specifications for an EventLogging schema to record useful information on new editor journeys while respecting their data privacy. Our team is close to posting the specifications on wiki and in Phabricator, and it will be summarized in an update here.

For "Personalized first day", we have designed initial concepts for how this set of questions will be asked to the new account holders, and the engineers are beginning to prototype them. It's possible to explore those mockups here -- but it is important to note that the specific language in the mockups has not yet been worked on. The language in those mockups is placeholder language for when we write the finalized words that will be translated into Korean and Czech. Writing those words is our current highest priority task for this project.

Update 2018-10-02: priorities for October through December[edit]

Background[edit]

The Growth team is very appreciative of the time that community members took to respond to our ideas for increasing new editor retention.  We received helpful comments and opinions from around the world, both from newcomers and experienced editors, that have allowed us to put together our plan for the next several months of work.  Here is how we have proceeded (along with relevant links):

  1. We developed an understanding of new editor experiences and challenges through the New Editor Experiences project.
  2. We put together a list of ideas for how to increase new editor retention.
  3. These ideas were discussed in many different communities in three different languages: English, Czech, and Korean.
  4. The Growth team met to discuss and summarize all the community feedback.
  5. The Growth team made the following development plan that takes into account community thoughts and several other factors.

Theme[edit]

The theme of our work for the coming months is “Day One”. That means we are focusing on the experience of new editors on the day that they create their account.  We’re focusing on day one because our data shows that about two thirds of people who create accounts never make an edit, and of those who do, about two thirds of them never come back to edit again. This makes us think that the first day of a new editor’s experience is an important time to help them start editing and want to continue editing.  We will go into more detail on the data behind this in another update.

Plan[edit]

With the theme of “Day One” in mind, we decided to prioritize the following three ideas for Czech and Korean Wikipedias.  We are not building any of them fully. Instead, we will build a small piece of each so that we can learn whether the idea has potential, and quickly decide whether to continue.  We’ll be working on them in October, November, and December 2018.

  • Understanding first day: we are prioritizing this because it will help us learn a lot about new editors.  It will not change any user’s experience. We’ll simply add logging so that we can see what new editors do right after creating their accounts.  We will be careful with user privacy, and we hope to share initial results in December.
  • Personalized first day: this idea will also help us learn a lot about new editors.  To start, we will add some optional questions to the new editor’s registration process.  This will help us learn what new editors are trying to accomplish, and by looking at the data, and with the help of the communities, we’ll be able to think about how to personalize a new editor’s experience so that they can accomplish what they’re trying to do.  We hope to share initial results in December.
  • Focus on help desk: we are prioritizing this because it was shown to work in other contexts.  We will likely direct traffic to existing Help Desks through a bot that invites new editors on their talk pages, or through a new button in the editing experience inviting new editors to “Ask a question”. That button would be an opportunity to learn whether the “In-context questions or chat” idea has potential.  We hope to have an initial experiment running in December.

We decided to prioritize those ideas for these reasons:

  • They have support from multiple communities.
  • They address issues brought up in the New Editor Experiences research.
  • They have been proven to work in Wikipedia and non-Wikipedia contexts.
  • They help us learn quickly.
  • They can be deployed in small pieces without a major commitment at the beginning.
  • They can be deployed relatively easily to other communities.

Next steps[edit]

Next, we will be planning the work for each of those three ideas and beginning to write code.  We’ll continue to post updates, and we will be clear when we need help from community members. We have already started to create Phabricator tasks for the "Understanding first day" work. For the “Focus on help desk” project, we will need experienced editors to dedicate time to answering questions in the Help Desk, because we will expect many more new editors to be asking questions.  We’re looking forward to continuing to work with the Czech and Korean communities, and we are looking for additional communities to begin to work with.

Update 2018-09-21: summaries of community discussion and planning sessions[edit]

Over the past few weeks, we received a tremendous amount of useful thoughts and feedback on our list of eight ideas. We're grateful to the members of the many wiki communities (including several mid-size wikis) who responded on the talk page, on their own wikis, or sent us their thoughts through other channels. We were also able to get feedback on the ideas from actual new editors in Czech and Korean Wikipedias, who provided a first-hand perspective on which ideas would have helped them as they were starting out. Summaries of the feedback we received on each idea are now posted, along with some follow-up questions on the talk page.

To take action on this feedback and plan the coming months, the Growth team came together in person last week. We did two main things: short-term planning and long-term thinking.

Short-term planning

A poster made by the Growth team when discussing a potential idea for increasing new editor retention.

To determine what our team will work on first, we spent a day processing the community feedback and reflecting on the eight ideas. We split up into groups, each of which was assigned two of the ideas. For each of their two ideas, those groups read through and discussed community comments in detail. They then summarized the comments into a poster to explain the nuanced perspectives to the rest of the team. This format helped us discuss each idea, making sure to take into account the pro's and con's from different communities. Posted here is an example of one of the posters we made, to give a sense of our process. To read summaries of what we learned from communities, see this page.

Long-term thinking

The Growth team intends to be working on new editor retention for many quarters, and we know that we will need to be continuously learning about new editors in order to be successful. Though we already know a lot about new editors from research done in the past year, there are still many open questions about how new editors work and what they need to be successful. We spent time determining which questions are most important so that our team can use upcoming opportunities to find answers. Please visit this page to see our most important open questions and read about the exercise we did to generate them.

Next steps

We are now in the process of deciding exactly which things to build and in what order, taking into consideration engineering challenges, measurement challenges, and how to work with the Czech and Korean communities. Our next update will contain our plan for the coming months.

Update 2018-09-04: final week of community discussion[edit]

Over the past two weeks, we have posted the list of ideas for discussion in many places, and we would like to gather all the feedback we can by the end of this week from as many different communities as possible. If you have planned on weighing in, please do so in the next few days!

Next week, the Growth team will be taking a substantial amount of time to process and digest all the thoughts and reactions that have been posted, and we will make a plan for which ideas make the most sense to pursue, and we'll post that here. In making that plan, we'll take into account community thoughts, technical considerations, and the burden that new development will put on existing communities.

Here are the places we have posted the list of ideas for comment:

Update 2018-08-22: community discussion begins on ideas[edit]

We've now posted a list of the main ideas we're considering for the team to work on first! We hope as many people from as many communities as possible participate in the discussion on that talk page. The list of ideas is based on a lot of research and conversations the team has been taking in, but they are by no means solidified -- that's why this community discussion is really important to us. We're hoping to hear the majority of the community thoughts within the next two weeks (by 2018-09-05).

The list is also being translated in Czech and Korean and discussed on those wikis, since those are the places where the features would be deployed first. If you are interested in translating all or part of the list for your own wiki, please let us know! It would be great to have additional languages discussing.

We'll also be sending out our first team newsletter about this. Please sign up here if you are interested in receiving the newsletter.

Update 2018-08-14: narrowing down on ideas[edit]

The team is now spending an increasing amount of time planning for this first project, even as our engineers work on the Articles for Creation and New Pages Feed project for English Wikipedia. A few short updates are below, with more extensive information coming over the next week:

  • Our top priority right now is to start the conversations around which feature ideas we should pursue first. The team has been compiling many ideas for what we can build and has started to narrow to the ones that seem to have potential. We're going to be asking the Czech and Korean communities to weigh in, and we will also post them here and in other venues so that as many communities as possible can add their thoughts. We'll be looking for new ideas, thoughts on existing ideas, and information on when similar things have already been attempted. Below are some examples of the types of ideas we're talking about. More detail will be posted here in the coming weeks, as well as the reasons we think these ideas could be impactful.
    • Inviting new editors to help desks. Most wikis have a help desk, but most newbies don't know about it.
    • Making it possible for newbies to ask questions in the context where they are editing, instead of having to go to a different page.
    • Communicating a new editor's impact to them via email, perhaps by telling them about the page views on pages they've edited.
    • Gathering optional information from new editors when they create accounts about what they are trying to do and the topics in which they are interested, so we can direct them to information that serves their needs right away.
    • Instrumenting the new editor experience to learn more about what new editors do right after creating their accounts -- whether they read documentation, attempt to edit but don't succeed, or something else.
  • We have completed the staffing of our team by adding Rita Ho, our user experience designer, and Morten Warncke-Wang, our product analyst. Rita joins us from the team that built the Wikipedia Android app, and has important expertise on mobile usage that will be relevant for new editors. Morten is a longtime researcher of Wikimedia projects and has deep experience understanding the issues newbies face and experience working with Mediawiki data. Morten is going to be working on the "analyzing data" work mentioned in the update from 2018-07-06 below.
  • Several of our team members attended Wikimania 2018 and had great discussions with editors from many mid-size Wikipedias, including Arabic, Bengali, Serbian, Ukrainian, and others. We gathered information on the experience of new editors in those wikis, what is the same and what is different across them, and what specific tactics and features are working well to retain new editors. We'll be assembling and posting what we learned, along with the work our Czech and Korean ambassadors have already done on that front to catalog existing help content in their wikis.

Update 2018-07-06: data, planning, and research[edit]

While the majority of the team's effort is currently going toward the Articles for Creation and New Pages Feed project for English Wikipedia, the team is laying the ground work for this initial Growth project by doing three main things:

  • Analyzing data about contributor behavior in Czech and Korean Wikipedias. We are setting up the queries and reports that will allow us to track the contributor retention rates in our target wikis, and to understand exactly where in the editing process new contributors are most likely to drop off. We will post this data on wiki as soon as we're able.
  • Partnering with our ambassadors in the Czech and Korean communities to document the current resources that help new contributors in those wikis. We want to make sure that anything we build compliments the existing pages and tutorials that already exist, and that we are not duplicating efforts with any community members. This is the related Phabricator task.
  • Summarizing existing ideas for features that could increase retention. Many WMF staff and community members have attempted and proposed ideas for new editor retention in the past. We're making sure to collect those ideas and use them as a starting point. We will post that information on wiki as soon as we're able.

We have also spent time this week establishing the processes for our new team, such as when we have meetings, how our Phabricator board works, and how we'll balance new work with maintenance on previous features built by engineers on the team. Those previous features include Notifications and the new filters on the Recent Changes and Watchlist pages.