Growth/Growth team updates

From MediaWiki.org
Jump to navigation Jump to search

Growth
Projects: Understanding first day (EditorJourney) Personalized first day (Welcome survey , Newcomer homepage ) • Focus on help desk (Help panel ) • Community resources

Last news: Growth team updates Newsletters Contact

This page contains updates on the Growth team's work to increase new editor retention in Czech and Korean Wikipedias. The planning for this work began in June 2018. The team works on multiple projects at a time, but they will all be discussed on this page. During July, August, September, and October 2018, the team was also simultaneously working on a project to improve the Articles for Creation process and New Pages Feed in English Wikipedia, which is discussed on its own page.

Contents

How to get involved[edit]

It is important that our work is grounded in the reality of the communities we hope to help. If you have thoughts or ideas around this project or our team's work, please comment on this project's talk page.

Project updates[edit]

Example mentor introduction
Screenshot of Mediawiki's newcomer mobile homepage feature in beta
Session about new editors at Wikimedia Hackathon 2019

Update 2019-07-15: current goals and priorities[edit]

The team has set goals for the coming three months. These are part of longer-term objectives for the coming year that we will be discussing with communities in the weeks ahead. We'll be posting more information on all these goals and initiatives.

  • Newcomer homepage: increase activity through a task recommendations module. Now that we have seen several weeks of positive activity on the newcomer homepage, we think that the most important thing to add is a way for newcomers to find tasks to work on. The challenge will be recommending the right kind of tasks at the right point of their journey.
  • Newcomer homepage: increase feature discovery rate by 100%. Right now, only 20% - 30% of newcomers ever visit their homepage. We want to double that number by making sure all newcomers know how to find it via features linked from this task.
  • Deploy new experiments to four target wikis simultaneously (Korean, Czech, Vietnamese, Arabic). The team is now working with four target wikis, and we want all wikis to be receiving all our work at the same time, which increases our team's efficiency.
  • Deploy stable Growth features to five non-target wikis. Some of the Growth features, such as the welcome survey and help panel, are strong enough that wikis who want to try them are welcome to by working on this checklist.
  • Research and design for next project, either (a) structured user profile or (b) topic/neighborhood features. After we work on task recommendations, there are future developments to the homepage we want to plan. See our team's early notes on structured user profile here and on topic/neighborhood features here.
  • Help panel: increase usefulness through improvements to affordance, search, and UX flow. We have looked closely at data and anecdotes from the usage of the help panel, and we plan to pursue specific improvements to increase its effectiveness.

Update 2019-07-12: features to aid discovery[edit]

  • Newcomer homepage
    • Releases
      • Mentor introductions: we've seen in homepage data so far that the mentorship module is much more popular for asking questions than the help module. We think that may be because of the "personal touch" that comes from having a specific mentor. To accent that personal touch, we have released the ability for mentors to customize their own introductory messages. See the screenshot at right and this set of guidelines for writing good introductions.
      • Renamed mobile menu item: to avoid user confusion between their homepage and the existing Wikipedia main page on mobile, we have altered the mobile side menu to refer to the main page as "Main page", instead of as "Home". See this task for images.
    • Priorities
      • We are currently prioritizing "features to aid discovery", in pursuit of our goal to increase the number of newcomers who discover their homepage.
      • GuidedTour after account creation: after creating their accounts, all newcomers will be informed of how to find their homepage through a GuidedTour.
      • Button on empty contributions page: many newcomers visit their Contributions page seeking to make their first contribution, but instead they find an empty page with no results. When their page is empty, a button will encourage them to visit their homepage.
      • Redirect from email confirmation: many newcomers confirm their email addresses, but the wiki page they land on after confirming doesn't explain what to do next. This feature will bring newcomers to their homepage when they confirm their email address.

Update 2019-07-05: mobile homepage release and homepage leading indicators[edit]

  • Newcomer homepage
    • Releases
      • The mobile version of the homepage was released on June 27. Newcomers in our target wikis can now access their homepage both from the mobile devices and from desktop. We expect this to increase the number of newcomers who visit their homepage by about 50%.
      • As of July 1, the homepage is also activated in Vietnamese Wikipedia (both mobile and desktop).
    • Leading indicators
      • Before deploying the homepage, the team decided on a set of metrics that we would evaluate to see whether the feature is behaving in a healthy way. We've published our first evaluation of those numbers here. In summary, we are happy about the homepage's performance so far. We see about half of visitors clicking on something on the page, and many of them visiting multiple times. We also see that they are taking actions that the page recommends, such as confirming their email address, starting their user page, or asking a question to their mentor. One of the most interesting observations so far is that while 25 users have asked questions to their mentors, only one has asked a question to the help desk.

Update 2019-06-22: mobile homepage work[edit]

  • Newcomer homepage
    • This week has been about fixing bugs and doing validation on the mobile newcomer homepage. We currently plan to release the mobile version during the week of June 24.
    • A couple of the most important issues have been:
      • Making sure that text in some of the longer languages (such as Czech) fit comfortable on a mobile screen. We developed "scrolling tabs" to help deal with this issue.
      • Allowing homepage modules to load as overlays instead of as new pages, which will make navigation faster and smoother.
      • Making sure that all our logging data is recorded correctly so we can evaluate the impact of the homepage.

Update 2019-06-14: upcoming mobile homepage[edit]

  • Newcomer homepage
    • The team is largely focused on the release of the mobile version of the newcomer homepage. We are continuously making small fixes as the software is being tested on the Beta Wiki.
    • About 30% of Czech and Korean Wikipedia accounts are created from mobile devices, so we expect this work to open up the homepage to a substantial number of newcomers.
    • Anyone can try out the mobile homepage by going to this link on a mobile device.
  • Growth features becoming available to more wikis
    • Several features built by the Growth team are now stable enough to be turned on for additional communities who want to experiment with them.
    • We have assembled this checklist that lays out the translations and configurations needed before the features can be turned on.
    • Please contact us if you are interested in moving toward turning on the features in your wiki.

Update 2019-06-07: adding Arabic Wikipedia[edit]

  • The Growth team will begin to work with the Arabic Wikipedia community as a new target wiki. This is in addition to Korean, Czech, and Vietnamese Wikipedias.
  • Wikimania 2019 is coming up in August. The conference will include a "Community Growth" space, for sessions about how our communities expand through software and programs. If you have ideas for sessions, please submit them before the deadline on June 9.
  • Newcomer homepage
    • After about a month of usage, we see a few interesting trends that make us believe usage is going well:
      • About half of users who visit the homepage click on a link or button.
      • About half of users visit the homepage more than once, with about a fifth of users visiting on multiple days.
      • Users are interacting with all the different modules on the page -- there is no clear favorite.
      • Users have been asking questions to their mentors -- but not on the help desk.
    • The primary work right now is about making the homepage work on mobile, which is expected to increase the number of newcomers seeing it. See the accompanying screenshot for the beta version of the software currently being tested on Beta Wiki. You can also see our notes from live user tests that we did of the mobile homepage here. We expect to deploy this to users in two weeks.
    • The team is also working on several features to help newcomers discover their homepage. Our goal is for all newcomers to learn that they have a homepage and how to find it.

Update 2019-05-28: takeaways from Hackathon[edit]

  • Members of the Growth team were at Wikimedia Hackathon in Prague from May 17 - 19. One of the three Hackathon focus areas was "new editor experiences", and this was an opportunity to help the participants understand the challenges of new editors so they could work on projects that helped.
  • Hackathon was also an opportunity to spend time with the members of the Czech community who have been answering incoming questions from newcomers via the help panel and newcomer homepage.
  • We held two sessions with participants, each of which was attended by about 20 people. We also held a special session for members of the Czech community.
  • We spoke to members of many different communities and started conversations about the potential for Growth team features to be deployed in their Wikipedias: Hungarian, Basque, Slovak, Ukrainian, Armenian, Tagalog.
  • Several Hackathon projects directly related to Growth team features or to newcomers in general:
  • We heard many interesting ideas from attendees as they learned about Growth team work:
    • We had multiple discussions about the benefits and challenges with live chat for the help panel. We talked about potential hybrid solutions that would solve the problems of volunteer coverage to answer chats, and challenges around moderation. We'll continue to have this conversation on our team.
    • Maybe after newcomers have their homepage for a while, their mentorship module could ask them if they want to become a mentor.
    • After creating an account, a user's experience barely changes at all. Some newcomers don't realize their account creation succeeded, and they create a second account.
    • Maybe users should be able to drag the help panel button around the page so it doesn't get in the way.
    • Perhaps we could use the Welcome Survey to re-survey newcomers after a month or so to learn about their experience so far.
    • We could make it easy for newcomers to "thank" mentors or help desk responders right from the page itself (instead of going to View History).
    • The Homepage Impact module can also show crosswiki edits or user contributions to different wiki projects - Translate, Education program etc

Update 2019-05-10: newcomers receiving homepage[edit]

  • Hackathon: many members of the Growth team will be at Wikimedia Hackathon next week, where we will have in-person time with the Czech community and will help participants pursue projects related to the Hackathon focus area of "new editor experiences". Please visit with us if you will be at Hackathon!
  • Newcomer homepage
    Email confirmation screen -- a good moment to bring a newcomer to their homepage (shown in French Wikipedia)
    • After fixing a couple of small issues, we began the homepage experiment on May 6. Half of all new accounts in Czech and Korean Wikipedias now have access to their homepages on desktop by clicking their username in personal tools.
    • Because only 15 - 30% of newcomers will discover their homepage this way, we're working on two things to increase discovery.
    • First, we are working on the mobile version. This past week, we ran five user tests that showed we have a strong design, but that we need to be clearer in explaining the impact and mentorship modules. Full notes from these tests are forthcoming.
    • Second, we are working on specific features to increase discoverability, such as redirecting to the homepage after email confirmation, or pointing out the homepage with a Guided Tour.

Update 2019-05-03: homepage release and help panel experiment results[edit]

  • Members of WMF's Growth team working on designs for a potential "user profile" feature for Mediawiki
    Newcomer homepage
    • The homepage was deployed in Czech and Korean Wikipedias on May 2, but is not yet being exposed to any newcomers. This weekend, community ambassadors in those Wikipedias will test things out to make sure there are not any major bugs.
    • The experiment will start on May 6, in which half of new accounts on those Wikipedias will have access to their homepages.
    • We've published our plan for how we will be tracking and measuring usage of the homepage here.
    • Vietnamese Wikipedia is still in the process of translating the feature and assembling a list of mentors. We will deploy in that wiki once those elements are ready.
    • The team is now starting to build the mobile version of the homepage, which is tracked here. The current mockups of the mobile experience can be viewed here.
  • Help panel
    • We published the results of the first activation experiment, meant to detect whether the help panel causes more newcomers to make their first edit. This experiment finds no effect on activation from the help panel. This is disappointing, but it is only one of several ways we will judge the help panel. Other sources of data make us feel like the help panel has a good amount of potential that has not yet been unlocked. We will publish more information in the coming weeks.
    • The team is currently planning our next set of work for the help panel, which is meant to help more users find answers to their questions.
  • Week of planning activities
    • We published a set of notes from a team activity in which we thought about a "user profile" project to accompany the newcomer homepage.

Update 2019-04-26: homepage release next week[edit]

  • Mockups of potential designs for mobile version of homepage
    Newcomer homepage
    • The homepage is planned for release on May 2 in Czech, Korean, and Vietnamese Wikipedias.
    • Once deployed, it will be available on desktop for half of all new accounts in those wikis as part of an A/B test.
    • After deployment, the next priority will be building a mobile version. See the accompanying mockups for how this design is starting to look, and please comment here with any reactions. We'll be user testing this design in the coming weeks as we begin building it.
    • We also published the results from the user tests of our desktop design, which we have been incorporating as the release date approaches.
  • Week of planning activities
    • Last week, the Growth team had several all-day planning sessions to talk about the future of the help panel, the newcomer homepage, and of the team's work in general.
    • We will be publishing notes and decisions from those sessions in the coming week.

Update 2019-04-12: longer-term planning[edit]

  • Newcomer homepage
    • Our prospective release date for the newcomer homepage has moved to May 2 because of technical deployment schedules.
      Mockup of a potential layout for the "Your recent questions" capability in the mentorship module.
    • The final touches of the homepage's UI are now complete and being tested.
    • We will be attempting to include one additional feature before release: an easy way for newcomers who ask questions to their mentors or to the help desk to navigate back to their responses. See the accompanying mockup, and please post any thoughts or ideas you have to the project's talk page.
  • Week of planning activities
    • Next week, the Growth team will be having a series of discussions in which we'll start to lay out the future of our existing features, and to plan for our next year. We'll discuss:
      • Evaluation of the help panel and options for continued iteration
      • Prioritizing the next set of features for the newcomer homepage
      • How our knowledge of newcomers and their needs has changed, and what that means for our longer-term strategy
    • We will report back on what we learned and discussed in two weeks.

Update 2019-04-05: planning for newcomer homepage release[edit]

  • Newcomer homepage
    Screenshot of homepage in development on Beta Wiki
    • We have set a prospective release date for the newcomer homepage of April 23. During that week, we plan to deploy the homepage to half of newcomers on Czech, Korean, and Vietnamese Wikipedias.
    • The user interface of the homepage is now falling into place, with final touches being worked on now. See a screenshot from Beta Wiki on the right (which looks a lot like last week's mockup!)
    • We are in the process of getting privacy and engineering approval for our instrumentation approach, and testing that approach.

Update 2019-03-28: help panel leading indicators[edit]

  • Newcomer homepage
    • We've finished designing the start module, and have begun to implement it. It's possible to click through its various states in these interactive mockups.
      Mockup of potential layout for newcomer homepage
    • We have also decided on a design and layout for the homepage as a whole. See the accompanying image, which shows that the start module has the highest affordance, and also highlights the help module on the righthand side.
    • We are now working on instrumentation of the homepage so that we'll be able to learn exactly how newcomers use it, and therefore be able to improve. That is under development in T219250.
    • We currently plan to deploy this feature during the month of April.
  • Help panel
    • Before deploying the help panel, the team decided on a set of quantitative leading indicators that we would evaluate to see whether the feature is behaving in a healthy way. We've published our first evaluation of those numbers here. In summary, the help panel seems to be behaving in a healthy way, with good numbers of newcomers opening and using it. The leading indicators have exposed a couple areas for improvement, which we have worked on by introducing search and by displaying the feature in additional contexts.

Update 2019-03-22: start module[edit]

  • Newcomer homepage
    • Mockup of potential design for start module
      The biggest takeaway from our live users tests of newcomer homepage mockups is that users are looking for a clear place to get started.
    • Learning from that takeaway, we reframed the "account completion" module to be a "start here" module that gives clear and concrete ways for newcomers to start learning or editing. It also gives newcomers feedback as they complete tasks, showing that they are done.
    • The latest mockup of this module is in the accompanying image. The detailed specifications are in this Phabricator task. We welcome any thoughts and reactions from community members on the project talk page.
    • Next week, we'll begin work on the instrumentation of the homepage so that we'll be able to understand how newcomers do and do not use it.
  • Welcome survey
    • The experiment comparing versions of the welcome survey in Vietnamese Wikipedia has exposed that the abandonment rate for desktop accounts is substantially higher when those accounts receive the survey.
    • Curiously, mobile accounts do not have a higher abandonment rate when they receive the survey. The detailed statistics are in this Phabricator task.
    • We are not sure what is causing this difference in abandonment rate, but we turned off the survey in Vietnamese Wikipedia while we investigate. We want to make sure it is not causing good-faith newcomers to leave the site. It is possible that bot accounts are causing this.

Update 2019-03-15: plans for the coming months[edit]

  • Mockups of potential design for newcomer homepage
    Plans for the coming months
    • Our team has laid out our plans for the months of April, May, and June.
    • We have decided to focus our effort on iterating the help panel and newcomer homepage projects. We have decided not to continue work on the engagement emails project. We think that by focusing on two projects, we will be able to make more progress than by spreading across three.
  • Newcomer homepage
    • The team is currently coding on the mentorship module.
    • We have decided what the initial version of the homepage will contain. See here for the details.
    • We completed five live user tests of newcomer homepage mockups and we are analyzing the results to decide whether our approach needs any changes. The accompanying image is our latest mockup.
    • We are also in the process of deciding exactly how we will instrument and measure usage of the homepage.
  • Help panel
    • Bugs around help panel instrumentation have largely been solved and data is being recorded properly.

Update 2019-03-08: continued work on homepage and survey decisions[edit]

  • Screenshot of the state of Mediawiki's "homepage" feature in development.
    Help panel
    • First help panel questions from Vietnamese Wikipedia have started to come in. You can follow along here on the Vietnamese help desk.
    • We are currently troubleshooting some issues we're having with recording data on help panel usage.
  • Newcomer homepage
    • The latest code improves the visuals of the impact module and also adds the beginnings of the help module, which is like a help panel embedded right into the page. See the corresponding image f its current state on the beta wiki.
    • Coding has begun for the mentorship module.
    • We are planning out how we will instrument and measure the homepage. It is more complex than our previous work, because a homepage with four modules is like a feature with four mini-features, each of which have different elements we will want to measure.
  • Welcome survey
    • We have completed the experiment that compares Variation A to Variation C, which were two versions of the survey interface.
    • Variation A (the minimal interface) outperforms or ties with Variation C (the modern interface) on all measures, including response rate, abandonment rate, and activation rate.
    • Therefore, we are now showing Variation A to all new users in Czech and Korean Wikipedias, with no further experiments being run in those wikis. The experiment to compare surveys is continuing in Vietnamese Wikipedia.

Update 2019-03-01: first visuals for evolving newcomer homepage[edit]

  • Help panel
    • Help panel was deployed in Vietnamese on 2019-02-28.
    • Help panel is now being shown in the reading context on Help and Wikipedia namespace pages.
    • Team will be discussing next steps for help panel next week.
      Screenshot of the state of Mediawiki's "homepage" feature in development.
  • Newcomer homepage
    • Code has been merged that creates the homepage and adds the first module, the impact module. See the corresponding image of its current state on the beta wiki -- and note that we have not yet spent time on making the user interface look right.
    • Code is being written for the help module.
    • Working on requirements and designs for the mentorship module.
  • Engagement emails
    • Code is being written to increase "emailability" -- the number of newcomers who it is possible to be email. Changes will be tested to the "Special:CreateAccount" page and to the content of the verification email in our target wikis.

Update 2019-02-22: help panel expansion and newcomer homepage work[edit]

  • Mockup of a potential "mentorship module" in the "newcomer homepage" of Mediawiki
    Help panel
    • The help panel will be deployed in Vietnamese likely next week.
    • We will begin to show the help panel in the reading context on Help and Wikipedia namespace pages.
    • We now have enough data to evaluate the feature's performance. Metrics will be published in the next two weeks.
  • Newcomer homepage
    • Coding continues to build the homepage and the first modules: help, impact, and mentorship.
    • We are undertaking a comparative review of the homepages of other software platforms so that we can refine our designs.

Update 2019-02-15: search is deployed, new projects begin[edit]

Wikipedia help panel feature in Korean Wikipedia, with the "search" functionality in use.
  • Help panel
    • We deployed the ability to search for help to Czech and Korean Wikipedias on February 14.
  • Newcomer homepage / engagement emails
    • We have received some thoughts from community members on this project, and we're hoping for more. Please comment on the talk page!
    • We have decided to build the homepage as an additional "tab" in the User space, that newcomers will be redirected to when they click on their username.
    • The first Phabricator tasks have been created and work is beginning.
    • We are considering ideas for how to increase the number of users that have verified email addresses. See this section to read about and give thoughts on the ideas we're working on.
  • Welcome survey
    • It looks like Variation A (the simpler design) is getting higher response rates than Variation C (the more sophisticated design), in all wikis and platforms. We will publish more in-depth analysis, but it is likely that Variation A is the version of the survey that we continue to use going forward for all wikis that our team works with.
    • The early results from Vietnamese Wikipedia show higher abandonment rates than expected (people leaving the site after seeing the survey), and we're investigating those.

Update 2019-02-07: new projects for discussion[edit]

  • New projects
    • We began the "Personalized first day" project with the welcome survey so that we could gather information about what newcomers are trying to accomplish. The next step is to use that information to create experiences that help the newcomers accomplish their goal -- actually personalizing their first day. We asked for community thoughts in this update, and after discussing with community members and amongst our team, we are now planning two projects as next steps: "engagement emails" and "newcomer homepage".
    • To do To do Engagement emails: this project was first discussed positively by community members here back in September 2018, and the team how has bandwidth to pursue it. The idea is that newcomers who leave the wiki don't get encouraged to return to the wiki and edit. We can engage them through emails that send them the specific information they need to be successful -- such as contact from a mentor, the impact of their edits, or task recommendations. Please read over the project page, and comment on its discussion page with any ideas, questions, or concerns. Do you think this is a good idea? Where could we go wrong?
    • To do To do Newcomer homepage: we developed the idea for this project after analyzing the data from the welcome survey and EditorJourney datasets. We saw that many newcomers seem to be looking for a place to get started -- a place that collects their past work, options for future work, and ways to learn more. We can build this place, and it can connect to the engagement emails. The content of both could be guided by what newcomers say they need during their welcome survey, and contain things like contact from a mentor, impact of their edits, or task recommendations. Please read over the project page, and comment on its discussion page with any ideas, questions, or concerns. Do you think this is a good idea? Where could we go wrong?
  • Existing projects
    • Help panel: about 20% of users who see the help panel open it up. About half of them click on a help link. And about 5% of Czech users and 1% of Korean users ask questions. 20 questions have been asked in Czech Wikipedia, 4 have been asked in Korean Wikipedia. Since many people are trying to find help themselves, and fewer are asking questions, we're going to deploy the ability to search for help, likely next week. We've also published our written plan for measurement and experimentation on the help panel here.
    • Welcome survey: comparison of Variation A and C continues, in order to determine which gets a higher response rate.

Update 2019-01-24: welcome survey in Vietnamese Wikipedia[edit]

  • Our A/B test of the welcome survey has been analyzed, which shows that receiving Variation A of the survey does not significantly decrease the rate at which newcomers edit.
  • The welcome survey has been deployed to Vietnamese Wikipedia, with half of newcomers getting Variation A and half getting Variation C.
  • For the help panel, 9 questions have been asked in Czech Wikipedia, and 0 have been asked in Korean Wikipedia, after about 200 users in each wiki have seen the help panel button, with about 40 users in each wiki opening it up.
  • We are developing an additional feature for help panel: the ability to search for help content. This feature was appealing to user testers, and would search in the Help and Wikipedia namespaces.
  • We are continuing to plan upcoming projects, with project pages to be posted in the coming days.

Update 2019-01-18: initial report on EditorJourney[edit]

  • This initial report on what new editors do on their first day is now published on wiki. Please feel free to add comments on questions to the talk page, or to translate it into your language.
  • Variation C of the welcome survey is now being tested against Variation A in Czech and Korean Wikipedias, to determine which one gets higher response rates.
  • The help panel has now been deployed for a week, and we are seeing substantial numbers of newcomers opening it and clicking links. Asking questions has been more rare: 4 questions have been asked in Czech, and none have been asked in Korean.
  • We are beginning to work with a third community: Vietnamese Wikipedia. The EditorJourney schema is now collecting data, and we will be starting the welcome survey there next week.
  • We are planning new projects to use the data from welcome surveys. We will post initial wiki pages about those projects next week.

Update 2019-01-11: help panel deployed in Czech and Korean Wikipedias[edit]

Focus on help desk (Help panel)

Yesterday, we deployed the help panel to Czech and Korean Wikipedias. This is the Growth team's first feature built specifically to increase new user activation (editing for their first time) and retention (coming back to edit again). Going forward, half of new accounts will see the help panel button in the lower right of their screen when they open any editor on any page -- both on desktop and mobile (but not mobile apps). The other half of users are in the control group of our experiments.

We carefully instrumented the feature so we would be able to tell exactly how users do and don't use it, and we're going to be watching the data closely to see how many people open the panel, click on links, and ask questions. After about 24 hours of the feature being live, about 25 users have seen the help panel, with only one question having been asked so far -- although several users have opened the panel and clicked on links inside. We'll post more detailed numbers next week. To follow along with the help panel activity, feel free to check out these links:

Personalized first day (Welcome survey)

The welcome survey was deployed such that half of newcomers would receive it, with the other half in a control group. This was so that we could detect whether receiving the survey caused newcomers not to edit -- the concern was that putting up another barrier to editing would deter them. We are happy to say that the experiment shows that the survey does not inhibit editing, and we will publish these results in the coming weeks. Because of this, we feel comfortable testing another version of the survey: Variation C. We're going to be showing Variation C to half of newcomers, and the original Variation A to the other half. We want to find out whether Variation C's more dynamic and streamlined interface increases participation. We'll know the initial results in about four weeks.

Update 2019-01-03: initial results from welcome survey[edit]

Right now, the team is mostly focused on our upcoming launch of the help panel next week, executing on the plans described in previous updates to test and launch on time. But the most exciting new thing we are looking at and thinking about is the initial report on the welcome survey results.

Personalized first day (Welcome survey)

  • The welcome survey launched in Czech and Korean Wikipedias on November 19th, and we collected hundreds of responses in each wiki in the following month. Our team had a goal of generating an initial report by the end of December.
  • The initial report is here, in which we can see the reasons people create accounts, the topic they're interested in editing, the Wikipedia experience they have already had, and their interest in getting help from experienced editors. Here are some toplines, with the rest in the report:
    • The survey has high response rates: 67% in Czech and 62% in Korean.
    • Many people create accounts just to read articles -- a potential opportunity to engage new editors.
    • High numbers of people indicate they are interested in being contacted for help: 36% in Czech and 53% in Korean.
  • We're still digesting and thinking about these responses, and discussing how to take action on them. We're pleased to see that many people respond to the survey and take it seriously. We're seeing a pretty wide spread of intentions and topics people are interested -- this makes us optimistic that we can successfully personalize the first day.
  • A high priority analysis that we're working on now is figuring out whether being shown the survey makes a newcomer leave the wiki, instead of continuing on to edit. This is the main concern raised by community members around this idea, and it's the reason we have only been displaying the survey to half of respondents, keeping the other half as a control group. Early results seem like the survey is not decreasing activation (our word for "making a first edit"), and we will publish those results in more depth.

Focus on help desk (Help panel)

  • The help panel is still on track to launch next week to Czech and Korean newcomers.
  • The text has been largely translated on Translate Wiki, and the team is currently testing and fixing bugs.
  • We measuring using the new HelpPanel EventLogging schema, and we received both engineering and privacy approvals on our plan. We'll be publishing that plan on wiki in the coming weeks.

Understanding first day (EditorJourney)

  • The data analysis for this project is underway, and we are close to an initial report to publish for community members to see.

Update 2018-12-21: upcoming help panel deployment[edit]

As the year wraps up and people head out on vacation, we've been primarily focused on two things: preparing for the initial deployment of the help panel and assembling initial reports on "Understanding first day" and "Personalized first day".

Focus on help desk (Help panel)

  • Over the past week, we have been steadily working on the remaining tickets for our initial release. Some of the most recent code to merge includes:
    • Support for help panel on mobile.
    • Ability to add an email address or verify an unverified address through the help panel.
    • Ability to easily turn the help panel off.
    • Support for the new "HelpPanel" schema, which is what we will use to track and measure interactions with the feature. In January, we'll post our "measurement plan" so that community members can see how we'll be evaluating success.
  • The wording in the help panel is now available for translation here on Translate Wiki.
  • The help panel is currently available on the beta cluster for any community members who want to try it out. To turn it on, go to the "Editing" section of Preferences and select "Enable the editor help panel".
  • Our plan is still to deploy this feature to Czech and Korean Wikipedias around January 10, and we'll be giving plenty of notification and recommendations to those communities on how they can be ready to receive and answer incoming questions at their help desks.

Initial reports on existing projects

  • We're nearing completion on initial drafts of reports for the findings from both "Understanding first day" and "Personalized first day". We're looking forward to hearing what community members think of the results.

Update 2018-12-13: working on help panel and preparing for next quarter[edit]

Thanks to everyone who weighed in on the open questions we posted last week. We got some helpful ideas and responses. Please keep discussing! The team has been spending most of our time coding and testing the help panel so that it can be deployed at the beginning of January. We also spent time in the past week planning our next quarter. And in the time remaining this month, we are working to complete initial reports on the "Understanding first day" and "Personalized first day" projects.

Focus on help desk (Help panel)

  • We've updated our release date based on when code can be deployed in January. The help panel is now slated to arrive in Czech and Korean Wikipedias on January 10. Our goal is to finish all the code by December 21, so that when team members come back from holidays, all that's left to do is final testing.
  • We have finalized how we want to instrument the feature so that we can test how users use it, and whether it has a positive impact on activation and retention. We'll be posting a page on wiki that describes our measurement strategy and planned experiments, and a summary is available here.

Planning our next three months

  • We posted here last week about how we were thinking about our future plans for "Personalized first day", and asked for some community thoughts here.
  • After some more team discussion, we are currently planning on pursuing the "engagement emails" idea to use email as a way to re-engage editors who haven't made their first edit, or haven't returned to edit again after their first edit. We are also thinking about whether this could go together with some work on a "newcomer homepage" or "landing place" that would help orient newcomers to the resources that could help them accomplish their goals. And in order to maximize the impact of an "engagement emails" project, we are thinking about how to encourage more newcomers to create accounts with email addresses.
  • We'll post more detailed thoughts on our plans next week.

Initial reports on existing projects

  • At the beginning of this quarter, we set milestones for our "Understanding first day" and "Personalized first day" projects. Since they're both data collection efforts, we wanted to be able to generate initial reports on the early results from those projects.
  • We are still on track to create those, and plan to post them by the end of the month so that communities can start thinking about the implications of the analysis.

Update 2018-12-05: Help Panel and Variation C[edit]

Now that two of our projects are deployed, we are primarily working on our third project, the help panel. We are also working on "Variation C" of the welcome survey, which we think could increase response rate. Data analysis for both the welcome survey and for the EditorJourney work is beginning now.

In this update, there are a few places where we hope for community members to look over our work and comment. Those spots are marked with a To do To do.

Focus on help desk (Help panel)

  • Engineering continues on this project, with the release in Czech and Korean Wikipedias still planned for January 7.
  • To do To do We are currently working from this interactive prototype, which reflects our planned workflow and wording. We encourage community members to click around the prototype and post any thoughts or reactions here.
  • To do To do We have also settled on some business rules for who will receive this feature and in what parts of the wiki, described here on the project page. Please comment here with any thoughts on whether those rules are on the right or wrong track.
  • We are in the process of planning out what we will measure and experiment on with this feature, and planning out the EventLogging schemas for that measurement.
  • We conducted eight user tests via usertesting.com, and posted the results here. Users generally liked the help panel and understood how it works, both on desktop and mobile.
  • Our Czech and Korean ambassadors are consulting with their communities to decide which help links are best to include in the help panel.

Personalized first day (Welcome survey)

  • After deploying Variation A on November 20, we have been steadily collecting data. In terms of data analysis, we only have some very early numbers, and we plan to post more stable numbers during the month of December.
  • As mentioned in the 2018-11-25 update below, we are also working on Variation C, which is an improved design for the welcome survey, meant to increase response rates. We are currently deciding whether to deploy that change before or after the Christmas holidays, and how to incorporate that into the controlled experiment we're currently running.
  • To do To do The "Personalized first day" project originally started because we want to be able to personalize the newcomer's initial experience to help them achieve their goals, and keep them coming back to editing. That's why we deployed the welcome survey: to gather some information from newcomers so that we can present them with content that will help them. Now that the survey is gathering data, the team is spending time thinking about what that next step will be, and our initial thoughts are posted here. We invite community members to join the conversation here.

Understanding first day (EditorJourney)

  • This project is continuing to run and collect data. No additional work has been done since deploying it and fixing some post-deployment bugs.
  • Data analysis will being next week.

Update 2018-11-25: two projects deployed[edit]

We're excited to have deployed two of this quarter's three projects! The EditorJourney schema is now collecting data on the activities of new users in Czech and Korean Wikipedias, and the Welcome Survey is being shown to new users in both those wikis (but only to half of those new users, because half are in the control group). These deployments are our team's biggest milestones to date, and we're excited to start learning from the incoming data. As that data comes in, we'll be turning our engineering attention to the Help Pane project, and to the next version of the Welcome Survey.

Personalized first day (Welcome survey)

  • We deployed this on November 20. This was two weeks later than our initial plan of November 6. The main reason for the delay was the translation of the survey privacy statement and the privacy policy into Czech and Korean. The external translation vendor took one week to translate the documents, and then our ambassadors needed an additional week to adjust and correct parts of the translations to use wiki language. For future projects, we will be mindful of finding out in advance all the legal documents that are needed, and planning for the two weeks it takes to finalize strong translations. We also now have a set of improved best practices for working with translation vendors.
  • The survey is now being shown to half of new users in both Czech and Korean Wikipedias, and responses are being recorded in the database (the other half of new users are the control group in our A/B test, where we are checking to make sure the survey does not depress the rate at which newcomers make edits). In the next week, we'll take an initial look at the data to identify any issues or trends.
  • There are two next steps for this project:
    • Variation C: during our design research for this project, we did a comparative review of how other platforms find out basic information about their users. From that, we created an ideal design of how the Welcome Survey would look, which we called "Variation C". We think that design will maximize the number of users who complete the survey and stay on the wiki. But instead of building that design first, we quickly deployed the more simple "Variation A", so that we could start gathering data. Now that Variation A is deployed, we have turned attention to upgrading to Variation C over the course of the next two to three weeks. We plan to A/B test the two variations to see if they cause different response rates or have different effects on whether new users stay on the wiki.
    • Personalizing: the original objective of this project was to gather some initial information on what new editors are trying to accomplish so that we can give them the help content, task recommendations, or mentorship they need to be successful. We're currently discussing the right first project to work on to take action on the data we're collecting. Some of the options we're thinking about are on the short list below. But each of those things would need a place to "live", and this is making us think about ideas for some kind of "newcomer portal" where newcomers would easily be able to find the tools and information they need. We'll develop this idea a little more, and propose some specifics so that experienced community members can react and tell us what they think.
      • Making it easy for newcomers to see editing activity around the topic areas in which they indicated that they're interested.
      • Connecting interested newcomers to experienced editors.
      • Surfacing the help content most relevant to the reason for which the newcomers created their accounts.

Understanding first day (EditorJourney)

  • We deployed this on November 15. This was nine days later than our initial plan of November 6. The main reasons for the delay were some additional security reviews done by engineers from outside our team, and on the rigorous testing and adjusting we did before deployment. We're glad that the additional security reviews took place, because it's critical that our software is safe, but we'll know for the future to incorporate our dependencies on other teams into our planning. The rigorous testing and adjusting took longer than expected because of the many idiosyncrasies and edge cases in the Mediawiki software: we kept finding ways that the event data wasn't being recorded correctly. We think we have now solved the vast majority of those cases.
  • Data is now being recorded for all new users in Czech and Korean Wikipedias, and we've been auditing the data and preparing to make initial reports.

Focus on help desk (Editor help pane)

  • Engineers are now working on building the initial iteration of the help pane, though most of our attention has been focused on completing the roll out of the two other projects listed above.
  • We are currently planning to deploy this during the week of January 7. We think our engineering work will take up until the holiday season, and we don't want to deploy something new when many engineers and support staff will be taking vacations.
  • In the coming week, we'll be drafting the written copy that will be part of the feature. This copy is important, because we want to make sure new users understand what will happen when they ask a question in the help pane. That work is happening in this task.
  • We're also going to be running some live user tests, like we did for the Welcome Survey.
  • In addition to giving the ability to ask a question, the help pane will also contain a set of links to existing help content. Our ambassadors are determining the right initial set of most helpful links in this task.
  • We have some open questions around the rules and design choices for this feature. Later this week, we'll post them and start a community discussion to get recommendations from editors who have experience working with newcomers.

Update 2018-11-13: final testing on two projects, work begins for "Editor help pane"[edit]

A quick note before the update is that we've given nicknames to our projects, since "Personalized first day", "Understanding first day", and "Focus on help desk" are difficult to say and type. While those phrases still represent the overall goals of the projects, we're now also referring to them on a day-to-day basis as "Welcome survey", "EditorJourney", and "Editor help pane" (respectively), which are more specific to what we'll be deploying in the near term around those three projects.

Personalized first day (Welcome survey)

  • Current planned release to Korean and Czech Wikipedias: Nov 15
  • The welcome survey is now deployed to our testing environment where we are making sure that all the questions are laid out as expected, that no words are being cut off, that data is being recorded correctly, and that all the business rules we planned out are being followed. We'll need to test and fix for a couple more days.
  • The translations of the survey privacy statement and full privacy policy have been returned from our translation vendor, and are currently being edited by our community ambassadors from Czech and Korean Wikipedias to make sure that the vocabulary used makes sense. Posting these is also a blocker for release.

Understanding first day (EditorJourney)

  • Current planned release to Korean and Czech Wikipedias: Nov 15
  • Several team members are using Test Wiki and Beta Cluster to make sure that this data is being recorded correctly for both the desktop and mobile experiences and that all the privacy safeguards we put in place are working right. For instance, we're making sure that when users read articles in the main article namespace, the actual title of the article is not saved. Testing EventLogging is generally tedious, because it involves clicking through as many different elements of the software as possible, and then checking the database to make sure the actions were recorded correctly. A few bugs and issues have been found so far and are in the process of being fixed. The specifics can be found on this Phabricator task.

Focus on help desk (Editor help pane)

  • The team completed an initial design process for this feature in which we dreamed big, and looked into the future of what this feature could be if it shows a lot of promise. To do this, our designer drew on the review of similar features in other software and on similar Mediawiki projects from the past.
  • After going through all the ideas that came out of the design process, we decided on what the initial version of this feature would contain. This Phabricator task lists all the subtasks that describe the various design ideas. These two are the ones we'll be doing for the initial version (T206717 and T209318), in the points below.
    • A button in the lower right for newcomers, giving them the opportunity to "Get help with editing".
    • Clicking the button opens up the "help pane".
    • The help pane contains a set of links to the most useful help pages in the wiki.
    • The help pane also contains an option to "Ask a question", in which newcomers will be able to type a question that will be posted automatically in their wiki's help desk.
  • We're starting to engineer on those initial requirements and hammering out many small details. Please check out the evolving project page to follow along and see mockups of what this will look like.

Update 2018-11-02: nearing deployment on two projects[edit]

For both the "Understanding first day" and "Personalized first day" projects, we are nearing deployment within the next two weeks. Both projects are getting their final approvals and are being tested. Next week, we'll post a detailed update on Czech and Korean Wikipedias explaining what will be changing in the wikis and when.

Since those two projects are on the verge of being deployed and beginning to collect data, our team is going to be turning attention to the "Focus on help desk" project. We'll be making a full project page for that within a couple days so that it is easier for the community to follow along with how we're thinking about it.

Personalized first day

  • We settled on paths forward for the open questions from last week's update:
    • Topics of interest: since we noticed that WikiProjects differ substantially from wiki to wiki, and since we are not completely sure how we plan to use the new user's topics of interest, we assembled a list of 27 topics that appear across many different lists of important topics, including Czech WikiProjects, Korean WikiProjects, English WikiProjects, categories, and Vital Articles. The hope is that many new users will find find a topic that interests them, and if not, they can type in their own topics. We will analyze the results to decide what to do next. The topics we chose are Arts, Science, Geography, History, Music, Sports, Literature, Religion, Popular culture, Entertainment, Food and drink, Biography, Military, Economics, Technology, Film, Philosophy, Business, Politics, Government, Engineering, Crafts and hobbies, Games, Health, Social science, Transportation, and Education.
    • Interest in mentorship contact: we decided on phrasing the question such that it's clear that we do not yet have an active mentorship program. This question is now phrased as, "We are considering starting a program for more experienced editors to help newer users with editing. Are you interested in being contacted to get help with editing?"
  • We incorporated results from human testing that occurred through usertesting.com. Details on this will be added to the project page.
  • The finalized survey copy is now in translatewiki.net, and is already translated into Czech and Korean (as well as many other languages).
  • We have finalized a survey privacy statement for the survey, and we are in the process of getting it and our full privacy policy translated into Czech and Korean. This will help new users understand how their responses to the survey will be used.
  • We have decided on some of the details for the A/B test we'll be running with the feature. One of the community's main concerns with this feature is the potential that a longer sign-up process would cause new uses to leave before editing. Our experiment will give the survey to some new users, and leave the experience of others unchanged. Essentially, if they continue to edit at the same rates, than we can conclude that the survey is not causing users to leave before editing.
  • Up next, we'll be deploying the survey to our testing environments and checking to make sure everything works as expected.

Understanding first day

  • We've completed privacy review, and we are confident that we'll be handling user data in a way that helps our team learn while respecting user privacy.
  • We are finalizing some of the engineering and security aspects of this work, after getting review from several engineers around WMF.
  • We are now testing that data is being recorded and obfuscated correctly by using our test environments.

Focus on help desk

Update 2018-10-25: new project pages and open questions[edit]

Over the last week, we have made a lot of progress on the "Understanding first day" and "Personalized first day" projects. We currently hope that we can deploy both of those projects to some extent during the week of November 6th, but there are still many details to resolve in the next two weeks. We are hoping for any community wisdom to help us resolve some of the open questions laid out below. To help make it easier to follow along with our work, we now have full project pages for each of those two projects:

Those pages contain things like designs, decisions, and implementation details. The project page for "Focus on help desk" is coming next.

Below are some updates on the two projects above.

Personalized first day

Our team has been iterating on the exact wording for the text of the questions and responses in the survey. The mockups now reflect more up-to-date question wording, but we're still finalizing it. We've asked some survey experts to weigh in, and we have also engaged some human testers to try out the surveys and give feedback. We have a couple of open questions that we're not sure how to resolve yet:

  • Topics of interest: we want to ask new users about the topics they're interested in editing. That's because one day we may want to recommend editing tasks that fit their interests, direct them to WikiProjects about their interests, or connect them with other editors who share their interests. We are trying to figure out what topics to offer as choices. One idea is to use WikiProjects, but we know that the majority of WikiProjects tend to be inactive. We are also concerned that presenting very general topics -- "Art", "Sports", or "Geography" --are not specific enough to align with someone's actual interests, which may be "Ceramics", "FC Barcelona", or "Sahara desert". We're still gathering ideas and experimenting with the right way to do this.
  • Interest in mentorship contact: one of the questions on the survey asks whether the user is interested in being contacted by an experienced editor for help with their editing. This is important because it will help us gauge whether many new users are interested in mentorship, which might encourage our team to develop ideas around promoting mentorship on-wiki. We are trying to figure out a good way to ask that question without making new users wait to receive contact that won't be coming immediately, since we haven't yet developed the way to actually connect them with experienced editors. We don't want our questions to be confusing or misleading.

We are also in the process of making sure that we have thought through any data privacy elements with the survey so that users can feel good about answering the questions.

Understanding first day

We have largely settled on exactly what data we will be collecting and the questions we plan to answer with it. The project page describes those questions, and the exact technical specifications for measurement can be found here on Meta Wiki, with the associated business rules on its talk page. In short, we'll be recording the pages that new users in Czech and Korean Wikipedias visit during the first 24 hours after creating their accounts -- but only pages in non-sensitive namespaces that are important for our work, such as the Help, Wikipedia, Special, and User namespaces. For pages in the Article, File, Portal, and Draft namespaces, however, we will only record that a visit to that namespace occurred -- not the specific page. The identifying data will be deleted or aggregated after 90 days.

This new data will allow us to see the path new users take as they learn and prepare to edit. We're also going to connect the new data to existing measurement efforts in the visual and wikitext editors so we can see how the new user's path leads them to succeed or not succeed with their first edits.

Most of the technical work is complete, but we are still in the process of security, engineering, and privacy checks on our work. We're testing out the instrumentation in testing environments, and making sure that the other measurement efforts we want to use have the data we'll need.

Update 2018-10-18: starting three projects[edit]

Our team has set goals for what we'll accomplish by the end of the quarter (we are currently in 2018 Q2, which goes from October 2018 to December 2018.) Our goals can be found with the rest of the Audiences department's goals here. In brief, our goals for this quarter are:

  • Collect initial "Understanding first day" data from instrumentation, and post early numbers on wiki.
  • Collect initial "Personalized first day" data from user input, and post early numbers on wiki.
  • Deploy a "Focus on help desk" intervention experiment in production.
  • Develop a relationship with a third community beyond Czech Wikipedia and Korean Wikipedia.

Since choosing those three priority projects at the beginning of October, the Growth team has started work on each one, with most of our activity taking place in Phabricator under an "epic" task for each project:

Over the last two weeks, the team has primarily been working on "Understanding first day" and "Personalized first day".

For "Understanding first day", our engineers and data analyst have been developing the specifications for an EventLogging schema to record useful information on new editor journeys while respecting their data privacy. Our team is close to posting the specifications on wiki and in Phabricator, and it will be summarized in an update here.

For "Personalized first day", we have designed initial concepts for how this set of questions will be asked to the new account holders, and the engineers are beginning to prototype them. It's possible to explore those mockups here -- but it is important to note that the specific language in the mockups has not yet been worked on. The language in those mockups is placeholder language for when we write the finalized words that will be translated into Korean and Czech. Writing those words is our current highest priority task for this project.

Update 2018-10-02: priorities for October through December[edit]

Background[edit]

The Growth team is very appreciative of the time that community members took to respond to our ideas for increasing new editor retention.  We received helpful comments and opinions from around the world, both from newcomers and experienced editors, that have allowed us to put together our plan for the next several months of work.  Here is how we have proceeded (along with relevant links):

  1. We developed an understanding of new editor experiences and challenges through the New Editor Experiences project.
  2. We put together a list of ideas for how to increase new editor retention.
  3. These ideas were discussed in many different communities in three different languages: English, Czech, and Korean.
  4. The Growth team met to discuss and summarize all the community feedback.
  5. The Growth team made the following development plan that takes into account community thoughts and several other factors.

Theme[edit]

The theme of our work for the coming months is “Day One”. That means we are focusing on the experience of new editors on the day that they create their account.  We’re focusing on day one because our data shows that about two thirds of people who create accounts never make an edit, and of those who do, about two thirds of them never come back to edit again. This makes us think that the first day of a new editor’s experience is an important time to help them start editing and want to continue editing.  We will go into more detail on the data behind this in another update.

Plan[edit]

With the theme of “Day One” in mind, we decided to prioritize the following three ideas for Czech and Korean Wikipedias.  We are not building any of them fully. Instead, we will build a small piece of each so that we can learn whether the idea has potential, and quickly decide whether to continue.  We’ll be working on them in October, November, and December 2018.

  • Understanding first day: we are prioritizing this because it will help us learn a lot about new editors.  It will not change any user’s experience. We’ll simply add logging so that we can see what new editors do right after creating their accounts.  We will be careful with user privacy, and we hope to share initial results in December.
  • Personalized first day: this idea will also help us learn a lot about new editors.  To start, we will add some optional questions to the new editor’s registration process.  This will help us learn what new editors are trying to accomplish, and by looking at the data, and with the help of the communities, we’ll be able to think about how to personalize a new editor’s experience so that they can accomplish what they’re trying to do.  We hope to share initial results in December.
  • Focus on help desk: we are prioritizing this because it was shown to work in other contexts.  We will likely direct traffic to existing Help Desks through a bot that invites new editors on their talk pages, or through a new button in the editing experience inviting new editors to “Ask a question”. That button would be an opportunity to learn whether the “In-context questions or chat” idea has potential.  We hope to have an initial experiment running in December.

We decided to prioritize those ideas for these reasons:

  • They have support from multiple communities.
  • They address issues brought up in the New Editor Experiences research.
  • They have been proven to work in Wikipedia and non-Wikipedia contexts.
  • They help us learn quickly.
  • They can be deployed in small pieces without a major commitment at the beginning.
  • They can be deployed relatively easily to other communities.

Next steps[edit]

Next, we will be planning the work for each of those three ideas and beginning to write code.  We’ll continue to post updates, and we will be clear when we need help from community members. We have already started to create Phabricator tasks for the "Understanding first day" work. For the “Focus on help desk” project, we will need experienced editors to dedicate time to answering questions in the Help Desk, because we will expect many more new editors to be asking questions.  We’re looking forward to continuing to work with the Czech and Korean communities, and we are looking for additional communities to begin to work with.

Update 2018-09-21: summaries of community discussion and planning sessions[edit]

Over the past few weeks, we received a tremendous amount of useful thoughts and feedback on our list of eight ideas. We're grateful to the members of the many wiki communities (including several mid-size wikis) who responded on the talk page, on their own wikis, or sent us their thoughts through other channels. We were also able to get feedback on the ideas from actual new editors in Czech and Korean Wikipedias, who provided a first-hand perspective on which ideas would have helped them as they were starting out. Summaries of the feedback we received on each idea are now posted, along with some follow-up questions on the talk page.

To take action on this feedback and plan the coming months, the Growth team came together in person last week. We did two main things: short-term planning and long-term thinking.

Short-term planning

A poster made by the Growth team when discussing a potential idea for increasing new editor retention.

To determine what our team will work on first, we spent a day processing the community feedback and reflecting on the eight ideas. We split up into groups, each of which was assigned two of the ideas. For each of their two ideas, those groups read through and discussed community comments in detail. They then summarized the comments into a poster to explain the nuanced perspectives to the rest of the team. This format helped us discuss each idea, making sure to take into account the pro's and con's from different communities. Posted here is an example of one of the posters we made, to give a sense of our process. To read summaries of what we learned from communities, see this page.

Long-term thinking

The Growth team intends to be working on new editor retention for many quarters, and we know that we will need to be continuously learning about new editors in order to be successful. Though we already know a lot about new editors from research done in the past year, there are still many open questions about how new editors work and what they need to be successful. We spent time determining which questions are most important so that our team can use upcoming opportunities to find answers. Please visit this page to see our most important open questions and read about the exercise we did to generate them.

Next steps

We are now in the process of deciding exactly which things to build and in what order, taking into consideration engineering challenges, measurement challenges, and how to work with the Czech and Korean communities. Our next update will contain our plan for the coming months.

Update 2018-09-04: final week of community discussion[edit]

Over the past two weeks, we have posted the list of ideas for discussion in many places, and we would like to gather all the feedback we can by the end of this week from as many different communities as possible. If you have planned on weighing in, please do so in the next few days!

Next week, the Growth team will be taking a substantial amount of time to process and digest all the thoughts and reactions that have been posted, and we will make a plan for which ideas make the most sense to pursue, and we'll post that here. In making that plan, we'll take into account community thoughts, technical considerations, and the burden that new development will put on existing communities.

Here are the places we have posted the list of ideas for comment:

Update 2018-08-22: community discussion begins on ideas[edit]

We've now posted a list of the main ideas we're considering for the team to work on first! We hope as many people from as many communities as possible participate in the discussion on that talk page. The list of ideas is based on a lot of research and conversations the team has been taking in, but they are by no means solidified -- that's why this community discussion is really important to us. We're hoping to hear the majority of the community thoughts within the next two weeks (by 2018-09-05).

The list is also being translated in Czech and Korean and discussed on those wikis, since those are the places where the features would be deployed first. If you are interested in translating all or part of the list for your own wiki, please let us know! It would be great to have additional languages discussing.

We'll also be sending out our first team newsletter about this. Please sign up here if you are interested in receiving the newsletter.

Update 2018-08-14: narrowing down on ideas[edit]

The team is now spending an increasing amount of time planning for this first project, even as our engineers work on the Articles for Creation and New Pages Feed project for English Wikipedia. A few short updates are below, with more extensive information coming over the next week:

  • Our top priority right now is to start the conversations around which feature ideas we should pursue first. The team has been compiling many ideas for what we can build and has started to narrow to the ones that seem to have potential. We're going to be asking the Czech and Korean communities to weigh in, and we will also post them here and in other venues so that as many communities as possible can add their thoughts. We'll be looking for new ideas, thoughts on existing ideas, and information on when similar things have already been attempted. Below are some examples of the types of ideas we're talking about. More detail will be posted here in the coming weeks, as well as the reasons we think these ideas could be impactful.
    • Inviting new editors to help desks. Most wikis have a help desk, but most newbies don't know about it.
    • Making it possible for newbies to ask questions in the context where they are editing, instead of having to go to a different page.
    • Communicating a new editor's impact to them via email, perhaps by telling them about the page views on pages they've edited.
    • Gathering optional information from new editors when they create accounts about what they are trying to do and the topics in which they are interested, so we can direct them to information that serves their needs right away.
    • Instrumenting the new editor experience to learn more about what new editors do right after creating their accounts -- whether they read documentation, attempt to edit but don't succeed, or something else.
  • We have completed the staffing of our team by adding Rita Ho, our user experience designer, and Morten Warncke-Wang, our product analyst. Rita joins us from the team that built the Wikipedia Android app, and has important expertise on mobile usage that will be relevant for new editors. Morten is a longtime researcher of Wikimedia projects and has deep experience understanding the issues newbies face and experience working with Mediawiki data. Morten is going to be working on the "analyzing data" work mentioned in the update from 2018-07-06 below.
  • Several of our team members attended Wikimania 2018 and had great discussions with editors from many mid-size Wikipedias, including Arabic, Bengali, Serbian, Ukrainian, and others. We gathered information on the experience of new editors in those wikis, what is the same and what is different across them, and what specific tactics and features are working well to retain new editors. We'll be assembling and posting what we learned, along with the work our Czech and Korean ambassadors have already done on that front to catalog existing help content in their wikis.

Update 2018-07-06: data, planning, and research[edit]

While the majority of the team's effort is currently going toward the Articles for Creation and New Pages Feed project for English Wikipedia, the team is laying the ground work for this initial Growth project by doing three main things:

  • Analyzing data about contributor behavior in Czech and Korean Wikipedias. We are setting up the queries and reports that will allow us to track the contributor retention rates in our target wikis, and to understand exactly where in the editing process new contributors are most likely to drop off. We will post this data on wiki as soon as we're able.
  • Partnering with our ambassadors in the Czech and Korean communities to document the current resources that help new contributors in those wikis. We want to make sure that anything we build compliments the existing pages and tutorials that already exist, and that we are not duplicating efforts with any community members. This is the related Phabricator task.
  • Summarizing existing ideas for features that could increase retention. Many WMF staff and community members have attempted and proposed ideas for new editor retention in the past. We're making sure to collect those ideas and use them as a starting point. We will post that information on wiki as soon as we're able.

We have also spent time this week establishing the processes for our new team, such as when we have meetings, how our Phabricator board works, and how we'll balance new work with maintenance on previous features built by engineers on the team. Those previous features include Notifications and the new filters on the Recent Changes and Watchlist pages.