Growth/Growth team updates/2021

Update 2021-02-15: results from second variant test

 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * We're also continuing to write the copy for the feature and to develop the measurement plan.
 * Variant C vs. D - Mobile 2021-02-03.pngVariant C vs. D
 * We recently completed the analysis of our second variant test of the newcomer homepage. These tests are meant to try different arrangements of the newcomer homepage to see which ones increase newcomer engagement.  Half the newcomers get one variant, and half get the other.  Our first variant test was completed in June 2020, leading us to deploy "Variant A" to all users.
 * This test compared Variants C and D, both of which built on learnings from the first test. Both variants make suggested edits the most prominent part of the newcomer homepage, but differ in how users are onboarded to suggested edits.
 * We found that both Variants C and D led to higher interactions and edits than Variants A and B. Variant D led to more users editing on desktop and Variant C led to more users editing on mobile.
 * To take action on these results, we will soon be shifting all users of the desktop experience to Variant D and all users of the mobile experience to Variant C.

Update 2021-02-08: statistics and summaries for "add an image"

 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * We're also continuing to write the copy for the feature and to develop the measurement plan.
 * "Add an image"
 * Last week, we posted three pieces of information that provide more detail on our team's and the community's thinking about this project.
 * Notes from community discussions: a summary of the points brought up by community members in five languages around this feature idea.
 * Algorithm coverage: these statistics show how many image suggestions the algorithm will be able to make.
 * Metadata coverage: these statistics show how much non-English metadata exists for image suggestions.
 * Mentor dashboard
 * We're conducting interviews with mentors in multiple communities to inform the design of this feature.
 * If community members want to weigh in with their opinions, they can contact us on this talk page.

Update 2021-01-31: user test results for "add an image"

 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * We're also continuing to write the copy for the feature and to develop the measurement plan, which will explain which data we're measuring and how we'll set up any A/B tests.
 * "Add an image"
 * In December, we ran 15 user tests with newcomers on the "add an image" idea, using this interactive prototype. The objective was to observe as many user judgments on image matches as possible, so we can understand the strength of user judgment and figure out how to design to increase it.
 * The full notes on the user tests are here. We are encouraging community members to read and react to them on the project talk page.
 * In summary, we think that these user tests confirm that we could successfully build an "add an image" feature, but it will only work if we design it right. Many of the testers understood the task well, took it seriously, and made good decisions. On the other hand, many other users were confused about the point of the task and made weak decisions -- but for those confused users, it was easy for us to see ways to improve the design to help them succeed.
 * Another important development for this project is the start of work on the Android MVP (minimum viable product): the Android app team will be building a simple image matching workflow as a "suggested edit" in the app. The MVP will not save any edits.  It is only for the purposes of learning how to improve the image matching algorithm and the design, so that when the Growth team works on this idea, we can benefit from the learnings.

Update 2021-01-25: progress on "add a link"

 * Team changes
 * Our team has been undergoing some changes as we gain new members and old members transition to other teams. We're expanding, which is good!  But it means we'll be spending effort over the next few weeks onboarding new members and helping them get up to speed.
 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * The beginnings of the experience will start being available in beta wikis this week. We will let volunteers know when the pieces start being ready to try out.
 * We're now writing all the copy for the feature (the words that users will see).
 * We're also writing our measurement plan, which defines which data we'll gather and how we'll use it. This will get posted on-wiki for community members.
 * "Add an image"
 * We've finished gathering our notes on the user tests that we ran around this idea. We'll be posting our findings on-wiki this week so community members can see them.
 * The Android team is going to be building an MVP (minimum viable product) around image recommendations. It's not going to make any edits to Wikipedia: it's just going to help us learn how users react to an image-related task, and give us a sense of how strong their judgment will be when they do the task.  We'll link to more details on this idea once it takes shape.

Update 2021-01-18: design work on mentorship

 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * "Add an image"
 * We are finishing up the final threads on the discussion page for this project. We'll summarize the state of the conversation this week, but please feel free to join in at any time.
 * We'll also be posting learnings from our 15 user tests on the image recommendation prototype.
 * On the technical side, the Platform Engineering team is scoping out a "proof of concept" API for serving image recommendations. The Growth and Android teams will be able to use this to try out building an "add an image" feature and get a sense for hw such an API would function.
 * Mentorship dashboardSketch_of_mentor_dashboard_feature_for_Mediawiki_2020-05-11.jpg
 * Mentorship is an important part of the Growth team feature set, and it's the one in which community members participate most, and have the most ideas and questions.
 * But because our team spends the majority of our resources on newcomer tasks, we have not been able to dedicate time to improvements for mentorship.
 * The team now has a design intern working with us for the coming months, who will be focused on mentorship features.
 * Over the last years, the most requested mentorship feature is some sort of page where mentors can see who their mentees are and monitor how they are progressing.
 * The design intern will be working on building this, using the existing project page as a starting point and the existing community thoughts on its talk page.
 * If you have opinions or ideas, please speak up on the talk page so we can take them into account.

Update 2020-01-10: continued discussion on "add an image"

 * "Add a link"
 * We are continuing to engineer on both the backend of the link recommendation service and the frontend of the user experience.
 * "Add an image"
 * We are still having active discussion on the discussion page for this project. We hope community members can join in.  Some of the important thoughts being discussed include:
 * How the existence of Wikidata infoboxes might affect how we want to design this project, and where images should be placed in articles.
 * To what extent non-English speaking users can participate in this feature, given that most metadata in Commons is in English.
 * Next week, we'll summarize what we've learned so far from community conversations around this project.