Growth/Growth team updates/2023

---

Update 2023-05-05: Annual planning, Personalized praise release, and Section-level image evaluation

 * Growth team annual planning
 * Read about current Growth team annual planning ideas here: Growth/Annual_Plan_2023-2024. This is still a draft, and we will start to engage more with communities to get feedback on these ideas. The first quarter of the fiscal year will likely be focused on committed work (IP Masking project) and moving Community configuration out of the GrowthExperiments extension so it can be extended and used by other WMF teams and community developers, but plans for the remainder of the year are still to be determined.
 * The Growth team participated in a focus group call on Thursday: Moderators conversation. Several ideas were discussed that relate to Growth team work, notes are available here.


 * Positive reinforcement
 * The new Mentor dashboard module is now released at Arabic Wikipedia, Bengali Wikipedia, and Czech Wikipedia (T317817 & T334630). We will release the feature to Spanish Wikipedia soon.
 * This is the “beta” release of the new mentor dashboard module (T334630). For the "beta" release, the new Mentor dashboard module will be available to all mentors at these wikis, but notifications are disabled. This approach should allow us to collect some initial feedback before we start surfacing the feature to mentors via notifications.
 * There are a few improvements already planned based on feedback from mentors (T333036 & T333035).


 * Section-level “add an image” structured task
 * The Growth team is partnering with the Structured Data team to work on a new section-level variation of the “add an image” task. Growth team Ambassadors completed a second round of evaluation of Section-level image suggestions, and results are posted in the associated task (T330784). Suggestions improved considerably since the first round of evaluation (T316151), this is due to several improvements made to filter out sections and suggestions that are likely not a good fit. As expected the level of accuracy varies considerably by language and by the type of suggestion. The structured data team plans to ramp up the confidence threshold for suggestions, which will result in a smaller number of more reliable suggestions.
 * The Growth team is conducting a community consultation with Growth pilot wikis to review the localized designs and plans (T332530).
 * This section-level “add an image” task is one aspect of the Structured Data Across Wikipedia project, so the Growth team will need to release this task to at least one wiki by the end of June to meet grant deadlines.


 * Growth feature scaling:
 * On Monday, May 1st, German Wikipedia scaled Growth features to 100% of newcomers. Wikimedia Deutschland was conducting their own analysis of Growth features and Mentorship, so they had requested to keep a control group of 20% to conduct their own analysis of features. They reviewed data and requested to scale Growth features to all newcomers.

Update 2023-04-28: Progress on new “add an image” task & beta release of Personalized praise

 * Section-level “add an image” structured task
 * The Growth team is now partnering with the Structured Data team to work on a new section-level variation of the “add an image” task. This new task will provide image suggestions that are relevant to a particular section within an article. This section-level "add an image" task will be considered a more difficult task that will only be suggested to newcomers who are successful at the current article-level “add an image” task.
 * This section-level “add an image” task is one aspect of the Structured Data Across Wikipedia project. Read more about the Structured Data Across Wikimedia team’s work here: Section-level image suggestions.
 * This week, Growth team Ambassadors completed a second round of evaluation of Section-level image suggestions. Results will be posted in the associated task soon (T330784).
 * We also completed user testing of the "add an image" design prototype. Next week we will start a discussion with Growth pilot wikis. We will post localized designs, plans, and questions at Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, Spanish Wikipedia, as well as share further details on the "add an image" project talk page.
 * Positive reinforcement
 * The new Personalized praise Mentor dashboard module has been available at Beta and Test wikis for several weeks, and we plan to deploy a Beta version to Growth pilot wikis next week (T317817 & T334630).

Update 2023-04-21: Positive Reinforcement updates & Mentorship on English Wikipedia

 * Positive Reinforcement
 * Leveling up leading indicators and plans have been posted. Full results will be posted soon (T328055). So far results seem positive:
 * Newcomers who complete 5 edits of a given Suggested edit task type are asked to try a different task so they can gain experience with other types of tasks.  So far, almost 70% of newcomers opt to try the new task type!
 * Newcomers who completed a regular article edit will be prompted to try Suggested Edits in the post-edit dialogue on their 3rd and 7th edit. Currently about 20% of newcomers click through to try a Suggested edit.  We will track the actual completion rates and retention rates in future experiment analysis (T328057).
 * We are looking into mobile activation rates further, as we are seeing a slight difference in activation, where there should be none (T334411). Before a user edits for the first time (activates) the A/B tests should be essentially the same, so we shouldn’t see any differences in activation. We plan to follow up and investigate a few issues (T335125 & T335005) and re-run the mobile experiment.
 * Personalized praise:
 * Growth team Ambassadors and our QA engineer are testing these features and we hope to set a release date next week.  We’ve decided on a very low impact way to release these features to mentors in an incremental way (T334630).
 * Mentorship
 * English Wikipedia is recruiting Mentors via a Watchlist message that will display through the end of April. Since the message was added, the number of mentors at English Wikipedia has grown by 55%! This will allow English Wikipedia to increase the number of newcomers who receive Mentorship when they create accounts (T323045).

Update 2023-04-14: Mentorship recruitment & Community configuration

 * Mentorship
 * English Wikipedia is trying to recruit more volunteers to help newcomers via Growth’s Mentorship features. For the rest of April, the following message will display on the Watchlist on English Wikipedia:
 * A portion of new users are automatically matched with mentors to help guide and support them; experienced editors are invited to sign up to be a mentor.
 * Growth team representatives participated in a Let’s Connect Learning Clinic to share Growth tools with campaign organizers. A summary of what was shared and learned is available on our Community events report page.


 * Community Configuration
 * This year’s Wikimedia hackathon schedule will include a session focusing on Community configuration – letting communities take control by JSON configuration. This is a joint proposal from Growth’s Martin Urbanec (WMF) in partnership with Norwegian admin and bureaucrat, Jon Harald Søby (T331865).


 * New notification
 * The Growth team helped fulfill a long-standing request to add Notifications for user page edits (T3876). This was the fourth most voted for item in the 2023 Community Wishlist.
 * Newcomer Experience Projects
 * We completed a quick analysis of the Donor Thank you page work that was recently scaled to more wikis. Initial analysis shows that this flow from donation to creating an account continues to be healthy and a low-effort way to encourage new types of people to edit Wikipedia (T331495).

Update 2023-04-07: Newcomer tools for campaigns and scaling “Add a link”

 * Newcomer tools for campaign organizers
 * Growth team representatives will participate in a Let’s Connect Learning Clinic. The focus of the session will be to share effective strategies for retaining and re-engaging new editors during and after the Wiki for Human Rights campaign. The session will be held on Apr 11, 2023 and is open to all Wiki for Human Rights campaign organizers in the Central and Eastern Europe region and in other regions who are participating in this year's campaign. Learn more here.
 * We added a new help page about Mentorship for workshop or campaign organizers.
 * Add a link
 * Over 30 more Wikipedias will receive the "Add a link" task on Wednesday, April 12 (T304551 & T308133). The Wikipedia language editions that will receive this newcomer structured task include:
 * Arpitan Wikipedia frp
 * Basque Wikipedia eu
 * Danish Wikipedia da
 * Darija Wikipedia ary
 * Dinka Wikipedia din
 * Emiliano-Romagnolo Wikipedia eml
 * Esperanto Wikipedia eo
 * Estonian Wikipedia et
 * Ewe Wikipedia ee
 * Extremaduran Wikipedia ext
 * Faroese Wikipedia fo
 * Fijian Wikipedia fj
 * Finnish Wikipedia fi
 * Friulian Wikipedia fur
 * Fulah Wikipedia ff
 * Gagauz Wikipedia gag
 * Galician Wikipedia gl
 * Gilaki Wikipedia glk
 * Goan Konkani Wikipedia gom
 * Gothic Wikipedia got
 * Greek Wikipedia el
 * Guarani Wikipedia gn
 * Guianan Creole Wikipedia gcr
 * Gujarati Wikipedia gu
 * Irish Wikipedia ga
 * Lower Sorbian Wikipedia dsb
 * Manx Wikipedia gv
 * Northern Frisian Wikipedia frr
 * Scottish Gaelic Wikipedia gd
 * Tumbuka Wikipedia tum
 * Võro Wikipedia fiu-vro
 * Another group of almost 20 wikis will receive “add a link” soon (T308134).

Update 2023-03-31: Positive reinforcement and what’s next for the Growth team

 * Positive reinforcement
 * There are three main parts to this project, and soon all three parts will be released to Growth pilot wikis:
 * The new Impact module was released to Growth pilot wikis in December 2022. We’ve reported on initial leading indicators, and plan to follow up with full experiment analysis soon (T327581).
 * Leveling up features were released as an A/B test on Growth pilot wikis. We will report on initial leading indicators soon (T328055).
 * The Personalized praise project involves some improvements to the Mentor dashboard. The new dashboard module will surface “praiseworthy” new editors to Mentors, and encourage Mentors to send them encouragement. The new Personalized praise Mentor dashboard module was deployed to Beta wiki this week. After a few more improvements and further testing it will be ready for deployment to Growth pilot wikis (T322443).
 * Mentorship
 * On English Wikipedia and Spanish Wikipedia, so many new accounts are created that there aren't enough mentors to adequately support all newcomers; so currently not all newcomers are assigned to a mentor. The number of mentors on Spanish Wikipedia has increased enough that we were able to increase the percentage of newcomers that receive a mentor to 50% (T332737).
 * What’s next for the Growth team?
 * Add an image
 * We have started work on a new section-level “add an image” task, that will suggest images to new editors that likely fit within a specific section in an article (T321754).
 * We will work on improvements to the “add an image” feature based on user feedback and funnel analysis (T293711).
 * IP Masking
 * The Growth team will soon start engineering work on the product improvements needed to support the IP Masking project (T326877 ).
 * We have already been working on initial designs (T300273), preparing translated designs for user testing (T328616), and conducting user research to help refine designs (T328368).
 * Annual planning
 * The Growth team will soon set objectives and key results that align with the overall Product and Technology Annual Plan. We’ll discuss potential ideas with community members, and schedule a focus group discussion. If you are interested in participating, please just let us know on the Growth talk page.

Update 2023-03-24: Thanks usage analysis, Leveling up release

 * Positive Reinforcement
 * We completed a quick analysis of Thanks, which is now available on Commons: Thanks usage (T332441). Our quick analysis does not find a clear signal that the wider availability of the feature resulted in more edits being thanked. Patterns of usage are different between wikis, both in the sense of what proportion of edits are thanked as well as whether we see differences in January and February compared to December. There are clear differences when it comes to Thanks given by editor tenure.
 * As of Wednesday, March 22, 2023, Leveling up features have been released as part of the Positive Reinforcement A/B test on Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia (T330358)! We will carefully monitor interactions with these new notifications and features. We will share initial data soon (T328055) and more detailed experiment analysis once we’ve gathered enough data (T328057).
 * Growth team Ambassadors completed a second community consultation about Personalized praise. There were requests for additional functionality, and those ideas have been added to Phabricator.  Thank you to all community members who offered feedback on these designs!

Update 2023-03-17: Add a Link scaling & donors in more languages are invited to try editing

 * Add a link
 * A new set of Wikipedias received the "Add a link" task: Chavacano de Zamboanga Wikipedia, Min Dong Chinese Wikipedia, Chechen Wikipedia, Cebuano Wikipedia, Chamorro Wikipedia, Cherokee Wikipedia, Cheyenne Wikipedia, Central Kurdish Wikipedia, Corsican Wikipedia, Kashubian Wikipedia, Church Slavic Wikipedia, Chuvash Wikipedia, Welsh Wikipedia, and Italian Wikipedia (T304550).
 * Communities can configure how this feature works locally and view stats on how many “add a link” tasks are completed via the Growth team product KPIs.
 * Donor Thank you page experiment
 * Based on positive results from the Thank you page experiment, we scaled this change to five more language wikis: Swedish Wikipedia, Italian Wikipedia, French Wikipedia, Dutch Wikipedia, and Japanese Wikipedia (T321370). Donors land on a “thank you” page after donation, and that landing page now also includes a call to action to try editing: Example Thank you page in French.
 * Newcomer Tasks
 * All newcomer suggested edits receive a “Newcomer task” tag.  Previously that tag was linking to the Newcomer tasks project page, which contained a full project overview rather than a succinct explanation for patrollers. Each newcomer task should now link to a section within a Newcomer Tasks documentation page this is aimed to address the questions of patrollers and other content moderators (T329906)
 * Positive Reinforcement
 * Next week Leveling up features will be released as part of the Positive Reinforcement A/B test on Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia (T330358).

Update 2023-03-10: Leveling up release & upcoming shift in focus to “add an image”

 * Positive Reinforcement
 * Growth team engineers are hard at work finishing up Leveling up tasks and testing the new features. Leveling up features will be released as part of the Positive Reinforcement A/B test on Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia on Monday, March 20th (T330358).
 * We will quickly check some of the experiment leading indicators to make sure initial results look promising (T328055) and then we will follow up with a full experiment analysis report once we’ve gathered enough data to have significant newcomer retention data (T328057).
 * Add an image
 * In the month of February 2023, over 5,000 images were added to Wikipedia articles via the newcomer “add an image” feature. Since “add an image” was launched, over 36,000 images have been added to articles via this newcomer task.
 * After the Growth team wraps up Positive Reinforcement work, we will re-focus on “add an image” work. We will work on improvements to the feature based on user feedback and funnel analysis (T293711).
 * We will also start work on a new section-level “add an image” task (T321754). In other words: the current “add an image” task suggests images that illustrate the article as a whole, while this new task will suggest images to articles that already have an article-level image, but could use an image to illustrate a specific section within the article.
 * Mentorship
 * We fulfilled a community request from Spanish Wikipedia to add a talk page link for each mentee in the “Your mentees” module on the Mentor dashboard (T322750).
 * Add a link
 * Growth pilot wikis have been testing a new “add a link” prioritization formula that prioritizes articles that are more likely to be underlinked. This new prioritization formula seems to be working well for newcomers and possibly even decreasing the ratio of edits that end up reverted. We are now scaling this change to all wikis (T330535).
 * Newcomer Experience Projects
 * The Thank you page experiment proved that inviting Wikipedia donors to create accounts through the Thank you page is a sustainable and low-impact way to grow the number of new editors on Wikipedia. We are continuing to expand this work to more languages and regions.  Next Thursday this new “Try editing Wikipedia” call to action will be added to the donor thank you page on Swedish Wikipedia, Italian Wikipedia, French Wikipedia, Dutch Wikipedia, and Japanese Wikipedia (T321370). We will follow up with a quick analysis (T331495) and evaluate the funnel (thank you page visitors > account registrations > constructive activation) once we have enough data.

Update 2023-03-03: Positive Reinforcement & Add a Link

 * Positive Reinforcement
 * Impact module
 * Experiment analysis for the Impact module A/B test has started (T327581), and we plan to make some improvements to the empty state of the Impact module based on initial experiment results (T330614).
 * Leveling up
 * Engineering continues on Leveling up work, and we hope to release the initial Leveling up MVP changes to Growth pilot wikis in mid-March (T317813)
 * Personalized praise
 * Since our initial community consultation about Personalized praise plans, we’ve iterated on ideas and designs, so we started a second community consultation to review the current plan. This consultation will include  discussions on Mediawiki, English Wikipedia, Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia.
 * Engineering on the new Mentor Dashboard module for “personalized praise” continues (T317817).
 * Add a Link
 * We are preparing the backend to deploy “add an link” to more Wikipedias (T304550, T304551, T308133, T308134)
 * Testing
 * Growth features are now the default experience on both test.wikipedia.org and test2.wikipedia.org. Now that Growth features are the default for all newcomers, it was a time our testing environments had the same user experience (T330748).

Update 2023-02-24: Leveling up & Add a Link

 * Leveling Up
 * We completed a community consultation in which we reviewed the most recent Leveling up designs with the Growth Pilot wikis. This consultation was completed in English on Mediawiki, and at Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia (T328356). In general, feedback was quite positive. These two tasks help address the only critical feedback mentioned by those that provided feedback:
 * Leveling up: Community configuration (T328386)
 * Leveling up: Second design iteration of "Try a new task" dialog (T330543)
 * The Growth team is wrapping up a few remaining tasks that are necessary for the initial release of Leveling Up features.  After testing is complete and translations are in place, we will release these new features in an A/B test to Arabic, Bengali, Czech, and Spanish Wikipedias.
 * Add a Link
 * We looked at edit and revert data on Growth pilot wikis before and after the release of T301096. It appears that the new prioritization formula may result in more newcomers completing "add a link" tasks and also result in a lower revert rate (although this was not a true A/B test, and revert rates are low enough that it is difficult to draw a conclusion with confidence). Given these results combined with a prior manual evaluation by Growth Ambassadors, and positive reception from Pilot communities, we plan to move forward with releasing this new prioritization formula to all wikis that currently have the "add a link" structured task (T330535).

Update 2023-02-17: Leveling up community consultation & evaluation of section-level image suggestions

 * Leveling Up community consultation
 * The Growth team is almost entirely focused on completing Leveling Up work (which is one part of the Positive Reinforcement project).  We have started a community consultation in which we are following up with community members who offered us feedback in a previous community discussion. We are also sharing our current designs and plans, and asking some questions to help guide future work. This consultation is happening in English on Mediawiki, and at Arabic Wikipedia, Bengali Wikipedia, Czech Wikipedia, and Spanish Wikipedia (T328350).
 * Add an image: section-level image suggestions
 * The current “add an image” tool only suggests images to entirely unillustrated articles. The "add an image" feature suggests article-level image suggestions; images meant to illustrate the article as a whole. The WMF Structured Data team is working on the backend needed to provide image suggestions for specific sections in an article. These section-level image suggestions will power a new Growth team structured task for section-level image suggestions. We are starting design work on this task (T321756), and Growth team Ambassadors helped with a first evaluation of these section-level image suggestions (T316151).

Update 2023-02-10: Positive reinforcement community discussions & Suggested edits topic filtering

 * Positive Reinforcement
 * Leveling Up & Personalized Praise work continues. Designs have been localized into Arabic, Bengali, Czech, French, and Spanish so we can discuss further with Growth pilot communities (T329229).
 * Suggested edits
 * A UI improvement was released to help differentiate between and/or topic filtering (T318597). This “and” topic matching feature is currently only released to Growth pilot wikis (T305399).


 * Mentorship
 * We finished some code cleanup after the release of the structured mentor list (T321501)
 * Several Mentorship bugs were fixed (T328773, T328444, T324615)

Update 2023-02-03: Positive reinforcement, Mentorship for workshop attendees, & "Try editing" prompt for donors

 * Positive Reinforcement
 * Impact Module
 * The new Impact module is released as an A/B test to Growth pilot wikis and we’ve released several minor improvements and bug fixes (T327849, T324675, T327824). Next up we will complete experiment analysis to determine how this change has affected newcomers (T327581).
 * Leveling Up
 * We are wrapping up some analysis to better understand where we should set baseline Leveling Up prompts (T322433)
 * Engineering work has started on improving the post-edit dialog when a newcomer has topics or tasks selected that result in no suggestions (T324177).
 * The main purpose of Leveling up work is to encourage newcomers to try new and more difficult types of tasks, and our initial work on that effort has started (T322386).
 * Personalized Praise
 * The backend work needed to support the new “personalized praise” module on the Mentor dashboard has been completed (T322444).
 * Mentorship
 * Workshop hosts now have an option to claim workshop attendees as mentees (T323038). Example:
 * A newcomer goes to: https://en.wikipedia.org/wiki/Special:CreateAccount?forceMentor=Trizek|Martin_Urbanec
 * When the newcomer completes their account from this URL, then they will have either Trizek or Martin_Urbanec as their mentor.
 * Newcomer experience projects
 * Special account creation pages for donors are now available on SV, IT, JA, FR, and NL Wikipedias (T321370). This work expands the Thank you page experiment in which we encouraged donors in certain languages / on certain wikis to try editing Wikipedia.
 * Welcome Survey
 * Vietnamese Wikipedia now has a newcomer welcome survey (T325376).

Update 2023-01-27: Positive reinforcement, SpecialPages, PageTriage, & Mentorship

 * Positive Reinforcement
 * We added more details to the Positive Reinforcement project page about some initial experiment data (T323788).  And we’ve now gathered enough data that we can start the full experiment analysis (T327581).
 * We’ve been busy finishing Impact module work (T222310). The Growth team is now shifting to focus on Leveling Up work (T317813) and Personalized Praise work (T317817).
 * Growth SpecialPages
 * Now you can find all Growth SpecialPages together under “Growth tools” on Special:SpecialPages.
 * Check Special:NewcomerTasksInfo for an overview of which topics newcomers are editing on your wiki.
 * PageTriage
 * The Moderator Tools team is gradually taking over ownership of PageTriage and has a project planned to work on improvements to PageTriage.
 * In the meantime the Growth team added some better baseline instrumentation: PageTriage Debugging Grafana dashboard (T324747).
 * Mentorship
 * When someone wants to sign up as a mentor, they are now informed if they don't meet the defined criteria (T321500).

Update 2023-01-19: Newcomer experience projects & Positive reinforcement update

 * Newcomer experience projects
 * The Growth team has been partnering with other WMF teams to conduct several experiments aimed at increasing new editor activation and retention. An overview of the four main projects completed are available here: Newcomer experience projects
 * Positive reinforcement
 * There are three parts to the Positive reinforcement project, and we are nearly complete with the first. The work needed to complete the new Impact module is almost complete (T222310). The Growth team is now shifting to focus on Leveling Up (T317813) and Personalized Praise (T317817).
 * Welcome survey
 * On January 30th, we will deploy the welcome survey to Vietnamese Wikipedia (T325376). After this update, all language editions of Wikipedia with Growth tools enabled will receive the Welcome survey.
 * Newcomer experience + New article creation
 * A few Growth team members are meeting with New Pages Patrol representatives and other English wikipedia community members to discuss improvements for newcomers who come to the wikis primarily interested in article creation (meeting details). The Growth team hopes to learn more about current NPP challenges and consider how a project around Article Creation for Newcomers fits into the plans of the Growth team. If this project is prioritized we will spend more time gathering feedback from other wikis and especially focus on feedback from new editors.

Update 2023-01-13: Positive reinforcement updates and Mentorship improvement

 * Positive Reinforcement
 * We released several Impact module improvements including a change to the logic of how we are displaying “Total edits” (T325216).
 * The new Impact module is still being A/B tested on Pilot wikis. We will conduct a thorough analysis once the experiment has run long enough to gather retention metrics, but in the meantime the impact module leading indicators appear healthy.
 * Thanks has been added to Watchlists, Contributions, and Recent Changes (T51541, T90404). We will soon work on design ideas for improvements to the confirmation step (T315859).
 * Mentorship
 * We have worked on a few changes recently to help ensure that when a Mentor is blocked locally or globally they don’t continue to receive questions from mentees (T325087, T318819).
 * Add a Link
 * Newcomers at Growth pilot wikis will now receive “add a link” suggestions for articles that are more clearly "underlinked" (T301096). After discussing this change with Pilot communities and looking at “add a link” metrics we will decide if this change should be released to more wikis.