Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. See Terms of Use for details.
As we plan this positive reinforcement project, we are interested in community thoughts on these questions (and anything else!)
I'm glad to see work in this area—I think that all three ideas are likely to prove helpful. A few initial thoughts:
First, when developing ways to praise newcomers, I'd prefer to see approaches that are integrated into existing systems rather than introducing new ones. We already have the thanks button for small improvements and (at least at en-WP) Barnstars for more major things. Any new system on top of that would seem to me to be redundant, and the community might struggle to figure out how to fit it into our culture. Instead, I'd prefer to see functionality that perhaps flags contributions that might be worthy of thanks and makes it more convenient to hand it out. One piece of low-hanging fruit is phab:T51087, which would make it easier to keep track of who you've thanked as a mentor or what you've been thanked for as a mentee.
Second, one thing I've noticed newcomers are often very eager for is awards to put on their userpage. People just inherently like designing their profile. I'm not sure if that correlates with also wanting to edit articles or if many users just make their page and then quit, but it's something to be aware of. Things like the userbox I introduced here for completing the tutorial have gotten fairly wide usage.
Third, I've mentioned this before, but I definitely encourage improvements to the impact module to be done in a way that scales as editors become more experienced. Wikipedia has struggled for a long time with directing editor energy to the articles that are most popular with readers (as opposed to most reflective of editor biases) and valuing quality content contributions over high-volume gnoming. A good way to showcase impact that incorporated both pageviews and edit type could go a long way toward addressing those major issues. Plus, as newcomers grow into more experienced editors, it'll be nicer for them if the modules they started with continue producing meaningful data.
Thanks, @Sdkb, for taking the time to respond, and sorry for the delayed response!
I agree that we already have some existing systems that we should utilize when praising newcomers. In the designs I’ve recently added to the Positive Reinforcement page, you can see we are currently considering ways to integrate Thanks and Wikilove into the newcomer homepage. I’ve added those images in the time since you made your comment. What do you think?
I'd prefer to see functionality that perhaps flags contributions that might be worthy of thanks and makes it more convenient to hand it out.
We are still brainstorming ways to surface praise-worthy edits to mentors or other contributors, but we are thinking along this same line. How do you think we should flag contributions that might be worthy of thanks?
Thanks for bringing up phab:T51087, I can see how that would make the [[Special:Log/thanks]] more useful. For instance, in the design in this mockup, maybe that would allow newcomers to click on “View all thanks (4)” and then see exactly which edits they got thanked for. Is this along the lines of what you were thinking?
Second, one thing I've noticed newcomers are often very eager for is awards to put on their userpage
This is a great point. Current designs include newcomer awards only being visible on the newcomer homepage, but awards and recognition are likely more effective if they can be shared or seen by others. We’ll keep this in mind as we start to refine the “Leveling up” idea. What do you think of an idea where newcomers could “port” awards over to their user pages? Or do you think that should happen automatically?
I definitely encourage improvements to the impact module to be done in a way that scales as editors become more experienced.
I agree that ideally the impact module can be built in a way that is scalable and useful as editors become more experienced, but currently our team is most focused on new editors – the parts of their journey that happen over their first week or two on the wiki. At what point in the lifecycle do you think a newcomer might want to start seeing a more advanced version of the impact module?
Thanks again for all of the valuable feedback! Please let us know if you have any further feedback now that more designs and details have been added to the Positive Reinforcements page.
Broadly, the positive reinforcement seems like it's on the right track.
For surfacing contributions that might be worthy of thanks, that could be tricky. If I were to search for candidate edits, I'd look for those from a newcomer that have a high ORES score and have been seen by an experienced editor who chose not to revert them.
For T51087, yes, that's generally what I'm envisioning, although I think it's important editors be able to navigate to a diff if they tap on an edit they were thanked for, as many editors will have made multiple edits to a page and it may not be clear which they were thanked for.
On porting, generally the lower friction something is, the better adoption it'll have, particularly among newcomers. So ideally, all someone would have to do is add a template to their userpage that'd then display their awards.
Re advanced version, that'd come into play as soon as the beginner version stops presenting useful data. My understanding is that currently it only checks your past 10 edits to look for popular pages, so as soon as someone passes that count, its usefulness starts diminishing.
I'm not sure any of the projects I'm familiar with (at varying levels) have figured out how to motivate editors. I know I was a person in earlier stages saying that we don't want new users to get "stuck" in certain tasks. The keyword there is stuck. If an editor only wants to, say, add links to articles and that's what makes them happy and they want to do that over some long period of time, that's great. It's just that, as you're finding here, that doesn't necessarily help with longterm engagement.
In terms of your ideas I think Impact has the most potential, especially because good impact tools could also help with editor retention. Obviously editor retention isn't the goal of this group, but pieces that could work those dual purposes are positive to me.
The leveling up piece is one that has caused problems on English Wikipedia as a motivator. It's certainly effective, but also produces quite a bit of effort that ends up requiring time and attention from other editors, and often the community at large, to deal with. So to the extent that you are going to do that, I would encourage it to be more like English Wikipedia's "Service awards" https://en.wikipedia.org/wiki/Wikipedia:Service_awards which have a time element involved so there's less of a motivation to reach a certain level all on its own.
I also think the personalized praise option has a lot of positive potential and like Sdkb's thoughts about barnstars, userboxes, and the phab feature request.
Thanks @Barkeep49 for taking the time to offer your feedback!
Yes, we hope the leveling up idea keeps users from getting “stuck” on one task but rather starts to expose newcomers to the vast range of possible tasks available to contributors.
I’m glad to hear you think the Impact improvements seem promising! We hope this project has a positive impact on new editor retention. Retention is a key metric we will be tracking closely as we work through this project.
Could you tell me any more about leveling up causing problems on English Wikipedia? Was there a specific effort or tool that you’re thinking of? Do you mean it required more time and attention from other editors due to bad edits?
I would encourage it to be more like English Wikipedia's "Service awards" https://en.wikipedia.org/wiki/Wikipedia:Service_awards which have a time element involved so there's less of a motivation to reach a certain level all on its own.
Including a time element isn’t currently part of the leveling up design, but might be something to consider.
I’m glad to hear you think the personalized praise idea has potential. It seems like the main challenge we will face is in surfacing potentially “praise-worthy” edits to users who would be willing to offer praise or guidance. How do you think we could do that?
Enwiki's experience has been that if you make things into a game people will play it like a game. So the goal becomes the task and in accumulating volume to get that "reward" rather than maintaining quality standards. We have seen this in any number of patrol areas, including with the use of Huggle and more recently in the somewhat rocky rollout of RedWarn (now called Ultraviolet).
I agree with barkeep that goal based awards are easy, but can be problematic. We are here to make the encyclopedia, not for ppl to score as many points as possible in "add short descriptions" games. This seems very difficult to grasp for newcomers. That it is about collectively achieving something for the world. I do think that time based service awards like barkeep was saying are better at that. And note the verbiage: "service", you are doing something that is not selfish. Also "x thousand people have benefitted from your changes" might work, or things like "you helped 10 fellow editors". The encyclopedia and community building aspects need emphasis, not "x edits".
Thanks @TheDJ and @Barkeep49, you bring up good points. We definitely want to be careful about how we incentivize edits and tasks to make sure we find the right balance; we want to better onboard newcomers but ensure it's done in a way that is engaging and feels attainable to a wider audience.
I like the idea of being more service-oriented in the language we use and the awards we consider. I think it's also important to recognize that that sense of service and giving back to the community evolves over time and new editors might initially be more extrinsically motivated. Research indicates motivations for Wikipedia editors are multifaceted, and shift over time and experience. Newcomers are often driven more by curiosity and social connection, but certainly these motivations evolve if they stay engaged and continue to contribute.
Does this "New editor learning path" design seem like the right balance to you? If not, what would you change?
I don't think what you're suggesting plays to either curiosity (the way suggested edits do) or social connection (the way mentors do). It does, instead, play to extrinsic motivations of badges/rewards/status which is a real motivation that does work. The problem remains that if you're encouraging 60 easy suggested edits, the caution that an editor might feel on suggested edit 1 is different than they'll feel on 2 and different than 30 and 60. If they are doing them all right, that's great. If they make a mistake but have been motivated to get the badge you now have 60 edits that need cleaning up. As an example of the way extrinsic motivation can cause real problems for the English Wikipedia community, I'll note the community's ongoing struggles with #WPWP contests https://en.wikipedia.org/wiki/Wikipedia:Administrators%27_noticeboard/Archive336##WPWP_is_back
Got it — that makes sense. Okay, we’re going to keep this in mind as we take another look at the designs.
The Growth features have a "quality gate" that we allow communities to customize. Basically communities are able to set a maximum number of structured tasks that a newcomer can do in a day. This somewhat addresses the concern that we are setting goals that could end in a lot of edits that need cleaning up. But I realize it's not a perfect answer to that issue, and we definitely need to be careful about setting intentions that could lead to messy edits. Thanks again for the feedback!
@Barkeep49 @TheDJ Based on your feedback, we added a few more design ideas to the "Leveling Up" idea.
The first mockup is around setting some quality "gates" around the newcomer challenge. The first part of the quality gate idea to put a pause to limit further damage. This is similar to the existing limits we have in place for newcomer structured tasks.
The second mockup is to have a 'verification' timer so that even if a newcomer completes edits quickly, they will still need to wait to ensure edits aren't reverted before receiving the award.
We are still brainstorming what sharing an award would look like, but I certainly think @Sdkb is correct in the thinking that many newcomers are eager for awards or some positive recognition on their userpage.
Are we moving in the right direction with these new designs? Do you have any additional concerns or feedback?
These are definitely a step in the right direction. I find it interesting that in the new movement strategy forum (courtesy ping to @Xeno (WMF)) that they explicitly have gates before an editor can do certain things in the community https://blog.discourse.org/2018/06/understanding-discourse-trust-levels/
I think that setting daily goals could led the newcomers to make bad edits only because they want to achieve the goal. Maybe they could even start to edit mechanically, editing only with suggested edits, without getting to create articles, for example.
Also, there have to be a way to eventually encourage newcomers to create articles and edit without suggested tasks.
Thanks for the feedback, @NGC 54!
The Growth features include "Create a new article" as a "Hard" task that we initially try to steer newcomers away from, although we also provide some guidance about the task: "To successfully create a new article, you'll need to use many of the skills you can learn through completing some easier tasks. To learn more about how to create a new article, click here."
But I agree, eventually we want newcomers to learn to edit and create articles without needing suggested tasks. We hope to address this with some of the "leveling up" ideas we will focus on with the Positive Reinforcement project. Obviously we don't prevent newcomers from creating an article, but we have the option to "unlock" this task after a newcomer reaches a certain level of expertise. Do you have any thoughts on how many edits a newcomer should have before we "unlock" that "Create a new article" task? Do you think communities would want to customize that number to fit their wiki?
@KStoller-WMF: In order to unlock, maybe a few easy tasks, just a little medium tasks and at least one "Expand Article" task. But the unlocking method could mislead the newcomer to think that the article creation is possible only after completing the requested tasks. There aren't any other better solutions?
For multilingual newcomers, I think that Section Translation should work...
The most annoying thing when mentoring are the newcomers that come at Wikipedia to write about themselves or non-notable people (so there are issues with notability, COI, NPOV) or their company/business (so there are again issues with notability, COI, NPOV). They create articles and/or user pages about the topic. There have to be a way to tell the newcomers about what does a user page is and isn't (maybe some text in the help panel?). Also, there have to be some short guidelines about creating articles. These guidelines should discourage the users to write about themselves, and so on, and the new users should be aware at least of notability before creating an article, not after. But in the same time, the guidelines should not discourage the newcomers to create useful articles.
Currently, there is only an internal link to an article wizard. I do not think that too many newcomers read long pages.
Some examples: 1, 2, 3, 4. Examples can be found on the user talk pages of other mentors from ro.wikipedia.org, too.
Hello @HLHJ @Mike Peel @Czar @John Cummings @T Cells @Pigsonthewing @Sdkb @John Broughton @Pelagic @NickK @LittlePuppers @Zoozaz1 @Nick Moyes @Galendalia @Barkeep49 @NGC 54 @ValeJappo @Ата --
I wanted to invite you all to a discussion on the feature set the Growth team is working on next. This "positive reinforcement" effort is our next big project following "add an image", which many of you helped guide. The idea behind "positive reinforcement" is that newcomer tasks (like "add a link" and "add an image") are doing a good job of helping newcomers make their first edits -- but they aren't having a clear impact on getting newcomers to come back another time to make more edits. In other words, they are helping with "activation", but not with "retention". We think the "positive reinforcement" project could increase retention by showing newcomers the impact they are having and encouraging them to continue. Also, when we discussed newcomer tasks with communities, many of you wanted to make sure that newcomers wouldn't get stuck doing easy tasks, and that they would progress to more valuable kinds of tasks. With positive reinforcement, we can encourage that progression. But we know that the wikis are not like other apps and website -- we don't want to motivate people to make bad edits just to rack up points and awards. So we definitely need community input to figure out how to motivate newcomers in the right way that fits our communities.
I also want to tell you about an important transition and introduce you to @KStoller-WMF. I'm moving into a new role at WMF as the manager over the Growth, Editing, and Web teams. Kirsten will be the new product manager for the Growth team, and we're excited about her background as a librarian and as a product manager for a website that also has an active community. Going forward, you'll mostly be interacting with Kirsten around Growth product work, but I will still be working closely with her on strategy and tying projects together across multiple teams. Thank you all for your help building the Growth team features over the previous years, and Kirsten and I are looking forward to continuing to learn from you as we keep making the wikis a better place for newcomers!
Back to positive reinforcement: it would be great if you can check out the background and some of the design ideas on the project page, and post any of your thoughts, concerns, and questions in this thread or in a new one. Here are some prompts to think about:
I'm fairly certain that the WMF tried, maybe five or so years ago, thanking new contributors (on their User Talk page, I think). I believe this was done randomly, and I think the conclusion was (a bit more nuanced that this, but basically) that it made no difference. So, if my memory is right [sorry, unable to find a page to point to, regarding that effort/project], personal praise isn't a magic bullet.
Thanks for the feedback @John Broughton! I’m not sure which study you are thinking of, but you are right, personalized praise definitely won’t be a magic bullet. But based on several past studies we think it’s reasonable to expect this idea to be a step in the right direction.
Here are a few of studies we looked at that found that newcomers’ editing activity is increased (at least in the short term) when they receive thanks and other positive messages about their work from an experienced editor or mentor. Here are few of the studies we looked at:
I think one of the major challenges this idea faces is that reviewing and thanking newcomers requires time from contributors. We are still brainstorming ways to surface praise-worthy edits to mentors or other contributors. Do you have any feedback? How do you think we should flag contributions that might be worthy of thanks?
Hello Kirsten - always good to have more library folk on board! And good luck in your new role, Marshall.
We show editors a "your edit has been saved" notice each time that happens. Perhaps for new accounts, we could mae use of that? Something like "Your edit has been saved - you're doing great work - why not do another one"?
Thanks for the warm welcome, @Pigsonthewing!
Nice, it sounds like we are thinking along the same lines as you. Here’s a mockup of one idea we are considering, which displays an encouraging prompt after a newcomer edit is saved. Is this similar to what you are thinking? Do you have any ideas for improving this idea?
Generally yes, but I'm not so keen on the "1/5", "5/5", "daily goal" part. Is that goal set by the user? We shouldn't be discouraging people who might otherwise make more than five edits, nor encouraging people to make unnecessary edits just to hit an arbitrary target.
Great, thanks for the feedback! I also have some of the same hesitations regarding the daily goal. I assume allowing users to set and customize their own goals would likely be more motivating. Customization would also add to the complexity of implementation, but it's certainly worth considering. We'll keep this in mind as we take another look at the designs.
As the project page says, "We believe that these reinforcements from real people would be more effective than automated ones coming from the system."
A general suggestion: I think it's worth learning from other sites how they do it. I think that often involves, e.g., an email a day later saying thank you for your first contribution, have you considered coming back to do more, and here's some suggestions? This is perhaps more what newbies will expect, and less what experienced Wikimedians would want? But I'd also suggest contacting people that work actively on this (I don't), I know @WereSpielChequers has worked on this, and there's lots of on-wiki projects that would have relevant expertise (new page patrol, Teahouse, etc.) - and always good to ask new editors who're just past the first edit stage, what kept them going?
Thanks Mike, I spent a lot of time on Quora in recent years, and there there was much more automated feedback on things like the number of people who'd read what you'd written. Obviously not worth mentioning if figures are low, but good to send people feedback if what they've done is read by a lot of people, we do this with the DYK process but could do it more widely. As far as Wikipedia is concerned, I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects. Even if the WikiProject concerned is not currently active, it is likely to contain a bunch of resources and list some relevant and potentially tempting tasks.
Thanks, @Mike Peel and @WereSpielChequers. Agreed, it's fairly standard to receive a series of welcome emails when joining a new site. We've recently started some work in that direction, and plan to closely measure the impact: Engagement emails
One of the Positive Reinforcement ideas we are discussing includes Personalized Praise which could result in an email depending on the implementation method we pursue (along with user preferences and if the new user has an email address associated with their account). I think the main challenge we will face with that project is finding ways to encourage mentors and other contributors to spend time providing praise to newcomers who make high-quality initial edits. How do you think we should flag contributions from newcomers that might be worthy of thanks?
I think there is a big potential in tailored invites - people's first edits usually indicate some of their interests, and we could auto generate invites from relevant WikiProjects.
Currently the Growth team is using the Welcome survey to provide some personalization on the newcomer homepage. But certainly there is more we could do here. I love the idea of connecting users to the topics and WikiProjects that would appeal to them. Newcomers have so much to learn and read when they get started, it's easy for them to get lost and never find the right project or task that appeals to them. I wonder if we could use the topics a newcomer selects as interests as a way to also suggest a WikiProject that relates to that topic. Is that what you were thinking?
Do you have any other feedback or concerns about the Positive Reinforcement ideas and designs?
Hi KStoller, I was thinking in terms of the Wikiprojects tagged on talkpages of articles newbies have edited or that are relevant to categories those articles are in. This has the drawback that some tags will be completely irrelevant to the user, however transparancy could be maintained if you send hem an invite that contains links to all the relevant Wikiprojects - they can then hone in on the ones they think are relevant to them. Hi <insert name hare> based on your first ten edits, we thought you might be interested in some of the following Wikiprojects Vulcanology, mountaineering, Washington state and skiing - WikiProjects are groups of Wikipedians like you who have an interest in a certain area. OK they may have edited that article on a mountain because of the botany or abandoned radar base, but they will have picked up the principle that Wikiprojects exist and there might be ones of interest to them. As for encouraging praise of newbies, I'd add some software to the welcoming process, enable those who welcome newbies to get feedback on those newbies such as their 100th or 500th edit, with an easy opt out such as these are newbies who you welcomed and whose userpage you still watch. You could also create a page on WikiProjects that enables members to give thanks to new editors who have begun to be active in that area, though I suspect that has been covered by watchlisting etc. Perhaps a better route would be to send messages to active Wikipedians who haven't yet issued thanks or barnstars explaining how to do so now that you've done your thousandth edit, have you considered encouraging Wikipedians who you see doing good edits, either by using the thank button or giving them a barnstar? WereSpielChequers (talk) 09:07, 19 May 2022 (UTC)
Thank you for detailed feedback, @WereSpielChequers.
I agree, there is more we could do to personalize suggestions based on what a newcomer has edited. The newcomer homepage currently suggests edits based on topics the individual has selected as interests, but certainly there is more personalization we should consider as we further onboard newcomers.
And thanks for all of the ideas related to encouraging personalized praise of newcomers. As you've noted, there are definitely many different routes we could consider. Many of the ideas we have discussed involve mentors, but I like that your ideas aren't limited to just that small (and likely already very busy) subset of contributors.
Thanks KStoller, this business of newbies selecting topics of interest is new to me. Unless we are talking about userpages, either it didn't exist in 2007 or I've forgotten about it. Can you tell me what proportion of new editors indicate such interests? If it is the great majority then this might be a more effective route than the one I detailed. WereSpielChequers (talk) 10:32, 20 May 2022 (UTC)
Hi @WereSpielChequers. This is a relatively new feature, so not something you would have encountered in 2007.
As part of the newcomer Homepage and newcomer tasks, we allow newcomers to select interests from about 40 topics. Here's an example screenshot. We then use that to personalize the articles / edits we suggest.
I'll have to dig into metrics to see how many newcomers complete this step of the onboarding funnel and select interests. Parsing the data is slightly complex since the percentage of new accounts which have the newcomer homepage and newcomer tasks enabled by default varies by wiki. But certainly for users who have self-selected interests, it's likely to be a fairly accurate way to understand which topics or WikiProjects they might find appealing.
If you are interested in checking out the newcomer Homepage, here's info in enabling the Homepage. And if you have any feedback, please let me know. Thanks again for the feedback about the Positive Reinforcement project!
Thanks, that's interesting. I see a disconnect in that forty topics is too few to match with the Wikiprojects. So Sports and Biography are in the forty, but Wikiprojects function more at the level of professional wrestling or Baseball. Sports is an umbrella. WereSpielChequers (talk) 00:14, 21 May 2022 (UTC)
You are totally right, the topics are very general, and the Wikiprojects can get very specific. It's also worth considering that the smaller and medium wikis will have fewer Wikiprojects than English. And I believe that some small wikis have no Wikiprojects, so it's perhaps not an idea that would work for all wikis.
But I still love the underlying idea: that as newcomers start advancing and trying more difficult tasks, eventually we should expose them to more advanced ways to contribute and involve themselves in the community. Learning to contribute as a newcomer feels like a labyrinth of technical, conceptual, and cultural challenges, and there's so much more we can do to help ease newcomers along the path to becoming successful contributors.
Sorry, yes I was thinking of English. Taking Georgian as an example, it is a few years since I went to a Tbilisi meetup, but they had a monthly focus project that a lot of them would participate on. If that's still their modus operandi, for that wiki you'd likely promote the idea of a monthly topic. What seems obvious to me, is that this requires asking the various wikis what process we should promote to newbies. Currently because of our software limitations we don't have autosign on talkpages, so we have to promote the idea of signing with four tildas in all welcomes to Newbies. Fix that problem and we have a lot of communication bandwidth that could be repurposed for a message more positive than "Our software is so klutzy you have to use this symbol you've never heard of to sign your posts".
Thanks for the feedback and example, @WereSpielChequers. Your Georgian example is in line with a concept the Growth team wants to focus on more: ensuring Growth features are extensible and customizable to better fit the unique needs of each wiki.
And you are right, manually signing talk pages is definitely not intuitive to newcomers. I know the Editing team has many Talk Page improvements in the works that should benefit newcomers. I really hope we see an improvement in new editor retention as we start to remove some of these common stumbling blocks and the Growth team works on features to ensure newcomers receive positive reinforcement along the way.
Thanks again for taking the time to chat through these ideas!
