Growth/Personalized first day/Newcomer tasks

From MediaWiki.org
Jump to navigation Jump to search

Other languages:
English • ‎Tiếng Việt • ‎čeština • ‎العربية • ‎한국어

Growth
Projects: Understanding first day (EditorJourney) Personalized first day (Welcome survey , Newcomer homepage , Newcomer tasks ) • Focus on help desk (Help panel ) • Community resources (Get the tools )

Last news: Growth team updates Newsletters Contact

This page describes the Growth team's work on the "newcomer tasks" project, which is a specific project under the larger "Personalized first day" initiative. This page contains major assets, designs, and decisions. Most incremental updates on progress will be posted on the general Growth team updates page, with some large or detailed updates posted here.

The design and planning for this project began on 2019-07-24.

Current status[edit]

  • 2019-07-24: first team meeting to discuss newcomer tasks
  • 2019-08-27: team meeting to go over design concepts.
  • 2019-09-09: Phabricator tasks created for engineering work
  • Next: team meeting to discuss technical approaches

Summary[edit]

We think that newcomers should have every opportunity to succeed when they first arrive at the wiki. But frequently, newcomers attempt a task that is too difficult for them, can't find a task they want to do, or can't find ideas for how to remain involved after their first edit. This leads to many of them leaving and not coming back. There have been successful attempts in the past at recommending tasks to editors, and so we believe that the newcomer homepage is a potential place to recommend relevant tasks for newcomers.

We'll need to keep in mind a few things:

  • Many newcomers arrive with something specific in mind they're trying to accomplish, like add a specific photo to a certain article. We don't want to get in the way of them accomplishing their goal.
  • Newcomers build up their skills over time by progressing from easier edits to hard ones.
  • When newcomers are successful early on, they are more motivated to continue editing.

Taking those things into account, we want to recommend tasks to newcomers that arrive at the right place and time for them, teach them skills they need to be successful, and relate to their interests.

A valuable tool we have for helping tasks be relevant to newcomers is the welcome survey, which was originally built specifically for this purpose: personalizing the newcomer's experience. We'll plan to use the optional information newcomers give about their goals and interests to recommend appropriate tasks for them.

One of the largest challenges is going to be figuring out how to gather tasks that are appropriate for newcomers to do. There are many existing sources, such as templates that call for work on articles, recommendations in the Content Translation tool, or suggestions from tools like Citation Hunt. The question will be which of those options help newcomers accomplish their goals.

At first, we'll focus on using the newcomer homepage as the place to recommend tasks, but in the longer term, we can imagine building features that extend into the editing experience to recommend and help newcomers accomplish recommended tasks.

Also in the longer term, we'll be thinking about ways to tie task recommendations into other parts of the newcomer experience, such as the impact module on the homepage, or into the help panel.

The sections below may be changed significantly in the coming weeks, are too technical or less relevant for the understanding of the project. We have decided not to have them translated.

Why this idea is prioritized[edit]

We know from research and experience that many newcomers fail early in their editing journey for one of these reasons:

  • They arrive with a very challenging edit in mind, such as writing a new article or adding an image. Those tasks are difficult enough that they likely fail and don't return.
  • They arrive without knowing what to edit, and can't find any edits to make.

We also know that on the newcomer homepage, the most frequently clicked-on module is the "user page" module -- the only thing on the page that encourages users to start editing. This makes us think that many users are looking for a clear way to get started with editing.

And from past Wikimedia endeavors, we've seen that task recommendations can be valuable. SuggestBot is a project that sends personalized recommendations to experienced users, and is a well-received service. The Content Translation tool also serves personalized recommendations based on past translations, and has been shown to increase the volume of editing.

For all these reasons, we think that recommending specific editing tasks for newcomers will give them a clear way to get started. For those newcomers that have an edit in mind that we want to do, we'll encourage them to try some easy edits first to build up their skills. For those newcomers who do not have a specific preference on what to edit, they'll hopefully find some good edits from this feature.

Glossary[edit]

There are many terms that sound similar and can be confusing. This section defines each of them.

"Task recommendations" or "Task suggestions"
Lists of articles that need editing work, suggested automatically to users.
"Personalized"
Software that adapts automatically to each user to fit their needs.
"Customized"
Software that the user adapts to fit their needs.
"Topic"
A content subject, such as "Art", "Music", or "Economics".
"Maintenance template"
Templates that are put on articles indicating that work needs to be done on them.

Recommending tasks[edit]

The core challenge to this project is: Where will the tasks come from and how will we give the right ones to the right newcomers?

The graphic below shows our priorities when recommending tasks to newcomers.

As shown in the graphic above, we would give newcomers tasks that...

  • ...arrive at the right time and place for a newcomer's journey.
  • ...teach relevant conceptual and technical skills.
  • ...gradually guide users to build up their editing abilities.
  • ...be personalized to their interests.
  • ...show them the value and impact of editing.
  • ...motivate them to participate continually.

For instance, we do not want to give newcomers tasks that are irrelevant to what they hope to accomplish. If a newcomer wants to write a new article, then asking them to add a title description will not teach them skills they need to be successful.

We're splitting this challenge into two parts: the sourcing the tasks and topic matching.

Sourcing the tasks[edit]

There are many different places we could find tasks for newcomers to do. Our team listed as many as we could think of and evaluated them for whether they seem to be achievable for the first version of the feature. Written below is our determination on the task type we'll be starting with, followed by a a table listing many of the task sources that we evaluated.

Maintenance templates[edit]

We're going to be starting by using maintenance templates and categories to identify articles that need work. All of our target wikis use some set of maintenance templates or categories on thousands of articles, tagging them as needing copyediting, references, images, links, or expanded sections. And previous task recommendations software, such as SuggestBot, have used them successfully. These are some examples of maintenance categories:

Example of maintenance template on English Wikipedia

In this Phabricator task, we investigated exactly which templates are present and in what quantities, to get a sense of whether there will be enough tasks for newcomers. There seem to be sufficient numbers for the initial version of this project. We are likely to incorporate other task sources from the table below in future versions.

It's also worth noting that it could be possible to supplement many of these maintenance templates with automation. For instance, it is possible to automatically identify articles that have no internal links, or articles that have no references. This is an area for future exploration.

Full evaluation of task types[edit]

Below is a table showing the many sources of tasks that we evaluated in coming to the decision to start by using maintenance templates.

Source of task Explanation Evaluation
Maintenance templates Most wikis use templates or categories to indicate articles that need copyediting, references, or other modifications. These are placed manually be experienced users. Easily accessible. Already used in SuggestBot and GettingStarted.
Work on newest articles New articles may be good candidates for work because they likely could be improved or expanded. They are also more likely to be about current topics. Easily accessible, but most new articles are created by experienced users, and may not need help from newcomers.
Add images from Commons There are articles that have images in some language Wikipedias but not in others. This could be a good task for a newcomer who created their account in order to add an image of their own. An idea with high potential, but would require a lot of work to build interfaces. There are also questions about how to identify whether an article needs an image, and which one to recommend.
Expand short articles Many articles are stubs that could be expanded. This task is probably too open-ended and difficult for a newcomer.
Link to orphan articles Many articles have no incoming links from any other articles. Users could find articles to link to the orphan articles. Easy to identify orphans, but may be confusing for a newcomer to have to go find other articles in order to do the task.
Add references Many articles are in need of additional references or citations. Probably a challenging task for a newcomer. Frequently covered by maintenance templates.
Add categories Categories are used for many purposes on the wikis, and adding them to articles that don't have them could be a low-pressure way to contribute. Newcomers may not have good judgment when it comes to adding categories. This also does not teach editing skills that they need for other tasks.
Content translation The Content Translation tool could be a good way to structure the editing experience and help newcomers write new articles without having to generate all the content on their own. An integration here could be great -- we may want to use the welcome survey to distinguish which newcomers are multilingual.
Add sections There are algorithms in development that can recommend additional section headers based on similar articles. Writing a new section from scratch may be too challenging a task for a newcomer.
Specific link recommendation Adding links is one of the best tasks for newcomers. It would be powerful if we could not only tell a newcomer that an article needs more links, but indicate which specific words or phrases should become the link. Some research has been done on this idea that the team will be looking into, as this idea could be a perfect first edit for a newcomer.
Specific copy edits Many articles need copyediting, but it would be a better experience for newcomers if we could suggest specific changes to make in article, such as words that are likely misspelled or sentences that likely need to be rephrased. While this would be an excellent experience for the newcomer, we don't have a way to approach this. Perhaps experienced could flag specific copy edit changes instead of fixing them.

Topic matching[edit]

Past research and development shows that users are more likely to do recommended tasks if the tasks match their topical interests. SuggestBot uses an editor's past editing history to find similar articles, and those intelligent results are shown in this paper to be executed on more often than random results. The Content Translation tool also recommends articles based on a user's previous translation history, and those recommendations have increased the translation volume.

Our challenge with newcomers is a "cold start problem", in that newcomers do not have any edit history to use when trying to find relevant articles for them to edit. We have several ideas for how to allow users to indicate topics of interest in a list, or to type topics of interest into a search. Those ideas are being investigated in this Phabricator task.

Design[edit]

Comparative review[edit]

Our team's designer reviewed the way that other platforms (e.g. TripAdvisor, Foursquare, Amazon Mechanical Turk, Google Crowdsource, Reddit) offer task recommendations to newcomers. We also reviewed Wikimedia projects that incorporate task recommendations, such as the Wikipedia Android app and SuggestBot. We think there are best practices we can learn from other software, especially when we see the same patterns across many different types of software. Even as we incorporate ideas from other software, we will still make sure to preserve Wikipedia's unique values of openness, clarity, and transparency. The main takeaways are below, and the full set of takeaways is on this page:

  • Task types – bucket into 4 types: Rating content, Creating content, Moderating/Verifying content, Translating content
  • Incentives – Most products offered intangible incentives mainly bucketed into the form of: Awards and ranking (badges), Personal pride and gratification (stats), or Unlocking features (access rights)
  • Reward incentives – promote badges or attainments of specific milestones (e.g., a badge for adding 50 citations)
  • Personalization/Customization – Most have at least one facet of personalization/customization. Most common customization is user input on surveys upon account creation or before a task, most common system-based personalization type is geolocalization
  • Visual design & layout – incentivizing features (stats, leaderboards, etc) and onboarding is visually rich compared to pared back, simple forms to complete short edits.
  • Guidance – Almost all products reviewed had at least basic guidance prior to task completion, most commonly introductory ‘tours’. In-context help was also provided in the form of instructional copy, tooltips, step-by-step flows,  as well as offering feedback mechanisms (ask questions, submit feedback)

Initial version[edit]

Our evolving designs can always be found in these mockups (use arrow keys to navigate). Those mockups contain explorations of all the difference parts of the user journey, which we have broken down into several parts:

  1. Gathering information from the newcomer: learning what we need in order to recommend relevant tasks.
  2. Feature discovery: the way the newcomer first encounters task recommendations.
  3. Task recommendations: the interface for filtering and choosing tasks.
  4. Guidance during editing: once the newcomer is doing a task, the guidance that helps them understand what to do.
  5. User feedback: ways in which the newcomer can indicate that they are not satisfied with the recommended task.
  6. Next edit: how we continue the user's momentum after the save an edit.

For the initial version of the project, we've decided on a subset of the user flow to deploy. This first version is itself broken down into a few parts that will roll out to users in sequence.

  • Version 1.0: users can initiate the suggested edits module, choose task types from maintenance templates based on difficulty, and click to go to the articles needing help. This version will help us see how many users are interested in receiving suggested edits, but we are not confident many will actually follow the suggestions until Version 1.1.
  • Version 1.1: users can also filter articles by topic area, such as "Art", "History", or "Technology". We expect that this will cause many more users to select edits to do, but we don't expect many will actually complete their edits until Version 1.2.
  • Version 1.2: after users click through on suggested tasks, they receive guidance on completing the task through the help panel. With this in place, we think it will be possible for users to find tasks they are interested in, and have enough informaiton to complete them.

Below are some of the current draft design concepts as the team continues to refine our approach.

User testing[edit]

To be written

Measurement and results[edit]

To be written