Topic on Talk:Growth/Personalized first day/Structured tasks

What are your main reactions to structured tasks? Do you think this could be helpful to newcomers and to your communities?

It would be nice to present a newcomer (perhaps after he/she indicates interest, say by clicking a link) with a set of structured tasks that he/she can do, as a way of becoming a more experienced Wikipedia editor. So: add an image, add a citation, do a copy edit, add a (wiki)link, add a category, add or change an article assessment (on a talk page), revert vandalism, and so on.

My concern is the the team is jumping into this process by building a structured task for something with a relatively low added value to Wikipedia - converting words in articles into internal links (wikilinks). Articles with too few internal links (or none at all - orphans) are typically articles that are rarely read. And while teaching someone to create a wikilink is straightforward, teaching them when - and when not to - create such a link is another matter - see https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Linking#Overlinking_and_underlinking .

More generally, I would urge the team to revisit the table at Growth/Personalized first day/Newcomer tasks#Sourcing the tasks, to think about what the first handful of structured tasks should be. And by "revisit", I mean getting the community actively involved in revising and expanding the table, since it looks like this is going to be a roadmap of sorts.

@John Broughton @Sdkb -- thank you for reading the project page and weighing in. I'm learning a lot from your input, and I'm glad to hear that your general impressions are positive.

I understand your doubts about whether "add a link" is the best place to start. We know that it's a lower-value edit than other kinds. Let me tell you why we're leaning there, and you tell me what you think:

• In the long run, we hope there could be many different kinds of structured tasks, like @John Broughton says (add an image, add a reference, etc.). But in the short run, the most important thing we would want to do first is to prove the concept that such a feature could work. Therefore, we would want to build the simplest one, so that we can get some software out there and see how it goes, without having to invest too much in the first version. If the first version goes well, then we would have the confidence to invest in types of tasks that are more difficult to build.
• When thinking through which one would be the simplest for us to build, we gravitated toward "add a link" because:
  • There already exists an algorithm built by the WMF Research team that seems to do a good job of suggesting wikilinks. We tested it in Czech, Korean, Arabic, and Vietnamese, and it looks like about 70-80% of suggestions it makes are good ones. That number puts it into the right range for a "human-machine partnership" -- in other words, the algorithm is smart enough to point the human to the right spots, but not so smart that it doesn't need the human to confirm its work. What this means is that the user can hopefully make good edits and absorb good linking habits from the algorithm's recommendations without having to understand all the concepts right away. For details on the algorithm, see this Phabricator task.
  • Adding a wikilink doesn't usually require the newcomer to type anything of their own, which we think will make it particularly simple for us to design and build -- and for the newcomer to accomplish.
• Part of building this well will be figuring out how to teach concepts to the user. We're actually going to be taking our first step in that direction in the next few weeks when we deploy "guidance", which makes simple help content available to the user while they're doing their suggested edits. We worked hard on the content so that it is brief, explanatory, and contains examples.
• The plus side of "add a link" being a low-value edit might be that it is also low-risk -- in other words, perhaps an article can't be too damaged by overlinking, whereas mistakes with adding references or images would be worse.

Regarding other task types, I think @John Broughton is right that the list here could, in an ideal world, become a sort of roadmap of future structured tasks (but only if the first ones we try show promise). I actually think "add an image" would be a good one to think about next, because a really simple algorithm could be: for an article with no images, see if that article in another language has images from Commons. If so, recommend that image. The Research team has confirmed with me that that is possible, plus more sophistication using machine-learning.

What do you think of all this?

That makes sense that it's best to do a simple proof of concept-type thing first. Thanks for the explanation!

@John Broughton Hi, I'm Phuong, I'm the Vietnamese Ambassador of the Growth team. I want to share my perspective on your opinion that "Articles with too few internal links (or none at all - orphans) are typically articles that are rarely read". Well I think it's only the case for big Wikipedias like English Wikipedia. In medium Wikipedias like the Vietnamese, we have a lot of articles that are fairly important but have few to no links. The article quality in Vietnamese Wikipedia is not very high as we don't have enough man-power to polish every articles. So a structured task for adding link would be a great start for us medium Wikipedias : )

Thanks, Phuong, I was planning to ask whether small–medium Wikipedias have more underlinking than overlinking.

It would be great if the tool could also detect and suggest removal of excessive links, so that we’re not training a cohort of newcomers to overlink indiscriminately.

@Pelagic I would have envisaged that the software would only identify the first example of a word that needed wikilinking, and would not offer a user the chance to link to an already-linked word.

I think that inviting a new editor to remove overlinked words would require a greater understanding of why this isn't a good idea. Wouldn't they just say to themselves "why do I need to do that - links are good, surely?". So under-linking ought to be easier for a new editor to fix than over-linking.

@Pelagic -- interesting that you should mention this idea of turning the algorithm on its head to point out links that should not be present! The researcher working on the algorithm suggested this as a possibility, and we didn't think it would be a very compelling task for newcomers. But I do like the idea that by mixing in some suggested removals with the additions, that might go a long way toward teaching newcomers that there can be too many links, and that judgment is required -- perhaps it might be more effective than just telling them that in the instructions. Tagging our team's designer, @RHo (WMF), so she can think about this.

@Nick Moyes -- yes, the algorithm will be able to include the rule of only adding a link for the first occurrence of a phrase, so I think we should be covered there. We've also discovered through testing that there are a host of other conventions that we want to respect, which we're trying to incorporate into the algorithm. Things like:

• Not linking years and dates (some wikis have this best practice).
• Linking the largest possible phrase, e.g. "London School of Economics" instead of just "London".
• Not linking in the References section of an article.

If you can help think of other "gotchas", we would love to add them to the list (we're keeping them in this Phabricator task). I know we can also draw on the manual of style to sharpen the algorithm.

To add to what John said, I think it would be much easier to have a filter somewhere where we can select the new users in a certain period with the number of contributions in the main space. We have a conglomerate of tools to use, however, the keyword being "tools" I feel we should have one tool that would pull this information for us or edit the "User Creation" special page to allow us to do that. That will help out in getting to the new comers.

It would also be great to have an automated script push out the welcome template (I have one I particularly like due to the content) to the new users after a certain threshold is met (say since account was created & number of edits & in time frame & no reverts).

@Galendalia -- I think what you're saying is along the lines of how a few things work already. For instance, Teahouse invites automatically go to the user talk pages of newcomers who have shown that they are generally acting in good faith. The main way that newcomers currently find their way to Growth team features is through the newcomer homepage. After a user creates their account, a few different things (popups, buttons) point the user to their homepage as a place to get started. This works well so far -- about half or a bit more of newcomers winds up checking out their homepage (in wikis that have the feature).

Maybe I am wrong here but an editor has to post an invite to the TeaHouse in en wiki. I know I’ve had to do it to a couple new editors. It’s the same with the welcome page it has to be done manually to an editors talk page.

@Galendalia, I come across automated Teahouse invitations from HostBot all the time. It's not done immediately/for everyone, but there's some set of criteria that triggers it.

Thanks for that @Sdkb:. Looks like they only send out once a day that’s why I don’t see them.

Seconded about wikilinks being more difficult than it might appear; that's something I previously mentioned.

Overall, this is something that I'm excited to see implemented. It seems like it has good potential to help newcomers feel comfortable making edits.

I would say automated tasks should be clearly useful and quickly make newbies understand what they are doing. The worst thing that can happen is an automatic tool suggesting a wrong edit and a newbie gladly accepting it (e.g. adding a link cs:rozvoje in cs:Société Générale is wrong), it would be frustrating both for the community (some WMF guys did it wrong again...) and for the newbies (they might get reverted on their first edit).

@NickK -- thank you for joining the conversation! I'm glad to hear you think structured tasks would be useful. Your comment reminds me that in the Android app, they keep track of how many times a newcomer's structured tasks are reverted (there, they are mostly adding Wikidata title descriptions). If the newcomer gets reverted too many times, they are no longer allowed to do structured tasks. This is an idea we can think about for our version of structured tasks.

@MMiller (WMF): I think there is a misunderstanding. Structured tasks would be useful indeed. My point is that automated tasks should not suggest something that is not clearly useful. I don't want newbies to be victims of bad suggestions in tools. If a newbie is suggested some link that in reality is wrong, adds it and gets reverted, that would be a major frustration.

@NickK -- you're right, I did misunderstand. Thank you for clarifying. I definitely agree that if the system urges newcomers to make edits that then end up getting reverted, both the newcomer and the patroller will be frustrated. I think that anytime we are using an algorithm, there are going to be some mistakes, no matter how hard we work to improve it. We know this already from ORES models, which are usually right, but sometimes wrong (which is why humans need to make the final decision, instead of reverts happening automatically based on ORES). I think we can try to prevent the frustrating outcome in two ways:

1. Try to make the algorithm as good as possible, of course. For the link recommendation algorithm we have now, this is the Phab task about making improvements.
2. Design the interface so that the newcomer knows that the suggestions come from an algorithm that can be wrong sometimes, and understands that the important part is that they use their judgment with the suggestions. And we will have to show them the concepts to use so that they develop good judgment.

Does that sound right?

Thank you, this is the approach I was thinking about.

Regarding 1, it is too English-centric. For instance, French Wikipedia recommends linking dates, and I believe concepts of names and common words will be different in Chinese. In Slavic languages, for instance, grammar and indirect cases will be the main problem, i.e. we will need to use a dictionary. E.g. to add a link to the article France (uk:Франція) to uk:Société Générale), you would have to link the world французьких (genitive plural adjective).

On the second point, I would suggest adding one-line description of the article the link is being added to, this should eliminate the most irrelevant ones.

Got it, thanks @NickK. So far we have gotten strong performance out of the algorithm in Czech, Korean, Arabic, and English. But we saw only moderate performance in Vietnamese. I definitely have concerns that there will be languages that have issues. As we try to expand it, could we ask for your help in evaluating how it performs in Ukrainian? Perhaps that can be our first Cyrillic language.

For the second point about adding a one-line description of the suggested article, that is probably something that our team's designer, @RHo (WMF), is thinking about, but tagging her just in case.

@MMiller (WMF): Thanks, that's interesting. Ukrainian grammar should be overall similar to the Czech one, so the approach should work the same way. I think the Slavic factor should be more important than the Cyrillic one, but I might be wrong. Is it possible to test the Ukrainian algorithm if it is already available, or the Czech one if it is not? Thanks

@NickK -- you're welcome to look at the Czech results that we already have. The materials are in the task description here: T245330. The way it works is that we uploaded 20 Czech articles to Test Wikipedia with links created by the algorithm (they're all red because none of those articles actually exist in Test Wikipedia). Then our Czech ambassador, @Martin Urbanec (WMF), went through and added green or red templates to indicate whether they look like links that should be made. The counts of those are in the task description, and the total was that it looks like 86% of the suggestions should be links. Does that make sense?

As we continue to improve the algorithm (happening in T253279), maybe we can try Ukrainian as the next language. I will get back to you about it.

Yes, my current thinking is we would re-use the same pattern of showing the wikidata description under the linked article title much like when it is shown in VE when a link is selected (as shown in this screenshot) or search results on mobile.

I like that idea. It'd be good to have a softer warning, though, after the editor's first or second reversion, giving a gentle nudge toward reviewing the pertinent "learn how to do this" material.

Having now managed to read through the project page and these discussions, I'd like to offer my support.

As I understand, WMF want to develop and test a simple but effective way for newcomers to make quick, efficient edits (of one particular type to begin with) that will be:

• Easy to develop and trial
• Easy for newcomers to enhance Wikipedias without having to learn the complexities of editing processes
• Not too damaging to articles if misused
• Expandable (if successful) to include other types of structured task.
• Encouraging and motivating for newcomers to use.

I welcome this idea. wholeheartedly. Whilst the opportunity to make very small improvements in this way without having to directly edit pages won't teach someone how to open and edit a page, it will immediately give someone the feeling that they have made a small, but worthwhile contribution to the encyclopaedia that "anyone can edit". I suspect that if they're able to get feedback and satisfaction from that task, they may move on to other structured tasks, and may eventually become interested enough to do more in-depth editing.

A by-product of this approach will be to offer experienced editors who are "on the go" to be able to make small, easy contributions on a mobile, whether in a bus queue, doctor's waiting room or (ahem) under the covers at night, unable to get to sleep.

I don't understand the explanation in the table of Newcomer Tasks that offering discrete copyedits such as spelling checks as one of the early structured tasks is too difficult. It seems that if an article already contains a probable error, very little harm can be done by changing it to another spelling variant. And I would have thought it would have been quite easy to integrate spelling dictionaries from a multitude of languages into a structured task. I have spent many a lunch hour using either Lupin's spellchecker or AutoWikiBrowser to fix things like 1980's > 1980s, or recieved>received, and derived quite some satisfaction of time well spent when I couldn't do more technical work. Doing that on the go via Structured Tasks would have been brilliant. If it's not too much trouble, could I ask for a (simple!) explanation why finding and presenting probable spelling errors is too technical a task to achieve? It seems counter-intuitive not to offer spelling errors. On en-wiki we even have our own list of common errors (see here).

Failing that, I would have thought that adding recommended Wikilinks would be slightly less rewarding but, again, relatively little harm would be done if the wrong word were linked to. So I would support this as a trial Structures Task if Spell-checking is really off the cards.

@Nick Moyes -- wow, thanks for engaging deeply with this content and for summarizing it. I think your summary is really correct clear -- clearer than what I've written -- and I'm sharing it with others. Following the suggestions of many community members on this page, we went out and learned more about the prospects for copyedit and spellchecking tasks, and I'll post about it under a different heading so we can discuss further.

I have to admit I am not satisfied with the application or using the mobile web browser on a standard phone. A phablet or tablet would work, but, I for instance, do not always carry my ipad with me and editing it difficult on my phone.

I use a tiny iPhone 5S for a lot of my editing (like replying right now) but rarely for activities requiring more than 4 or. 5 sentences. I see Structured Tasks as ideally suited to a mobile phone.

@Nick Moyes You need a bigger phone then lol. I can see that for the tasks, but I am speaking to the end results of editing a full article to bring the status up to a new level. I find it hard to add citations, references, wikilinks, external links, etc. Just typing out a few sentences, yeah that is ok. When you look at the tasks overall though, that is about the only thing the mobile is good for. So I guess in a nutshell what I am trying to say is the mobile experience would not be good for the task list of things to be completed.

@NickK has reason to worry about poor suggestions: example.

The "editing is hard" list on the associated project page does not quite reflect how I began to edit. My flow was more:

I was already reading an article on a topic of interest, and found an informational omission (not an editing-goal-directed behaviour, happens spontaneously). Then:

1. Click to start editing (one option).
2. Type the sentence in the right place (easy, it's a markup like HTML). Preview until it looks right (a great confidence-builder).
3. Fill out an edit summary (because it asks).
4. Click to publish the edit.

Note the lack of citations; when I did later add citations, they were usually bare URLs inside ref tags. I was also attracted to the informational content; had I been offered semi-automated tasklets, I would have found it very boring. The app may be good at recruiting wikignomes (and I really value gnomes), but I'm not sure it will recruit editors with other interests effectively. While we can never have too many gnomes, I fear that the app may put off other potential editors, who might get the idea that the app-recommended tasks are all of what editors do.