Flow/Research

New User Test Videos
The Foundation conducted several user tests to get a glimpse into the problems facing new users when they are exposed to what are relatively simple (and common) scenarios with user talk pages.

Note that the links to these tests leave the Foundation website and go to an external site, usertesting.com, which may have a different privacy policy than the Wikimedia Foundation's.

Tests have been annotated, calling out specific issues that users face and behaviors that they expect.

Test 1
This test was a very simple task for new users who have made an edit and then were reverted. We tested to see if the user was able to see that they had new messages and were able to successfully reply to the messages. The page was a very simple and common scenario.


 * Aeldar02
 * Online Bee
 * Paul

Test 1.1
This test was the same as the first except for a minor clarification in one of the tasks (referring to a specific edit to "Dwarf cat" over generically described "edits").


 * GlobalE

Test 2
This test involves a more complex scenario, wherein the user has asked for help from another user. It involves a "talk back" template and introduces the concept of minor threading.


 * Kayaker
 * kawzx7

Test 2.1
This test is the same as the previous, only fixing a small bug in the text of the test.


 * PJensen66

Quora Question
User:Jorm (WMF) asked on Quora: What do you find confusing about Wikipedia's discussion systems?. Here is the question and answers.

What do you find confusing about Wikipedia's discussion systems?
Not everyone is aware that there actually *is* a discussion system on Wikipedia. There are two basic types: discussions about articles and then per-user discussion spaces (these are called "Talk pages").

I am currently working on a design proposal to build a better discussion system. I know what I think is confusing about Talk pages (there's a big list) but I want to hear what other people think, too, so that I can fold issues I may have missed into the design.

Pete Forsyth
Pete moved his answer from Quora to mediawiki.org

Since perspective is important on this, here's where I'm coming from: I'm a long time (8 year) heavy user of MediaWiki (many different sites), most notably English Wikipedia; and I do a great deal of work with first-time and developing wiki contributors. So, my comments reflect both my own needs, and my perception of the needs of the people I teach.

Things that don't get discussed much, but are big opportunities for improvement:
 * Links (to talk pages, to section headers of talk pages) do not persist through through archiving. Asynchronous, eventualist communication is one of the core strengths of wiki software; this is a major exception. (e.g., I link today to a conversation -- from Twitter, say -- but next month, that link goes nowhere useful.)
 * It's difficult to provide context. Outside Wikipedia, most of my collaborative writing is in Google Docs; and the "comment" feature, where you highlight exactly the chunk of text you are talking about, is incredibly useful. I find myself missing the ability to easily point others to exactly what I'm talking about frustrating.
 * There's no easy way to "search ahead" when typing somebody's username or other internal link. The HotCat extension has this capability; on Twitter and Facebook, when I type "@" followed by the first few letters of a friend's name, it starts to guess what I mean. A feature like this in MediaWiki would be hugely helpful.
 * No "ping" notification. If somebody tags me in Facebook, I am notified; if somebody links my name in a Wikipedia discussion, there's no notification capability.
 * Posting diffs can be a really important element of descriptive or persuasive discussion. A quicker and/or more intuitive way to find and paste a diff would be a big help.

Obvious/well-known issues with perceiving what is going on:
 * When somebody comments on my user talk, do I reply on my own page or on theirs? There are pros and cons to both.
 * When somebody replies to me, I'd like to be notified. "Watch list" is too  general, expecting them to tell me on my own user talk is too onerous.
 * Same as #2, but cross-wiki: multiple languages, multiple projects.
 * Too-specific time stamps result in a whole lot of clutter. It's very, very rare that information about what precise time a comment was left is important to the discussion; if that information is preserved a click away (e.g. in edit history), it could be left out of the display in the context of discussion. Perhaps dynamically: for instance, anything more than 3 hours old suppresses the minutes and seconds, anything than two days old suppresses the time of day.

Obvious/well known issues with adding comments:
 * I shouldn't have to think about whether/how much I am indenting…much less need to use markup to do so.
 * I shouldn't have to remember to sign.

Waldir
This was a reply to Pete's comment on mediawiki.org

This includes most of the points that annoy me about the current discussion "system" in mediawiki. There are a few things that I would add, though:
 * Thread-independent management. Have each thread be a single entity would allow watchlisting only the topics you care about, moving threads to more appropriate pages, and archiving without losing edit history, or, if archival is done via moving the page to preserve the edit history, without having to split the page at unnatural places (e.g. if organizing archives by year, the first posts of a still active thread near the end of the year will have to be copy-pasted back to the current page, losing history; if organizing archives by number of threads, reactivating a prematurely archived thread would require manual copy-paste and loss of history)
 * Native support for quoting. This is especially relevant for long messages, since inserting replies below parts of the original post break its flow and make it unclear who is writing what, and quoting templates are a hack that can't, for instance, link reliably to the original post being quoted.
 * Features for consensus-making. Wikipedia discussions can get quite big and it would be great to have a way to provide summaries for threads, and voting for individual posts, so people who come across a long discussion don't have to read the whole thing to make sure they're not repeating a point already made (and potentially addressed/resolved), or overlooking important points that should be taken into account. Voting would also make it easier to ignore low-vote posts if one's short on time, and allowing the best posts to be highlighted would even allow some form of summarizing to the thread.
 * Automatic signatures. Not only the tildes markup is unintuitive, it's also very easy to forget to sign, even for experienced editors.

Joseph Fox
I would say the main issues are that it is too difficult to understand the "indenting" feature we use to thread conversations - I can't imagine a new user just guessing that that's what the colons do; the idea of four tildes to sign a comment seems a little strange given that on most sites with discussion features (Facebook, Twitter, Disqus) the user's handle is inserted by default; and it is much too easy to accidentally distort the fabric of what someone else has already said, and then of course be vilified for refactoring comments.

Perhaps I'm well off the mark given I've used Wikipedia and its talkpage format for almost seven years(!).

Dan Leveille
(Editor since 2006)

Discussion on Wikipedia is very unintuitive and has been frustrating to me. Glad to see that it's getting improvements!


 * It's a wiki rather than a discussion system. Wiki markup is confusing, especially for new users. As someone with HTML and programming knowledge, I always ask myself, how would non-programmers understand this?
 * Formatting. Some people indent to start discussions, some people do not indent properly, and sometimes it's hard to follow the conversations. Sometimes people will add their comments out of order.
 * It's unclear how to start a new discussion. For new users, it's unclear how to start a new discussion topic. Sometimes people will just add it to the top or bottom of the page without a heading.
 * Archiving discussion. Discussions get moved to archive pages, but this system is less than ideal and isn't entirely intuitive.
 * It's labeled "Talk". This isn't all that intuitive, and could imply that it's a discussion forum about the topic rather than the actual article.
 * People use it as a discussion forum about the topic. (WP:NOTFORUM)
 * You have to sign your comments. You have to type " ~ " to sign a comment. How would new users know this? I always used to forget. Unsigned comments sometimes get signed by a bot, which is confusing to new users and adds extra clutter.
 * There are no profile photos. Discussion is different when you're talking to a person rather than a username. If you see someone's face when you're talking to them, you will act and talk differently. Less anonymity makes people think more about what they're saying and would add more of a community and personal feel to Wikipedia.
 * Talk Page Notices add clutter. There's sometimes a ton of notices at the top of the Talk Page. It gets confusing and cluttered, and pushes discussions way down off the page. Not only that, but they have different looks and design styles, and it looks really messy.
 * Notifications. You don't get notificiations when your comment or discussion gets replied to, and you don't get notifications when new threads are started (unless you're watching the Talk page and happen to see that it was updated).
 * Subscribing to a thread. There's no way to "subscribe" to a specific conversation on a Talk Page. It would be awesome to see the number of subscribers to a conversation as well, to show how "hot" or "important" the discussion might be.
 * Upvoting. It would be great if people could upvote comments to show their support for points others make.
 * Proposing article merges or deletion gets confusing. I have to research how to do it, then place these templates on both articles and both Talk pages, and then choose one Talk page to discuss it on, then point to a Talk section about the merge? You definitely need to have friction, so that not everyone will do it, but right now, it's not the right type of friction. It just feels like a mess and overly complicated. I always feel like I might not be doing it right.

Erlan Vega
(Wikipedia editor since 2005 on eswiki)

I found that using the discussion is easy and intuitive. However, not many people do so. About user talk pages for example. Some pro users started answering posts on their talk pages rendering the notification system mute. To solve that, a bot runs or the user places a template saying that they will not answer in your talk page. Wikipedia is the only place where I know conversations are split in two pages, but, extrangely, it has worked for the most of us. I should say, though, that good people didn't stay because that was confusing and told me they expected something like facebook message system.

As for the talk pages for articles, if they ever get messages, few are answered and you don't get that nice orange notification you get when someone writes on your talk page. You somehow have to figure out that you have to place that page on your watchlist and then read it from time to time to see if someone, by chance, answered to your comment.

What's most frustrating I think is that some power users blame newcomers for not using a feature that isn't obvious to everyone.

Mark Savchuck
I had no idea it existed... And I use Wikipedia every day. So yeah...

I guess it doesn't stand out enough.

Also - maybe you can run the same thing you do when you fundraising? - Make a bright box on top that has a video explaining the feature and how to use it. Run it for a month (most of the active users will see it by then) and we will start using it more.

Thank you for the feature (because when I read Wikipedia, I usually have questions, and there is no way I can ask someone....).

Fabien Benetou
It is "too free".

People are used, by order of popularity, to
 * 1) Face-to-face dialogue
 * 2) Phone dialogue
 * 3) Linear forums

Basically anything slightly out of those formats are very hard to use because we lack familiarity.

Wikis are amazing because they provide an incredible freedom to focus on the content first.

Unfortunately discussions are as much about the content as about the social aspect.

I suppose then that successful platforms for discussions online lower cost to track "who said what and in what context".

For that I suggest following most guidelines from forums, but adapted to efficient Wikipedia editions


 * Making signing, ~, automatic
 * Facilitate quotation to the specific problematic part of a text (wiki page or discussion part)
 * "Suggested edits" a la Quora

Past research on user talk






"Ignored" new users
Since 2006, new users who are "ignored" (e.g., do not receive any talk page messages) are more likely to keep editing than users who do receive a talk page message. This suggests that the current way we communicate to new users is doing more harm than good.

However, for new users whose edits are reverted, not sending a warning is much more likely to make the user go away forever than sending a template warning. 41% of new users will continue editing after being sent a template warning for a test edit, versus 9% who will continue editing if reverted and sent no warning.


 * Documentation (1, 2)

First messages to new users
Since 2006, English Wikipedia has seen an explosion of semi-automated tools and bots that send welcome and warning messages to new users. By 2011, 80% of first messages to new users were templates sent by a tool or a bot, and 65% were warning templates.


 * Documentation

The impact of user talk templates on new users
About 50% of registered and unregistered users check their talk pages within 24 hours of receiving a message. The majority of users whose edits are reverted and who receive a warning never return to edit again. Changing the tone of template warnings has a small but significant effect on short-term retention, but not long-term retention.

Documentation

New user help requests on user talk
Based on a sample of new user edits from 2011, the majority of new editors ask for help on user talk pages – their own or another editor's.


 * Documentation