Wikimedia Research/Discovery and usability for brand new and casual editors

Introduction
Visual Editor is an editing tool under development by the Wikimedia Foundation, which aims to provide a better editing experience for new and casual editors Wikipedia and Wikimedia sites. This page is about user experience research that is being done to contribute to the development of Visual Editor.

Goals of this research
In this research, we are seeking a better understanding of the very first time a person tries to edit a wikipedia article, and the experience people have as they are learning to contribute to Wikipedia. We scheduled 16, hour long research sessions with 8 brand new editors (people who have some motivation to edit, but never have tried before) and 8 casual editors (who have some motivation to edit and have done a maximum of 10 contributions to wikipedia) for this research. The sessions are set up in sections:
 * About 10 minutes is learning about the participant and their relationship with Wikipedia.
 * about 15 - 20 minutes is the participant doing several tasks (related to the MVP of visual editor) in either the wikitext editor or visual editor)
 * about 15 - 20 minutes is the participant doing the same tasks as in the previous section, except in the editor they did not use in the previous section.
 * In the remaining time, the participant and researchers review the participant's experience in the two editors and also review a first time editing tutorial for feedback.

Findings
Please see this document for our findings. In this document we focused on the kinds of improvements the visual editor could use to make the experience for new editors more welcoming, easy to understand and usable.

There is a lot of data in these interviews and we may look at it again to gather more insights.

Note: This research was done on a beta site, and the UI for Visual editor and sometimes for wikitext editor were not always stable. Because of this we did focused usability testing on the basic editing tasks for the visual editor in a controlled environment. For the research described on this page, we were still able to learn a lot about people's experience in learning to edit Wikipedia.

Research Session Debriefs
Brand new editors:

N1

Session Video


 * uses Google to access Wikipedia pages
 * tries to click the page to begin editing instead of clicking the edit button first in WTE
 * wonders what 'edit source' meant, and wasn't sure what the difference was between edit and edit source
 * remarks that after in edit source page, that it would be difficult to find the specific 'smell' section
 * has difficulty finding the icon to bold text in WTE
 * mimics the code from an earlier part of the page to add the tail subsection
 * goes to help to figure out how to add a citation
 * clicks on the help link directly, and doesn't go to the help page the first time because she didn't want to lose her changes
 * confusion among references, citations, links, source terminology
 * when adding a citation, there is confusion between the text 'edit source' for editing source code and the concept of adding/editing an actual source (citation/reference)
 * uses help to figure out how to make an internal wikipedia link
 * initially confused about preview changes in VE, general contextual confusion between the two editors
 * doesn't make the conceptual link that making text bold is different from making text bold by selecting a heading level for it, wasn't able to create the subsection

N2

Session Video


 * tries to edit directly without pressing edit first ("why can't I copy-paste?") in VE
 * thinks that the add comment box in VE after pressing save is where she needs to type in the added text again
 * when adding the subsection in VE, only makes the header bold and does not make the headings connection
 * confusion about what source means in 'edit source' in WTE. was looking for the plain 'edit' button
 * not confident about being in WTE (at 38:00)
 * clicks see preview in WTE and then behaves as if she were in the article page itself/no longer in edit mode

N6

Session Video


 * uses Wikipedia for school
 * didn't realize she was in edit mode on VE when she pressed edit
 * confuses the concept of source and citation for the add citation task in VE; she says she isn't sure if she should click edit or edit source for this task (at 10:10)
 * does not realize link menu already found the internal link page for her in VE, and created an external link instead. realizes she didn't make an internal link, and tries again with Linnaeus, but makes a red-link through link option because she doesn't realize that she needs to search Carl Linnaeus within the link menu
 * WTE looks like plain text, looks code-y
 * in WTE, copies syntax for section headings and creates tail subsection correctly
 * thinks of figuring this out in WTE as a game; copy-pastes ref tags to make a citation footnote correctly. remarks that the additional fields in other citation examples, like citeweb and author, look confusing
 * dismisses VE tutorial. she likes to hack things on her own
 * preferred VE first because she sees it first, but ends up working through WTE and likes it more; more linear, more things can be done in one go
 * VE is more mouse/scrolling-heavy, does not like this as much
 * likes help dropdown. noted that others who aren't as computer-savvy might like using VE more
 * overall, didn't realize it was this easy to edit

N8

Session Video (requested to only share with WMF)


 * said wikitext looks intimidating
 * used VE first, and found WTE to respond more quickly
 * asked if there was spell-check in WTE
 * says she would copy-paste syntax to mirror how things look, but does not always manage to do this correctly in practice
 * found adding a reference in WTE to be much quicker, but does it in both
 * did not correctly add an internal link in WTE
 * has difficulty and was confused with the section heading task in VE, but was able to do it and confirm the subheading in WTE
 * didn't understand that she had to click the heading level dropdown in the menu; she simply made the Tail text bold and seemed to think that was sufficient
 * wanted to add tail to category
 * thinks the tutorial/walkthrough would've helped her to understand the purposes of the icons
 * liked 'the code' (WTE) better even though it freaked her out at first; found tasks easier to do once she figured out how it worked
 * readability is better in VE

N13

Session Video
 * has never edited before, but is an engineer and is technically savvy. said that once he learned the syntax WTE editing would be easier
 * uses wiki for work, look up terms, and for general knowledge
 * tried editing before, but it looked intimidating and backed out
 * hesitation to edit also has to do with his uncertainty about the rules around editing and his own expertise on topics
 * very concerned about validating all of his edits before finalizing/saving because he wanted to make sure he wasn't putting incorrect information or incorrectly formatted information on wiki
 * was okay with both editors, but preferred WTE more; described not liking gui because he can't see what's going on in the background
 * liked the WTE preview more because he could see the page as it would look, how it would be rendered
 * was confused by VE preview
 * liked that he could see the changes made in VE preview/show changes in WTE, but still preferred 'show preview' in WTE
 * stumped that he could not preview the link in a citation in both VE and WTE; in VE, because it was not yet created (he had not saved), and in WTE, because he couldn't click on the [#] and confirm the reference link prior to saving. realized later that it was because the reference had not yet been saved/finalized
 * generally self-reported high confidence levels (4-5) throughout the tasks
 * clicked through the tutorial, but said normally he would try to find a way to exit it as soon as possible
 * upon moderator's follow-up, mentioned that he didn't read the tutorial pop-ups at all even though he'd clicked through them
 * would rather have an editing 'cheat-sheet' he could refer to as opposed to this type of tutorial, because he said it would just go away and he couldn't refer back to a specific step in it later
 * was able to complete all link-related tasks on both VE and WTE with little confusion

N15

Session Video (requested to only share with WMF)
 * He uses Wikipedia when looking for quick answers, and sometimes to learn more about a subject. If he is looking deeper, he will read entire article. example learning about Python.
 * He skips over blue links while reading.
 * He uses blue links more often on shorter articles, because he is more curious about authenticity on shorter articles
 * In the Python article, there are122 ref so he feels pretty confident that its legit. but maybe if its longer ppl think its done and don't check as much, he wonders.
 * He has tried to edit once and backed out because of wikisyntax. A while ago, he wanted to blog but decided to try to contribute to Wikipedia so it would be contributing to knowledge (more people would be able to use his knowledge) but it was overwhelming getting started. He saw wikitext and decided not to contribute content to Wikipedia because if is a whole new way of writing. "do i really want to be learning another markup language that I wouldn't use elsewhere? no idea how long it would take to learn...so I decided to blog instead."
 * In VE it took him a while to figure out how to add a section (he did a bunch of experimentation), and knew about hierarchies in creating page because of previous work in other software, but he said people without his experience might only think to make the text bold instead of highlighting the section title and making it a "sub-heading".
 * He says "Edit source" has bigger learning curve. Other editor (ve) is what you expect from most systems these days. Wordpress, squarespace... pretty easy to use."
 * He liked seeing his changes in real time in ve, and noted that he spent time going back and forth to the preview in "edit source".
 * insert paragraph - why do they appear some places not others. He said he would just start typing, and doesn't want the "insert paragraph" Would rather have it for insert section but not sure how that would nest.
 * He didn't know how to leave ve when he pressed "edit" and then decided to not edit. "No cancel button".

N17

Session Video


 * tried editing back in the day, years ago, and feels that syntax editing was hard to grasp; felt it was an overall bad experience because his edit was reverted
 * has difficulty finding the correct place to land while editing; often clicked edit for the whole page
 * replicates syntax for creating tail subsection/heading in WTE
 * uses preview to double-check his edits before saving
 * uses reference button in menu successfully to add a citation in WTE
 * knows to look at other examples of syntax to complete tasks, but when he copy-pastes syntax to replicate what he needs to do for a specific task, he sometimes gets the specifics of the execution not quite right, or entirely wrong
 * says upon first seeing VE that it is more like the other editing tools he's seen, mentions similarity to MSWord
 * though he first makes the tail subheading bold, he knows to compare it with a similar subheading (coat) to make sure they are the same style/header level
 * he gets into edit mode in VE for the citation task, and doesn't know how to get out of it
 * he has conceptual confusion between link/citation and how to create either in context; tries link button twice, ultimately also tries cite button and is successful adding the reference
 * has trouble figuring out how the link button generates link; might not be clear to him that link button has a search associated to it
 * has trouble understanding the workflow for linking
 * didn't find the tutorial prototype for VE annoying, mentions that each step in the tutorial could have links to more details
 * likes VE, feels that Wikipedia needs this kind of editor
 * reiterates his confusion around links/citations/external links

N18

Session Video (requested to only share with WMF) WTE VE (to be finished at later date)
 * visits both fr and en wikis. uses Wikipedia mostly to read about physics and history
 * has not edited but has tried, was confused by the syntax. he also does not feel like he's enough of an expert to add to Wikipedia
 * he would like the show preview to highlight the changes he made
 * he thinks that his tail subsection won't show up in results because his comparison of his subsection with the coat subsection is different (the main page line under the subsection doesn't exist)
 * exhibits contextual confusion about the concept of link/citation/ref/etc., and also has confusion about how to add these
 * finds the cite dropdown (this does not seem to be present
 * previews for adding citations aren't 'live' previews - footnote number(s) aren't updated, footnote itself cannot be viewed
 * used citation dropdown template in WTE; because he didn't add all the fields in the template, he gets red 'missing information' text in his citation/footnote
 * no indication in the template of what fields are required to be 'complete' and avoid red 'missing information' text
 * likes WYSIWYG, no preview, it's all right there
 * knows to check for subheading consistency while making tail subsection
 * citation icon looks like a bookmark, hovers over for the helptext

Casual Editors

C2

Session Video (requested to only share with WMF)
 * he edits more frequently than he self-reported (1-2 edits/day)
 * he is a creature of habit and might stick with WTE because of this
 * says VE is easy to use, he could see himself using it
 * says he might use VE if it is available all the time
 * says 'edit source' is not the name of the editor he has seen before, and says experienced editors will be confused
 * in VE, it is difficult to know how to make the correct type of heading
 * says it is sometimes hard for him to remember he is editing while in VE, because he is used to WTE
 * he forgets to save several times

C4

Session Video (requested to only share with WMF)
 * edits by section because seeing the whole page in WTE is confusing
 * does not like WYSIWYG editors because he is an engineer and doesn't know what the editor is doing in the background (eg causing errors somewhere?)
 * couldn't figure out a way to confirm section level/headings, but not a problem doing so in WTE
 * confused by VE citation dialogue. did trial and error and accidentally figured it out, but said he did not like this - confusing, and it worked when he didn't expect it to
 * link icon did not resonate with him
 * doesn't always understand what the icons mean, so for him, hover text help is very useful
 * said he would skip VE tutorial because he is confident with editing
 * tutorial doesn't describe enough about editing, especially about editing in more than one language)
 * he uses enwiki a lot because most topics he is interested in are in english
 * he likes being able to see the edit in WYSIWYG because he doesn't have to click preview and scroll a lot and go back and forth as much

C5

Session Video (requested to only share with WMF) C7
 * very comfortable/familiar with WTE, so defaulted to using it
 * said he would use VE for simple text edits
 * he knows how to discover/learn how to do things in WTE
 * figured out how to add an internal link in WTE by looking up the syntax example in another section of the article
 * created an internal link in VE easily, and said he liked it better than in WTE
 * it was confusing for him to add an external link in VE; liked the process more in WTE because he could see and use the syntax in the bullets above
 * said the VE tutorial would have been useful for him as a brand new user to better understand where tools are

Session Video
 * Some thinking about the "edit" vs "edit source" tab or link to begin editing. After looking at both he says“edit source allows me to make more sophisticated changes, like chaining the font, no wait, I can do the same things in edit, its a bit like MS word”
 * To initiate editing, he clicked into the article page without pressing "edit" tab/ link expecting to be able to start editing like a word doc.
 * Looks in the "insert" menu first when he is looking to add a wikilink.
 * Said he would use “simple edit” (ve) over the “edit source” it’s easy, in exchange I get more options. He would try to do everything in edit and if he gets stuck would then go to edit source.
 * He prints out articles routinely to take them with him on a trip or use it otherwise out in the world.
 * "I’ve edited maybe 3 times, only when I have an personal interest in the subject, [shows article about his student fraternity] looks to see if their has been vandalism, has changed (reverted) malicious edits to pages about the page. "There is a body of people who decides what is right and what is correct?"
 * He makes "google sites" for people who want websites and referred to the google UI a few times in reference to the ve UI.
 * He was a developer.. maybe retired now.



Findings
When the research is done, and we have completed analysis and synthesis, the findings will be reported here on this page. Also, the links to recordings of the research sessions will be posted here for those participants who have given permission to share publicly.