VisualEditor/Design/User testing

See also: Mobile VE user testing

May 20, 2014 general user tests
We surveyed 5 people with usertesting.com. Each participant was introduced into an example article in VisualEditor mode and asked to complete a few tasks, typical to basic editing: In general, users were able to quickly find what they needed on the toolbar and complete the tasks. But often they were not sure if they had completed the task successfully.
 * Enter text
 * Add a special character to text using toolbar
 * Create a link
 * Make text italic
 * Enter title, make it a subheading
 * Enter text, make it a paragraph under the subheading
 * Add a picture
 * Save their edits

Finding things
Users generally found things either right away or by looking through toolbar menus from left to right.

Sometimes when users couldn't find something looked to the left sidebar under tools.

Recommendations:
 * Less clutter:
 * ✅ Move bullet lists into their own menu
 * Consider moving bullet items elsewhere – possibly the text formatting menu.
 * Try moving the section menu into the text formatting menu.
 * Run a card sorting test to see if users expect different ways of organizing the toolbar.
 * Adopt a principle of having the most commonly used tools to the left of the toolbar.
 * Re-prioritize some of the items in the menus.
 * Break up into groups.
 * Special Character: widen the dialog to make it easier to scan through, and break characters up into smaller groups

Link
Most users were not sure they successfully added a link.

Recommendations:
 * Add a check button or something to show a user confirmation.
 * Add a way for user to go open the link in another tab.

Picture
Some users struggled to add a picture. 2 users tried to go to insert > gallery, and got really lost. Once a user had added a picture, they weren't sure that they had successfully added it to the correct place.

Recommendations:
 * ✅ Remove "gallery" from the insert menu because it's confusing for most users and very rarely something they want.
 * Move media icon to top-level toolbar.
 * Change dialogue title to "Add images or video"
 * Be able to add gallery from this dialogue?
 * Improve drag and drop, and indicate somehow that images fall where they can fit on right rail

Save
Most users saved easily and felt they had done it right. One user wanted to fill out a detailed edit summary and struggled to remember what she had changed.

Recommendations:
 * Move save dialogue to be anchored to the top right, closer to save button, so that user can more easily see their changes.
 * Have summary filled automatically

Edit source vs Edit beta
One user upon entering test immediately clicked edit source and never found VisualEditor.

Recommendations:
 * One tab, with ability to switch in the toolbar.
 * Tooltips – Give more information in link and reference tooltips so a user can quickly read / decide if they want to edit them.

Next test ideas

 * Watch more experienced editors use VE.
 * Do a comparison with 1st group - Wikitext users, 2nd group - VE users

July 31, 2014 General User tests
We again surveyed 5 people with usertesting.com. The tasks:
 * Enter text
 * Add a special character using toolbar
 * Create a link
 * Add a citation
 * Edit an existing citation
 * Add an image
 * Save their changes

Overall
Testers completed most tasks easily. 2 users were confused while creating a link – they weren't sure that they were successful. Almost all users struggled to find an existing citation to edit it.

Creating a link
Recommendations: 
 * "I was expecting something like a page. I guess this is already a link? But I'll just click Done"
 * "I don't know wether this is functional link because it doesn't have a page connect, it needs to be connected to some other page, but it seems for the sake of the test to have worked, it has this line on hover..."

Give users more information about the page they will be linking to.
 * Include the image thumbnail
 * Include first sentence of article
 * Allow for hovercards, which shows both image and text, but harder to discover
 * Easier way to visit the page

Finding an existing citation to edit
The test question read: "At the end of the first paragraph, you find a citation that contains a typo. Edit the citation to fix the misspelling of "British"." Which is probably not the best way to test this, a more realistic workflow is that a user would notice a typo or whatever in a citation while in read mode, then enter edit mode, so they would know where it is located. Even so, this test shows that its difficult to find citations in while in edit mode.

Users scrolled back and forth from references at bottom to citations in article, tried to do Find on this page, and were confused by citation tooltips. (Maybe because test didn't ask them to find a website citation.)

Recommendations: 

Include the entire reference text in the tooltip.