VisualEditor/Design/User testing

May 20, 2014 general user tests
We surveyed 5 people with usertesting.com. Each participant was introduced into an example article in VisualEditor mode and asked to complete a few tasks, typical to basic editing: In general, users were able to quickly find what they needed on the toolbar and complete the tasks. But often they were not sure if they had completed the task successfully.
 * Enter text
 * Add a special character to text using toolbar
 * Create a link
 * Make text italic
 * Enter title, make it a subheading
 * Enter text, make it a paragraph under the subheading
 * Add a picture
 * Save their edits

Finding things
Users generally found things either right away or by looking through toolbar menus from left to right.

Sometimes when users couldn't find something looked to the left sidebar under tools.

Recommendations:
 * Less clutter:
 * ✅ Move bullet lists into their own menu
 * Consider moving bullet items elsewhere – possibly the text formatting menu.
 * Try moving the section menu into the text formatting menu.
 * Run a card sorting test to see if users expect different ways of organizing the toolbar.
 * Adopt a principle of having the most commonly used tools to the left of the toolbar.
 * Re-prioritize some of the items in the menus.
 * Break up into groups.
 * Special Character: widen the dialog to make it easier to scan through, and break characters up into smaller groups

Link
Most users were not sure they successfully added a link.

Recommendations:
 * Add a check button or something to show a user confirmation.
 * Add a way for user to go open the link in another tab.

Picture
Some users struggled to add a picture. 2 users tried to go to insert > gallery, and got really lost. Once a user had added a picture, they weren't sure that they had successfully added it to the correct place.

Recommendations:
 * ✅ Remove "gallery" from the insert menu because it's confusing for most users and very rarely something they want.
 * Move media icon to top-level toolbar.
 * Change dialogue title to "Add images or video"
 * Be able to add gallery from this dialogue?
 * Improve drag and drop, and indicate somehow that images fall where they can fit on right rail

Save
Most users saved easily and felt they had done it right. One user wanted to fill out a detailed edit summary and struggled to remember what she had changed.

Recommendations:
 * Move save dialogue to be anchored to the top right, closer to save button, so that user can more easily see their changes.
 * Have summary filled automatically

Edit source vs Edit beta
One user upon entering test immediately clicked edit source and never found VisualEditor.

Recommendations:
 * One tab, with ability to switch in the toolbar.
 * Tooltips – Give more information in link and reference tooltips so a user can quickly read / decide if they want to edit them.

Next test ideas

 * Watch more experienced editors use VE.
 * Do a comparison with 1st group - Wikitext users, 2nd group - VE users