Wikimedia Research/VE on mobile

Guerilla Testing for iOS App
In order to get some quick feedback on the edit work flow on iPad, we decided to go out into the city and find a few people to test with.

The goal of this research was to observe people: To get started, we asked people a few basic questions first:
 * 1) Making a simple text edit
 * 2) If they can make some text bold,
 * 3) If they can switch from visual editor to wiki text editor
 * 1) Do you use an Android or Apple products usually?
 * 2) Do you ever use Wikipedia? (Goal of this question is to find out if they know you can edit wikipedia or not. If they anser,

"Yes, I use Wikipedia", we ask "how do you use it?" If they describe that they read, mostly and don't mention editing, then we ask "Do you know that you can edit Wikipedia?" We spoke with 3 participants in this round. We asked them the two getting started questions about themselves, and then presented them with an ipad that had a wiki page open (on em.m.wikipedia.beta.wmflabs.org). We had already gone in and misspelled a word, and when we presented the ipad to them, pointed out the word that was misspelled, and asked them to please correct the spelling error. Then we went through the other two questions if there was time.

Where
This time, we went to Starbucks so we could use wifi there.

Findings: Patterns observed and any hypothesis to be further tested
1. All three of the participants had difficulty understanding how to save. The icons were not communicating "save" to the participants. One person who did all three tasks, was able to save successfully the second time he had to, but the first time, it took him a while to figure it out. Hypothesis to be further tested: that the icons are not clearly communicating "save", so first time users of Visual Editor might have a hard time figuring out how to save. It is learnable, so, once they figure out how to save the first time, it will probably not be as difficult.

2. After pressing the arrow (to save) and they were landed on the "tell us what you changed" page - they were not sure what to do next. Two people did not seem to understand that this request was to describe what they had changed, and was an intermittent step within saving. It took all three participants some thinking to understand they needed to press the green check mark to complete their save.

Man (20-30s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * He usually uses Android, uses wikipedia to read, but not to edit. He did know that it is possible to edit.
 * He went right to the edit button -
 * When he made the edit, he knew how to get rid of the keyboard. Once he was done making the edit, and had not pressed the arrow to save, he asked "is this final? I see the edit is made, but I don't know if it is committed." I took him time to figure out to press the arrow to save. He said it would have been easier if he had seen the word "save" somewhere.
 * Once he pressed the arrow, and was on the "tell us what you changed" page, he said "they want me to repeat what I am changing", and typed in the change he made. It took him a little time to find the check box that makes the edit complete.
 * Was immediately successful with this task.
 * This time, he knew how to save with no hesitation.
 * He said he would not edit that way, but after a few more times asking him to pretend he wanted to edit in the code, he looked around and found the switch pretty easily.

Man (30s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * He has an iPhone
 * Rarely uses wikipedia. when he does, he usually scrolls to the bottom for the reference links as springboards. Doesn't edit, and didn't seem to realize he could.
 * Scrolled to the bottom of the page first, aligned to his description of his typical wiki use
 * Clicked top left edit button, chose visual editor - didn't know what he was choosing, and thought he could view what was edited/to be edited here
 * "How do I finalize?" - scrolled up/down and ultimately clicked the X (cancel) 3 separate times
 * Remarked that he had to invest a lot of time to edit this one word. Hit return on the keyboard, didn't finalize it either. Touched the green 'next' arrow, but that only got rid of the keyboard the first time he tried it. didn't attempt to press again. Refreshed the page, didn't work either
 * After prompting, got to the secondary comment page. Said he would read the Terms of Use, and clicked the check button to finalize the edit
 * didn't attempt
 * didn't attempt

Man (30-40s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * Had two Android phones on hand
 * Uses wikipedia, usually gets there via web searches
 * 'Didn't know I could' re editing
 * Clicked on the correct edit button right away. seemed to realize you had to tap once to get rid of the keyboard, but then had trouble finalizing the edit. Scrolled down, up. Tried the X (cancel) button, then tried the green next button
 * Clicked on the check button on the secondary save page - commented "pretty easy", "pretty simple".
 * didn't attempt
 * didn't attempt

Suggestions

 * Make the edit 'menu bar' clearer. (More) shaded background behind the icons, like for the browser Visual Editor, so users are more aware they're still in 'edit' mode
 * Clearer border, so the green next button is more 'in the picture'
 * Has been considered previously, but using text on the buttons vs symbols may be preferable