Wikimedia Research/VE on mobile

Guerrilla Testing for iOS mobile (iPad)
In order to get some quick feedback on the edit work flow on iPad, we decided to go out into the city and find a few people to test with.

The goal of this research was to observe people: To get started, we asked people a few basic questions first:
 * 1) Making a simple text edit
 * 2) If they can make some text bold,
 * 3) If they can switch from visual editor to wiki text editor
 * 1) Do you use an Android or Apple products usually?
 * 2) Do you ever use Wikipedia? (Goal of this question is to find out if they know you can edit wikipedia or not. If they answer, "Yes, I use Wikipedia", we ask "how do you use it?" If they describe that they read, mostly and don't mention editing, then we ask "Do you know that you can edit Wikipedia?"

We asked them the two getting started questions about themselves, and then presented them with an ipad that had a wiki page open (on em.m.wikipedia.beta.wmflabs.org). We had already gone in and misspelled a word, and when we presented the ipad to them, pointed out the word that was misspelled, and asked them to please correct the spelling error. Then we went through the other two questions if there was time.

Where
Monday June 2, 2014: We went to Starbucks so we could use wifi there.

Monday June 9, 2014: Internet was down at Specialty's and Starbucks, so we user-tested with WMF-ers on 6th.

Thursday, June 12, 2014: We have a Mifi to provide us with connectivity, so are free to roam the wilds of Yerba Buena Park again.

June 2, 2014

 * 1) All three of the participants had difficulty understanding how to save. The icons were not communicating "save" to the participants. One person who did all three tasks, was able to save successfully the second time he had to, but the first time, it took him a while to figure it out. Hypothesis to be further tested: that the icons are not clearly communicating "save", so first time users of Visual Editor might have a hard time figuring out how to save. It is learnable, so, once they figure out how to save the first time, it will probably not be as difficult.
 * 2) After pressing the arrow (to save) and they were landed on the "tell us what you changed" page - they were not sure what to do next. Two people did not seem to understand that this request was to describe what they had changed, and was an intermittent step within saving. It took all three participants some thinking to understand they needed to press the green check mark to complete their save.
 * 3) Requiring two taps - removing the keyboard and then clicking the green button seems to be an issue.
 * 4) Counterintuitive, but both users I (Daisy) took notes on tried to click the X (cancel) to save the edit.

June 9, 2014

 * 1) Hesitation on the part of the users continues regarding the general 'save edit' process - secondary save step is still mildly confusing, whether it is because of hesitancy in general or a hope on the part of users that the tester will confirm their actions (and their being able to avoid reading the content of the secondary save page)
 * 2) Some hesitation on clicking the green save
 * 3) Confusion about what the citation icon represents
 * 4) Users generally apprehensive about clicking/testing out the X cancel button

June 2, 2014

 * 1) Requiring two taps - removing the keyboard and then clicking the green button seems to be an issue - not sure if this is just an iPad thing or a bug we can fix
 * 2) Make the edit 'menu bar' clearer. (More) shaded background behind the icons, like for the browser Visual Editor, so users are more aware they're still in 'edit' mode
 * 3) Clearer border, so the green next button is more 'in the picture'
 * 4) Has been considered previously, but using text on the buttons vs symbols may be preferable

June 9, 2014

 * 1) Requiring two taps issue/bug remains
 * 2) Citation button dropdown menu doesn't go away upon tapping out of the menu. Will stay there unless user taps the citation button itself again
 * 3) When you're in edit mode and switch editors, there should be a 'save' dialog, but didn't happen in this beta

Man (20-30s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * He usually uses Android, uses wikipedia to read, but not to edit. He did know that it is possible to edit.
 * He went right to the edit button -
 * When he made the edit, he knew how to get rid of the keyboard. Once he was done making the edit, and had not pressed the arrow to save, he asked "is this final? I see the edit is made, but I don't know if it is committed." I took him time to figure out to press the arrow to save. He said it would have been easier if he had seen the word "save" somewhere.
 * Once he pressed the arrow, and was on the "tell us what you changed" page, he said "they want me to repeat what I am changing", and typed in the change he made. It took him a little time to find the check box that makes the edit complete.
 * Was immediately successful with this task.
 * This time, he knew how to save with no hesitation.
 * He said he would not edit that way, but after a few more times asking him to pretend he wanted to edit in the code, he looked around and found the switch pretty easily.

Limayli Huguet
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * iPhone user. Uses wikipedia (reads, doesn't edit)
 * Found edit button relatively quickly. couldn't tell if button worked at first, pushed 'bold' then typed
 * Unsure what to do at save page, clicked the check box
 * n/a
 * n/a

Amy Vossbrinck
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * Uses both Android and iPhone
 * Uses Wikipedia - reads and does a bit of editing
 * Noticed that the edit button changed/looks different
 * Couldn't find the save button; would want the button to say 'save'
 * Knew that the check button confirmed the save
 * n/a
 * Couldn't find source editing
 * Confused regarding visual vs source editing

Man (30s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * He has an iPhone
 * Rarely uses wikipedia. when he does, he usually scrolls to the bottom for the reference links as springboards. Doesn't edit, and didn't seem to realize he could.
 * Scrolled to the bottom of the page first, aligned to his description of his typical wiki use
 * Clicked top left edit button, chose visual editor - didn't know what he was choosing, and thought he could view what was edited/to be edited here
 * "How do I finalize?" - scrolled up/down and ultimately clicked the X (cancel) 3 separate times
 * Remarked that he had to invest a lot of time to edit this one word. Hit return on the keyboard, didn't finalize it either. Touched the green 'next' arrow, but that only got rid of the keyboard the first time he tried it. didn't attempt to press again. Refreshed the page, didn't work either
 * After prompting, got to the secondary comment page. Said he would read the Terms of Use, and clicked the check button to finalize the edit
 * didn't attempt
 * didn't attempt

Man (30-40s)
Getting Started Questions Correct Spelling Task Make something Bold Switch from Visual Editor to Wiki Text Editor
 * Had two Android phones on hand
 * Uses wikipedia, usually gets there via web searches
 * 'Didn't know I could' re editing
 * Clicked on the correct edit button right away. seemed to realize you had to tap once to get rid of the keyboard, but then had trouble finalizing the edit. Scrolled down, up. Tried the X (cancel) button, then tried the green next button
 * Clicked on the check button on the secondary save page - commented "pretty easy", "pretty simple".
 * didn't attempt
 * didn't attempt

Dennis Porter
Getting Started Questions Correct Spelling Task / Make something Bold Switch from Visual Editor to Wiki Text Editor Exploration
 * Android user
 * Mostly reads Wikipedia, has edited a couple times via edit source only, no VE
 * Immediately goes to top right edit button
 * Taps word, uses pop-up spell check hover functionality to edit the word
 * Uses bold button, no issue
 * Upon prompting regarding finalizing the edit, immediately taps the blue next button. Hesitation upon seeing secondary save screen - 'should I have done that?'
 * No hesitation on finalizing with green check icon
 * Part of the exploration below, not done as an official task. Clicking the edit button on the menubar was prompted, but no issue realizing what source editor was. Bug regarding switch toggle: should take to save dialog, but doesn't in this beta
 * X - After going back into VE edit mode, exploring buttons: 'I think that would delete the article. But I haven't done anything on the page yet'
 * Link icon - 'Link'
 * Citation icon - 'Is that a send button?'

Dan DeJarnatt
Getting Started Questions Correct Spelling Task / Make something Bold Switch from Visual Editor to Wiki Text Editor Exploration
 * iPhone user, never used iPads before
 * Mostly reads Wikipedia, hasn't edited. Only familiar with concept of VE, hasn't used.
 * Immediately goes to top right edit button
 * Changed a word, no problem. Highlighted a word and used the pop-up function to make it bold, instead of the button in menubar
 * 'How do I save? I don't know'
 * Clicks the blue next arrow - 'Is that saving?'
 * Part of the exploration below, not done as an official task. Clicking the edit button on the menubar was prompted. Confused about why there was another edit button - 'Is this maybe to draw?'
 * X - cancel/delete, general apprehension about clicking this. edit mode not very ingrained here
 * Link icon - not sure what this was
 * Citation icon - thought it was for bookmarking, also possibly how to save
 * Katie had him try to link word snow in the paragraph to a 'snow' wiki page. Very confusing. Green check arrow already next to 'snow' in options list, confused me (Daisy) too. Not very intuitive. No way to 'save' the change. Essentially navigated via link button, navigated back, and then 'snow' was a red link. Very confusing

July 30, 2014 test
In response to previous tests, we iterated on these features
 * Changed X icon to back arrow, so it doesn't appear destructive
 * Changed arrow icon to word "next", and check icon to word "save" to provide clearer progressive actions
 * Changed save page to have better hierarchy and wording
 * Changed switch mode icon to a settings gear icon, and moved to right side
 * Introduced context menu for existing references and links

Findings
We tested with 4 people using same questions as before. (With addition of asking them how they would edit a link/reference)

Participants:
 * Used back button and it did what they expected
 * Hesitated when asked to save but all were able to find "Next" button
 * Filled out the save screen appropriately, although 1 person said it looked like an error screen at first
 * When asked to switch to wikitext, tapped gear icon almost immediately, but several people still struggled with "edit" and "edit source" language. Everyone also struggled with the pop-up asking them to save before switching. Might be buggy?
 * Everyone struggled with the context menus. Most did not notice that the icon in the toolbar was highlighted. Nobody even noticed the context bars, and didn't know what they meant when I pointed to them.

User 1
Uses ios, not a frequent editor
 * found the edit pencil easily
 * paused to copy content
 * couldn't recognize link inspector icons
 * would just start pressing n looking for controls
 * finding wikisource was easy

User 2
Has used VE before - thinks is prevents you from messing stuff up. Prefers editing in VE vs wikitext
 * found the edit pencil easily
 * knew native controls to select and bold text
 * link inspector visual cues are subtle
 * understood the word may have already been linked
 * said the wheel settings may have a switcher between wikitext and wikisource
 * was looking for something to change defaults
 * found wikisource the second time but said the language wasn't clear

User 3
Has used ipads before
 * found the edit pencil easily
 * was able to select and italicize easily
 * didn't notice the link icon highlight or the link drawer but understood that it meant that the word had been linked
 * successfully saved an edit
 * easily switched between wikitext and wikisource

User 4
Has used ve on desktop before
 * uses ios and android, but uses IOS more
 * finds pencil easily
 * used the native IOS dialogue for bold/itlai controls instead of the top one
 * says she wouldnt look at the top toolbar
 * didn't know what to expect once she tapped on a link
 * found next as an intuitive step to save
 * found edit source in the gear menu to switch between wikitext and ve