Extension:WikiGrok/Version B v.2 Guerrilla Testing, December 2014

Guerrilla Testing for Wikigrok Version B v.2
The goal of this research was to observe people interact with the wikigrok version b v.2 micro-contributions beta feature and determine:
 * Goal
 * 1) do users notice the call to action
 * 2) how do users respond to the call to action
 * 3) do users understand the ‘none of these’ button?
 * 4) do users respond when the ‘none of these’ button changes to ‘next’?
 * 5) do users understand how to unselect a tag?
 * 6) did users understand the overall purpose of the feature and also the contents/purpose of the ‘tell me more’ link
 * 7) would users want to do more of these micro-contributions in a row?

To get started, we asked people a few basic questions first:
 * Questions
 * 1) What kind of phone do you use/what OS?
 * 2) Do you use a tablet, and if so, what model/OS?
 * 3) Do you use a computer, and if so, what model/OS?
 * 4) Do you ever use Wikipedia? (Goal of this question is to find out if they know anyone can edit wikipedia or not)
 * 5) If "Yes, I use Wikipedia", we ask "how do you use it?”
 * 6) If they describe that they mostly read and don't mention editing, then we ask "Do you know that you can edit Wikipedia?”

We presented users with either an iPod Touch (for iOS users) or a Nexus (for Android/other users).
 * Task

The users were shown the wmflabs beta page of the album ‘Wolf City’ to interact with the beta feature.

Friday, December 5, 2014: We went to Yerba Buena.
 * When/Where

Findings: Patterns Observed

 * 1) When asked to describe the page, users do not actively describe or mention the wikigrok feature.
 * 2) One user specifically pointed out that the language around the call to action is confusing; when clicking ‘ok’ on the feature, what exactly should users expect as a next step?
 * 3) One user mused, even while scanning the feature content, whether it was an advertisement.
 * 4) Two users did not actively try to test out the feature and only did so after prompting. Five out of the six users mentioned that they would most likely skip or not pay attention to the feature, and the same number would either not do more of these in a row or maybe would do a few depending on the situation.
 * 5) Two users did not understand the concept of unselecting tags. One user did not understand the concept of the ’none of these’ button.
 * 6) A few users (about three this round) expressed some degree of concern about the accuracy of Wikipedia as a result of possibly incorrect inputs (via normal or wikigrok editing). This has been a motif of wikigrok guerrilla testing.
 * 7) A few users expressed confusion about the tell me more link and the explanation of WikiGame. Most, if not all, did not have a clear grasp of the purpose of the feature and of WikiGame, and none of the users could give a sound explanation for the purpose of wikigrok/WikiGame when prompted. This has also been recurring throughout wikigrok guerrilla testing rounds.
 * 8) One user specifically pointed out the language, saying people who aren’t tech-savvy or well-educated may not understand something like ‘text corpus’.

Bugs and/or Suggestions

 * 1) It may be helpful to clarify and/or simplify the language used in the specific call to action, and also the language in the ‘tell me more’ section to address some users’ concerns and/or confusion.
 * 2) A user suggested the possibility of adding her own tag, or being able to edit existing tags (not sure if it is a possibility, or if it is something we would want to consider).
 * 3) (Kept this on the list from previous write-ups, as it is still relevant) Get aligned on the message we want on the end page of the tag feature. Do we want to: make the user feel great about participating, explain structured data, or do something else?
 * 4) Generally, we are seeing a lack of intrinsic motivation for these versions of wikigrok. The message, messaging, purpose and sometimes the functionality of this feature are still (at times) not very clear to our audience.

Test A

 * Male (26-35)
 * iPhone, Amazon Fire, Dell computer
 * Reads Wikipedia for more information, definitions, usually gets to wikipedia via search vs going to wikipedia site directly. knows anyone edit, but hasn't

Task:
 * User does not call out the feature, however, he later remarks that he ‘saw it, but didn’t bother to read it’
 * User very quickly answers the question
 * Understands the button/concept of ‘none of these’
 * no specific reaction when the button changes to ‘next’
 * Understands how to unselect tag
 * Comments that the tell me more link ‘says what i think it is’, that he should choose the answer that fits best after reading the article
 * Doing more of these: ‘wouldn’t mind’, ‘i guess so’, it’s ‘easy, quick'
 * This does make him second-guess the information on wikipedia, because ‘people can write anything'

Test B

 * Male (46-56)
 * Android, iPad, no computer
 * Reads Wikipedia and looks up information, did not know that anyone could edit

Task:
 * User actively notices the feature and jumps right in, though remarking that ‘i’d usually push no thanks’. glances over it, muses ‘is this an advertisement?'
 * Prompted to answer naturally, to click ‘none of these’ and also to unselect - user was not clear about any of these actions
 * User was prompted to look at the ‘tell me more’ section; guesses that it is a way to contribute to wikipedia
 * User indicates that he would most likely skip this feature if browsing on his own

Test C

 * Male (36-45)
 * iPhone, iPad and Microsoft Surface Pro, no computer
 * Reads Wikipedia, knows about editing but doesn’t. usually researches healthcare issues

Task:
 * User does not mention noticing the feature; upon prompting, responds that normally he would hit ‘no thanks’
 * User does not try to answer question, but is prompted to try. User selects ‘studio album’ tag, submits but comments that he is not necessarily adding additional information here
 * User is prompted to read ‘tell me more’ section; he notes that this is suppose to make the site better, but that tagging is more like reading comprehension and verifying what he just read versus adding additional information

Test D

 * Female (26-35)
 * Android, Motorola Xoom, Lenovo computer
 * Reads Wikipedia, mostly to ‘end arguments’ trivia/general information. knows anyone can edit, but hasn’t yet

Task:
 * User notices the feature, and remarks that generally she would probably skip it, and that the feature itself is unclear. ‘What does the ‘ok’ mean’? Is it just a statement, or will I be taken to a place where I can tag? Does like that the button is bright blue, otherwise she would’ve really skipped it
 * User very quickly answers the question. However, she comments that it was unclear earlier what exactly a tag was, and that it should be defined somewhere
 * User understood ‘none of these’ button, when it changed to ‘next’, and how to unselect a tag
 * User noticed and went to ‘tell me more’ section before even starting the task/feature. Remarks that the explanation makes sense to tech-savvy/educated population, but others might not know what a ‘text corpus’ is, for example
 * User not sure if she would do more in a row; would do it if it were on a topic she found interesting, or to correct an inaccuracy, or if she could add her own tag or change existing tags

Test E

 * Male (46-56)
 * Android, no tablet, PC computer
 * Reads Wikipedia, mostly for general reference, knows about editing but doesn’t

Task:
 * User notices the feature; ‘tagging info’, muses ‘i can add/edit?’, comments that he would typically skip this sort of thing
 * User answers the question after scanning the page briefly; understands the ‘none of these’ button, does not understand how to unselect a tag (he would just go back and restart the feature and reselect), no specific reaction to the button changing to ‘next’
 * Prompted to read the ‘tell me more’ section; understands, somewhat vaguely, that he can contribute via this feature
 * User says he would most likely not do more of these, but it would depend on the situation

Test F

 * Female (15-25)
 * iPhone, Nexus tablet, PC computer
 * Reads Wikipedia to ‘google stuff', knows about editing but feels she has enough knowledge to do so

Task:
 * User noticed the feature, and thinks the information on the article needs to be expanded; selected yes, remarks that that ‘tagged the article for expansion'
 * User understood ‘none of these’ button and how to unselect tag, no response to the button changing to ‘next’
 * Noticed ‘tell me more’, read it, but didn’t understand it. Thought it was tagging the article for improvement, both before and after she read
 * User considers her time valuable, but would maybe do one or two more because she likes to help improve things
 * Generally thinks the feature is fine