Winter/User tests

This page details various user test strategies and results that have been run against Winter and describes take-aways.

Abstract
Full user testing of Winter will require multiple test harnesses because:
 * Some questions will spoil the tester for other questions
 * Some questions require comparison between Vector and Winterized Vector
 * Some questions have multiple routes to the same answer

Harness One: Winter

 * 1) Can the user identify the search box? (Test search box identification)
 * 2) Can the user find the article discussion? (Test page-context action ribbon)
 * 3) Can the user find out how to edit the full article? (test page-context edit button)
 * 4) Second test with edit button as mw-ui-constructive at outset rather than on hover.
 * 5) Can the user discover how to find their contributions? (Test collapsed user menu)

Set Up

 * User is "auto-logged in" as User:Accedie under the hood, so that there is dummy data for contributions and the user page.
 * Page is pre-loaded with "Winter".

Introduction
Background: You're helping us test out a prototype of some software changes on Wikipedia, the world's largest free encyclopedia. Don't worry about messing anything up – this is just a prototype, not the real site! Mindset: You have made some edits to the encyclopedia in the past, but today you're researching various seasons on Wikipedia.

This test is best run using Google Chrome but will work well in Mozilla Firefox, too.

Tasks

 * 1) Navigate to the starting URL (http://unicorn.wmflabs.org/t/847). You will be asked to enter a user name; pick any name you like (maybe even use your own name). Don't worry, no data will be saved.
 * 2) Once you've "logged in", take a quick moment to look at this page and talk through what you're seeing.
 * 3) Use the search box to look up "Autumn" and go to that page (If you can't find this after 2-3 minutes, follow _this link_).
 * 4) All articles (and pages) on Wikipedia have an associated discussion. Can you find the discussion about "Autumn"?  (If you're stuck after 2-3 minutes, it's okay - just move on to the next task).
 * 5) If you haven't found the discussion, make sure you've scrolled to the top of the page and look at some links directly underneath the title of the page ("Autumn").  Click on the "Discussion" link and talk through what you think you see.
 * 6) Can you find your way back to the "Autumn" page from the discussion?  If you can't, just go to the next step.
 * 7) If you were unable to find your way back to the "Autumn" page, scroll back to the top of the page and click the "Article" link.  Once you're there, go on to the next step.
 * 8) Now that you're back on the main text of the article, let's say you discover a small typo in the first paragraph of the page "Autumn".  Can you find out how to make the change?  If you can't figure out how to make the change, go ahead and move on to the next step.
 * 9) If you weren't able to discover how to edit the page, scroll back to the top of the page and click on the "Edit" button in the top-right area of the article.  This should bring you to the edit screen.
 * 10) Take a few moments to describe what you see and then click "cancel" to return to the main article.
 * 11) Let's say you're an experienced editor who has made several hundred contributions and edits to the encyclopedia.  Can you find your list of contributions?  Take a few moments to look for them but if you can't find them after a minute or two just move on to the next step.
 * 12) Take a few moments to scroll around and describe your general impressions of this interface.  After that, we're done.

Questions

 * 1) How does this interface match with your expectations of using Wikipedia, what parts were easy to use and behaved as expected?
 * 2) In what ways did this interface not match your expectations of using Wikipedia, what parts were hard to use, or weren't quite clear to you?
 * 3) Are there tasks that you felt you were not successful in completing?
 * 4) Can we show this video to our editor community?

Test Videos
Five videos were made using Test Harness One. Videos are annotated with highlights, but major take-aways from each are included below.


 * 1) misterbeauds
 * 2) * Immediately notices the static header and thinks this is a good idea.
 * 3) * Discovers page context icons in the static header easily (though we were not testing this).
 * 4) * Discovers existence of Messages and Talk and thinks this is a a new feature for Wikipedia.
 * 5) * Quickly discovers page-context action ribbon. The fact that there are discussions about articles is completely new to him.
 * 6) * Instantly finds search box and expresses bafflement that anyone could not find it.
 * 7) * Figures out talk pages, but was expecting something more forum-like (speaks to Flow).
 * 8) * Instantly understands article-context navigation. "Easy-peasy".
 * 9) * No problems with edit or cancel.
 * 10) * Easily finds "Contributions" in the user menu.
 * 11) * Expressed that he really likes the search bar and the "new format". "It just feels more modern."
 * 12) kayaker
 * 13) * User actually reads the introductory text on the login screen and opens the MediaWiki page. Oops.  Time to remove that text.
 * 14) * User knew that tabs existed, but is pleased that they have moved.
 * 15) * Instantly finds search. "I like how it's big!  It's nice!"
 * 16) * Finds the edit button and the discussion link easily.
 * 17) * User has looked at talk pages before but only "kind of" understands them. Has confusion about "posts" (sections).
 * 18) * User finds the edit button easily enough, and also explores other options in the edit button drop down.
 * 19) ** She says, "I don't really want to edit visually, do I?" which speaks to a possible verbiage/branding problem with VE.
 * 20) * User cannot easily find the "cancel" button in the editor.
 * 21) * User correctly finds "Contributions" through the user menu, but encounters a different bug. She ends up back to contributions by going through her user page.
 * 22) * Is slightly confused by the way the buttons in the header change, but expresses a bit of joy shortly thereafter.
 * 23) * Definintely likes the bigger font.
 * 24) blairsmama
 * 25) * Easily finds Search.
 * 26) * Initially thinks that "Discussion" may be in the table of contents, but then quickly notices its location at the top of the screen.
 * 27) * Does not understand talk page templates.
 * 28) * User thinks that "Main page" link in the sidebar goes back to the "main page" for the article at first, but then quickly corrects that mistake.
 * 29) * Easily finds out how to edit the page.
 * 30) * No problems finding the editor's "Cancel" button.
 * 31) * User has IE, and runs afoul of broken menu bug (see below). Ends up finding "Contributions" through the in-page context action ribbon from the user page.
 * 32) Emilymae
 * 33) * Easily finds Search and Discussion.
 * 34) * Does not understand talk page templates.
 * 35) * Easily finds edit button.
 * 36) * No problems locating the editor's "Cancel" button.
 * 37) * First looks for "Contributions" under the Table of Contents icon but then quickly finds them in the user menu.
 * 38) Srap33
 * 39) * Easily finds Search and Discussion.
 * 40) * "It's pretty basic. Easy to read, easy to access information."
 * 41) * User understands purpose of talk pages but not necessarily how to use them.
 * 42) * "Edit was easy to find".
 * 43) * No problems locating the editor's "Cancel" button.
 * 44) * User has IE, and runs afoul of broken menu bug (see below). Ends up finding "Contributions" through the in-page context action ribbon.

Takeaways

 * No tester had difficulty recognizing the search bar.
 * All testers quickly grasped the in-page context action ribbon.
 * Several testers expressed surprise that Wikipedia had discussions, which lends even more credence to the idea that Vector tabs are effectively invisible.
 * The "Cancel" button from the editor may need work.
 * All testers found their way to the contributions page, even when faced with bugs/difficulties. They just did it in other routes.

Other Notes

 * The end of the test needs to be rewritten. Rather than discuss Winter itself, all participants discussed the faked-up "Contributions" view.
 * Several minor display bugs exist in Safari and Internet Explorer
 * The user drop down menu does not appear to work in Internet Explorer, which made for some confusion. The testers did not notice this, however, and completed the tasks anyway using other methods.

Harness Two: Winter

 * 1) After scrolling, can users identify the search box? (test adjusted search box identification)
 * 2) After scrolling, can users find the discussion? (test in-header discussion icon)
 * 3) After scrolling, can users find their own contributions? (test secondarily collapsed user menu)


 * It is quite likely that many of these will result in the user scrolling back to the top of the page.

Set Up

 * User is "auto-logged in" as User:Accedie under the hood, so that there is dummy data for contributions and the user page.
 * Page is pre-loaded with "Winter".

Introduction
Background: You're helping us test out a prototype of some software changes on Wikipedia, the world's largest free encyclopedia. Don't worry about messing anything up – this is just a prototype, not the real site! Mindset: You have made some edits to the encyclopedia in the past, but today you're researching various seasons on Wikipedia.

This test is best run using Google Chrome but will work well in Mozilla Firefox, too.

Tasks

 * 1) Please move the Usertesting instruction panel (this panel) to the lower left of your screen so that it does not cover the upper area of your browser.
 * 2) Navigate to the starting URL (http://unicorn.wmflabs.org/t/848). You will be asked to enter a user name; pick any name you like (maybe even use your own name). Don't worry, no data will be saved.
 * 3) Once you've "logged in", take a quick moment to look at this page and talk through what you're seeing.
 * 4) Navigate to the section of the page called "Festivals".  Did you have any difficulty doing so?
 * 5) Use the search box to look up "Autumn" and go to that page (If you can't find this after 2-3 minutes, follow _this link_).  Navigate to the section entitled "Melancholy association".
 * 6) All articles (and pages) on Wikipedia have an associated discussion. Can you find the discussion about "Autumn"?  (If you're stuck after 2-3 minutes, it's okay - just move on to the next task).
 * 7) If you haven't found the discussion, look in the top right area of the screen and find the icon that looks like two speech bubbles.  Click on this icon and talk through what you think you see.
 * 8) Can you find your way back to the "Autumn" page from the discussion? If you can't, scroll back to the top of the page and click the "Article" link, just below the article title.
 * 9) Now that you're back on the "Autumn" article, go to the section about "Tourism".
 * 10) Let's say you're an experienced editor who has made several hundred contributions and edits to the encyclopedia.  Can you find your list of contributions?  Take a few moments to look for them but if you can't find them after a minute or two just move on to the next step.
 * 11) Search for and go to the article "Summer".  Take a few moments to scroll around and describe your general impressions of this interface.  After that, we're done.

Questions

 * 1) How does this interface match with your expectations of using Wikipedia, what parts were easy to use and behaved as expected?
 * 2) In what ways did this interface not match your expectations of using Wikipedia, what parts were hard to use, or weren't quite clear to you?
 * 3) Are there tasks that you felt you were not successful in completing?
 * 4) Can we show this video to our editor community?

Test Videos
Five videos were made using Test Harness Two. Videos are annotated with highlights, but major take-aways from each are included below.


 * 1) asher
 * 2) * Instantly finds search after scrolling.
 * 3) * Did not know Wikipedia had discussions.
 * 4) * Confusion with double icons for talk page.
 * 5) * User assocatied icons in upper-right with "my stuff"; did not think that any page-context actions would appear there.
 * 6) * User had definite icon confusion.
 * 7) * Liked the in-header version of the Table of Contents
 * 8) * User found contributions but only by going to their user page first and then finding the Contributions in the in-page context ribbon.
 * 9) * Finds the edit button easily, once he's looking for it.
 * 10) limitlessappz - User did not give permission to share their test video.
 * 11) * User likes how the search box scrolls with the page; has no difficulty with it.
 * 12) * Immediately discovers Echo.
 * 13) * User is blind to the existence of the table of contents and does not use it until the very end.
 * 14) * User finds way to article talk, but through a link in a page template, not through navigation.
 * 15) * User finds Contributions but only by going to user page first and then using the context action menu.
 * 16) bablador
 * 17) * User scrolls to the top of the page to use search box. Does not notice the header being sticky.
 * 18) * User does not know that there are discussions.
 * 19) * User goes to discussion page using the context action ribbon, not the header icons.
 * 20) * Takes VERY little time to find Contributions link in the user menu pull down, but scrolls to the top of the page first.
 * 21) eatglitter
 * 22) * User is super happy that Wikipedia has added a table of contents.
 * 23) * Scrolls to the top to use the search box but finds it easily.
 * 24) * Goes to discussion through the action ribbon.
 * 25) * User correctly identifies the personal tools section.
 * 26) * User finds contributions but does so by going to their talk page and then clicking "contributions" in the page context ribbon.
 * 27) reneeineastbay
 * 28) * User easily finds search box. Does not scroll up.  "Just look for the little magnifying glass."
 * 29) * User scrolls to the top of the page to use the action ribbon to get to the discussion page.
 * 30) * While on the user page, user does not see the contributions link inside the context ribbon.
 * 31) * Does not recognize pull-down menu icon in personal tools.
 * 32) * Never finds contributions.

Takeaways

 * Search is very easy to find.
 * Users are almost hard-wired to return to the top of the page to look for actions.
 * No users correctly discovered the contextual header icons.
 * Everyone is baffled by talk page templates.

Harness Two: Electric Boogaloo

 * 1) After scrolling, can users identify the search box? (test adjusted search box identification)
 * 2) After scrolling, can users find the table of contents? (test in-header table of contents icon)
 * 3) After scrolling, can users find their own contributions? (test secondarily collapsed user menu)


 * It is quite likely that many of these will result in the user scrolling back to the top of the page.

Set Up

 * User is "auto-logged in" as User:Accedie under the hood, so that there is dummy data for contributions and the user page.
 * Page is pre-loaded with "Winter".

Introduction
Background: You're helping us test out a prototype of some software changes on Wikipedia, the world's largest free encyclopedia. Don't worry about messing anything up – this is just a prototype, not the real site! Mindset: You have made some edits to the encyclopedia in the past, but today you're researching various seasons on Wikipedia.

This test is best run using Google Chrome but will work well in Mozilla Firefox, too.

Tasks

 * 1) Please move the Usertesting instruction panel (this panel) to the lower left of your screen so that it does not cover the upper area of your browser.
 * 2) Navigate to the starting URL (http://unicorn.wmflabs.org/t/849). You will be asked to enter a user name; pick any name you like (maybe even use your own name). Don't worry, no data will be saved.
 * 3) Once you've "logged in", take a quick moment to look at this page and talk through what you're seeing.
 * 4) Navigate to the section of the page called "Festivals". Did you have any difficulty doing so?
 * 5) Use the search box to look up "Autumn" and go to that page (If you can't find this after 2-3 minutes, follow this link). Navigate to the section entitled "Melancholy association".
 * 6) All articles (and pages) on Wikipedia have an associated discussion. Can you find the discussion about "Autumn"? (If you're stuck after 2-3 minutes, it's okay - just move on to the next task).
 * 7) If you haven't found the discussion, scroll back to the top of the page and click the "Discussion" link, just below the article title. Talk through what you think you see.
 * 8) Can you find your way back to the "Autumn" page from the discussion? If you can't find it after a few minutes, it's okay - just move on to the next task.
 * 9) If you haven't found your way back to the article, scroll back to the top of the page and click the "Article" link, just below the title.
 * 10) Now that you're back on the "Autumn" article, go to the section about "Tourism".
 * 11) Let's say you're an experienced editor who has made several hundred contributions and edits to the encyclopedia. Can you find your list of contributions? Take a few moments to look for them but if you can't find them after a minute or two just move on to the next step.
 * 12) Search for and go to the article "Summer". Take a few moments to scroll around and describe your general impressions of this interface. After that, we're done.

Questions

 * 1) How does this interface match with your expectations of using Wikipedia, what parts were easy to use and behaved as expected?
 * 2) In what ways did this interface not match your expectations of using Wikipedia, what parts were hard to use, or weren't quite clear to you?
 * 3) Are there tasks that you felt you were not successful in completing?
 * 4) Can we show this video to our editor community?

Test Videos

 * 1) bryanfm
 * 2) * This tester is extremely literal minded. Only clicks on things when directed to.
 * 3) * User quickly discovers their personal bar (and identifies Echo)
 * 4) * User correctly finds search but scrolls to the top of the page first.
 * 5) * Finds the discussion link in the action ribbon. It takes a while, though.
 * 6) * Completely, utterly baffled by the article talk page.
 * 7) * No problems finding the "back to article" link now that he knows what the context ribbon is. Does not use header icons.
 * 8) * User scrolls to the top to access his personal toolbar. He does NOT associate the person icon in the static header to him.
 * 9) * User likes to zoom in a bit because he has "less than perfect eyesight"
 * 10) * Did not discover the in-header TOC until the very end.
 * 11) LuckyMe
 * 12) * User scrolls to the top to get to the search box but has no trouble finding it there.
 * 13) * User scrolls to the top of the screen and quickly locates the Discussion link in the article context ribbon.
 * 14) * User correctly identifies talk page topics, but isn't sure she is allowed to edit them. Cannot identify the authors of comments (doesn't recognize signatures).
 * 15) * User is absolutely not associating the person icon with "her stuff". Eventually she finds it, but only after her username is next to it.
 * 16) ddsurfer
 * 17) * Instantly finds search box. Does not scroll back up.
 * 18) * Finds the discussion link in the context action ribbon, but it takes a little time.
 * 19) * Navigates back to the article using the context ribbon.
 * 20) * She discovers the top bar icons, but she's looking for her contributions now, and doesn't think to open the user menu.
 * 21) * When looking for her contributions, the user clicks on the article history icon in the header. This is the first time I've seen anyone use those icons, and it's done incorrectly.
 * 22) * Does NOT find contributions and gives up looking. . .BUT she finds them at the very end (once the user name is visible again).
 * 23) HyruleHero57
 * 24) * User *immediately* recognizes that he has a "personal tools" area though he doesn't associate Wikipedia with having accounts.
 * 25) * User is fairly familiar with Wikipedia's interface. Finds the search box change and calls it out.
 * 26) * Finds the in-page context action ribbon fairly easily.
 * 27) * User scrolls to the top to use the search box.
 * 28) * Wonders a bit about how hard the search is to see (even though he found it instantly). Expresses delight at the auto-complete feature, but we've had that forever.
 * 29) * Scrolls to the top to use the context action ribbon to find the Discussion.
 * 30) * Does NOT see talk pages as "discussions". "It's just people writing stuff." ABSOLUTELY expects the talk page to behave like a modern discussion system.
 * 31) * Re: Contributions: "I assume it's going to be in this top-right corner." Scrolls to the top of the page.
 * 32) * Goes to the user page and ignores the pull down menu. Finds Contributions in the context ribbon.
 * 33) sav7
 * 34) * This tester is the only tester in both versions of this test to correctly find and use the header action icons. He also appears more technically savvy than the others.
 * 35) ** However, he does suffer from icon confusion. It's possible he thinks the icons are shuffled.
 * 36) * Immediately discovers the personal tools section and points out that he can see his username there.
 * 37) * Finds search without scrolling to the top.
 * 38) * Navigates to article talk by using the header icon. Is the only one to do so.
 * 39) * User understands talk pages but thinks they are confusing.
 * 40) * Navigates back to the article using the context ribbon.
 * 41) * Correctly finds the contributions page by using the personal tools pull down. Does not scroll to the top of the page (again, only user to do so).
 * 42) * Very happy with the personal tools bar and header icons.

Takeaways

 * Most testers do NOT associate the personal tools with "my stuff" unless the username is visible. If the username is there, they can find it easily.
 * With one exception, users do not notice the header icons changing in a meaningful way.
 * At this point, with 10 testers and a 10% success rate, I'd say that the benefits of putting page icons in the header are outweighed by the negatives of losing the username for discoverability.

Harness Three: Winter: No Header Icons

 * 1) After scrolling, can users identify the search box? (test adjusted search box identification)
 * 2) After scrolling, can users find the Discussion? (test location of context ribbon)
 * 3) After scrolling, can users find their own contributions? (test secondarily collapsed user menu)

Test Videos

 * 1) ragelle72
 * 2) * User instantly recognizes the personal tools area.
 * 3) * Easily finds search box; does not scroll to the top.
 * 4) * Goes to talk page through a link in an internal template. Does not understand what talk pages are.
 * 5) * When she discovers the action ribbon, indicates that she didn't see it at first (suggested a different color). However, uses it easily from that point on and says she likes it.
 * 6) * Quickly finds contributions in the user tools menu, but scrolls to the top of the page.
 * 7) * "It's very easy to navigate through and figure it out." She *loves* the personal tools area.
 * 8) ThatDuckGuy
 * 9) * This is a highlight test. The user is very chatty.  It is a 100% success.
 * 10) * Immediately notices the personal tools area and notes that he is logged in (discoverability of username).
 * 11) * Immediately finds the context action ribbon.
 * 12) * Notices that the page scrolled for him.
 * 13) * Immediately goes to the sticky search box; does not scroll.
 * 14) * Immediately scrolls to the top to find the discussion link.
 * 15) * Instantaneously goes to the user tools pulldown menu to find his contributions.
 * 16) * Likes the interface, thinks it is clean and modern. Loves the scrolling behavior.
 * 17) * Likes the context action ribbon. Thinks its easy to switch back and forth between pages.
 * 18) * Likes the search at the top, and likes that it is sticky. Likes the auto-complete.
 * 19) kittypurry
 * 20) * Tester's set up has a lot of microphone buzz, so be warned.
 * 21) * Instantly grasps their personal tools section by pointing out their user name.
 * 22) * Scrolls to the top to find search. Likes the size of the search bar.
 * 23) * User expects a link to the discussion in the Table of Contents. As with many others, also scrolls to the bottom of the page (expecting blog/news story like comments area).
 * 24) * Finds the discussion link in the action ribbon but it takes a little while.
 * 25) * Quickly goes to and uses the context action ribbon to return to the article.
 * 26) * Easily finds contributions in the user pull down menu. Has scrolled to the top, though.  Does not understand what the contributions are.
 * 27) alonb1977
 * 28) * Instantly notes that he is logged in by recognizing that his username is visible in the personal tools section. Examines each icon.
 * 29) * Notices the context action ribbon. Correctly grasps what the watch link does and associates it with his watchlist.
 * 30) * Instantly goes to the search box. Does not scroll up.
 * 31) * Immediately scrolls to the top to use the context action ribbon to go to the talk page. Has NEVER seen the discussions before.
 * 32) * Immediately and unconsciously uses the action ribbon to return to the article. He continues to use it to navigate for the rest of the session without hesitation.
 * 33) * Instantly goes to his user pull down menu to find contributions. No hesitation.
 * 34) * He has some good words for the help documentation.
 * 35) wldrlb
 * 36) * Notes that she is "logged in" by the presence of her name in the user tools section.
 * 37) * Scrolls to the top for the search box. Expects it in the upper right; finds it easily enough but thinks it's weird.  However, states later "Now that I know the search is here, I like it."
 * 38) * Scrolls to the top of the page and easily finds the Discussion link in the action ribbon.
 * 39) * Immediately goes to the "Article" link in the context ribbon to return to "Autumn"
 * 40) * Quickly and easily goes to her pull down menu in the user tools area to find her contributions.

Takeaways

 * All testers completed all tasks.
 * Most users quickly pointed out that they knew they were "logged in" because of their username appearing in the corner and noted that they have a set of personal or profile tools there. More than one tester referred to this area as "their profile".  No users had difficulty finding their contributions because of this.
 * With one exception, all users found the context action ribbon quickly. The one who did not later was observed using it with proficiency.
 * Many users unconsciously navigated with the context action ribbon afterwards.
 * The search box remains mostly discoverable. Several testers unconsciously used it in the sticky format without scrolling to the top.
 * No one used the table of contents in the header.

Harness Four: Winter: Only Header Actions

 * 1) After scrolling, can users identify the search box? (test adjusted search box identification)
 * 2) After scrolling, can users find the discussion? (test in-header discussion icon)
 * 3) After scrolling, can users find their own contributions? (test secondarily collapsed user menu)

Tasks

 * 1) Please move the Usertesting instruction panel (this panel) to the lower left of your screen so that it does not cover the upper area of your browser.
 * 2) Navigate to the starting URL (http://unicorn.wmflabs.org/t/851). You will be asked to enter a user name; pick any name you like (maybe even use your own name). Don't worry, no data will be saved.
 * 3) Once you've "logged in", take a quick moment to look at this page and talk through what you're seeing.
 * 4) Use the search box to look up "Autumn" and go to that page (If you can't find this after 2-3 minutes, follow this link).
 * 5) All articles (and pages) on Wikipedia have an associated discussion. Can you find the discussion about "Autumn"? (If you're stuck after 2-3 minutes, it's okay - just move on to the next task).
 * 6) If your found the discussion, great! Go ahead and go back to the "Autumn" article by clicking the browser back button or using _this link_.
 * 7) Now that you're back on the main text of the article, let's say you discover a small typo in the first paragraph of the page "Autumn". Can you find out how to make the change? # If you can't figure out how to make the change, go ahead and move on to the next step.
 * 8) If you weren't able to find out how to edit the article, that's okay. Go ahead and move on to the next step. If you were able to do so, take a few moments to describe what you see and then click "cancel" to return to the main article.
 * 9) Let's say you're an experienced editor who has made several hundred contributions and edits to the encyclopedia. Can you find your list of contributions? Take a few moments to look for them but if you can't find them after a minute or two just move on to the next step.
 * 10) Take a few moments to scroll around and describe your general impressions of this interface. After that, we're done.

Test Videos

 * 1) andreanicole
 * 2) * Quickly finds the search box. Likes it.
 * 3) * Takes a very long time to find the header icons. However, she does not distinguish between the article talk icon and her user talk icon, and ends up on her user talk page.
 * 4) * User cannot find the article discussion and gives up. However, she later finds the link during a different task. Expresses that it took way too long to find and that having two similar icons is confusing.
 * 5) * Does not find the full page edit control (in the header); only for a section.
 * 6) * She is unable to complete the contributions test because, contrary to instructions, she is doing the test with Internet Explorer. IE has a bug where the pull down menus are not working correctly.
 * 7) * She suffered a great deal from icon confusion and let us all know about it.
 * 8) pedagogue
 * 9) * Tester discovers page action icons while exploring the interface. Takes a moment to find the search box.
 * 10) * Quickly finds the discussion link now that he's been here, but points out the only reason he found it is because he was looking for the search bar.
 * 11) * Quickly finds the edit icon now that he knows where to look.
 * 12) * Has difficulty finding the user drop down menu; ends up on his user page while looking for his contributions. Does not find them.
 * 13) * Has icon confusion and cannot distinguish between his watchlist and the "watch this page" action.
 * 14) * Eventually finds contributions in the pull down.
 * 15) mflnamore
 * 16) * Instantly finds the search box.
 * 17) * Tries to find "discussion" using CTRL-F. Eventually finds the talk page but only through a link in a page template.
 * 18) * Takes a while to find the edit icon in the header. Relies heavily on the tooltips.
 * 19) * Goes to his user page to find contributions. Does not actually succeed; assumes that text on the user page is his contributions.
 * 20) jefflou
 * 21) * User cannot find the search box and gives up, using the link. The link actually opens a new instance of the site in a differnt browser (Opera).
 * 22) * Looks for the discussion at the bottom of the page before eventually finding the icon in the header.
 * 23) * Correctly finds the edit icon, now that he's been exposed to the header icons.
 * 24) * No trouble with the cancel button.
 * 25) * Interestingly, he finds the article actions pull down, but does not associate the same symbol (v) next to his own username with a pull down at first.
 * 26) * He finds the contributions link but runs into an Opera bug.
 * 27) * He wants labels on the icons.
 * 28) blazed75
 * 29) * This tester is a fast talker.
 * 30) * Instantly finds the search box and notes that it has changed. Really likes the changes.
 * 31) * Cannot find the discussion and moves on.
 * 32) * Ends up in the editor, but from a section edit link. Likes the editor over all.
 * 33) * Correctly grasps that their contributions will be associated with their account. Did not know that you can create an account.

Takeaways

 * As usual, the search box is fairly easy to find and is somewhat of a hit.
 * Most users encountered signficant icon confusion. In some cases, the confusion was enough to prevent them from accomplishing a task.  In other cases, the confusion will obviously be problematic as cental editing themes are confused.
 * The discoverability of the user tools section, when the username is present, is very high and remains so.
 * The discoverability of the page action icons is almost non-existent. Several users expressed a desire for action labels. Many users failed to accomplish tasks that required these icons, or assumed they had achieved success when they did not. It is my belief that at least two testers assumed that the page action icons were just parts of their personal tools, and never looked there for page actions.
 * No one has ever used the in-header Table of Contents.

Harness: Vector

 * 1) Can the user identify the search box?
 * 2) Can the user find the discussion button?
 * 3) Can the user find the edit button?

Harness Five and Five point One: Winter

 * 1) Can users find the history of the page?​​ (testing whether users can identify the 'last updated' action button as the history)
 * 2) Can users​ identify the ToC icon on top left as a way to access the ToC/additional elements and actions for the article? (testing intuitiveness/visibility of ToC icon as an alternative to scrolling)
 * 3) Do users find search intuitive/easy to find?​ (testing ability to find another wiki page)
 * 4) Can users find the talk/discussions page? (testing whether users can identify another button in the actions bar)

Set Up

 * User will log in with dummy account in winter-working​ prototype

Introduction
Today​, you'll be helping us test some potential software changes we may make to Wikipedia. You may be an experienced editor or a casual reader, and today you will be researching seasons on Wikipedia. The version of Wikipedia you'll be working with today is a prototype, not the actual site, so no need to worry about ​messing anything up as you move through the tasks below.​ Please make sure you're running this prototype on Google Chrome. Remember, we're testing our interface, not you, so if you can't complete a task or find it difficult, it's *our* fault, not yours.

Tasks

 * 1) Please move the Usertesting instruction panel (this panel) to the lower left of your screen so that it does not cover the upper area of your browser.
 * 2) Please navigate to this link (you're probably here already). You'll be prompted to enter a username; feel free to use a nickname or your own name.
 * 3) A​fter you've logged in, please take a moment to briefly describe your initial reactions ​ ​to the main elements you see on the page and tell us what purposes you think they might serve. In 2-3 minutes, please move on to the next task.
 * 4) Can you find a way to discover who edited this page last? If you don't find it after 1-2 minutes, don't worry about it, and move on to the next task.
 * 5) Let's make sure we're back on the Winter article. Please click this link back to the "Winter" article page and move on to the next task.
 * 6) Wikipedia allows users to discuss issues related to the article. Can you find and go to the Winter discussion page? If you can't find it in a minute, feel free to move to the next task.
 * 7) If you figured it out, fantastic! If you didn't find the discussion/talk page, go to the top of the page and click on the "Discussions" tab right under the article title. Take a minute or two to describe what you're seeing and thinking about this page and then move on to the next task.
 * 8) Can you find your way back to the "Winter" article?  If so, go ahead.  If not, click this link.
 * 9) Now that you're back on the article page, navigate to the "Festivals" section of the article. (If you can't find this after a minute, follow this link).
 * 10) Please search for the "Autumn" page and go to that page. Haven't found the page after 1-2 minutes? No worries, skip to the next task​.
 * 11) If you searched for and found yourself on the "Autumn" article, great! Otherwise, click this link. Scroll down the page until you see the photo titled "Halloween pumpkins".
 * 12) Without scrolling, can you find a way to jump to the section of this article titled "Harvest Association"? If you're stuck after a minute or two, just carry on to the next task.
 * 13) You're almost done! Using search, look up the "Summer" article page. Talk us through your general impressions and your experience finding and navigating the information on the site. If you'd like, use our short list of questions below to guide your responses. After this, you're all finished!

Questions

 * 1) How does this interface compare to what you expect from Wikipedia?​ Do you feel confident that you could find the information that you are looking for?
 * 2) What features were intuitive and easy to use?
 * 3) What did you find difficult to use, confusing, or generally not what you expected or envisioned?
 * 4) Do we have your permission to share this video with our volunteer community?

Test Videos
Five videos were made using Test Harness Five and Five point Zero. Videos are annotated with highlights, but major take-aways from each are included below.


 * 1) papabear
 * 2) * Thinks 'discussions' are a new feature
 * 3) * Confused about global activity sidebar - wikipedia generally or article-specific?
 * 4) * Page feels 'more interactive', 'more personalized and social' than current
 * 5) * Initial confusion about where page history can be found
 * 6) * Though unnecessary, scrolled up once to engage search
 * 7) * Didn't notice ToC button initially, but found it immediately upon the specific task prompt
 * 8) * Commented on ToC gutter, wants wrap text
 * 9) superth1 did not want video shared
 * 10) * Likes page immediately. Wants bigger font size for page title
 * 11) * Comments throughout that page looks more modern. Points out the personal tools
 * 12) * Notices majority of new features during initial scan (minus the ToC icon). Notes the language select option and comments that landing on a page on current site in a duff language can be confusing to navigate
 * 13) * Edit button hover options redirects user correctly to history last updated button
 * 14) * Does not figure out how to 'add a comment' to discussions page. Even clicks to edit and says, 'don't want to do that!'. notes that you can edit existing but not add. interesting
 * 15) * No luck finding ToC button. Not sure if issue was content-oriented or because of the 'box' syndrome around the page. However, did move cursor around the page extensively, like for searching how to edit the discussions page earlier
 * 16) * Wants ToC moved to the right (I'm assuming because of the white space it creates)
 * 17) ssoohoo
 * 18) * Notes the personal tools and in-page ToC section (as if it were not noticed previously on current version)
 * 19) * Thinks a gallery of images in the ToC gutter would be a good use of space
 * 20) * Finds history immediately and highlights Jared's username link
 * 21) * Finds discussions tab immediately - wants a more structured discussions page
 * 22) * Does not use search for 'Autumn' page - uses in page navigation
 * 23) * Does not find ToC button, searches in page instead, not looking for a new navigation tool
 * 24) * Uses search for last task, initially expresses confusion, changes mind
 * 25) cbr600ps
 * 26) * User has issues with finding history page. Brandon notes that last modified timestamp on bottom of the page needs to be fixed. Clicks on discussion, noting that it's right next to the history page. Then clicks history
 * 27) * Finds discussions page quickly, but has already been here
 * 28) * Confused about discussions page. goes to his own talk page. realizes discussions page is the discussions/talk page
 * 29) * Does not use in-page ToC section to go to festivals - scrolls through page. Also searches in-page for 'Autumn', and ultimately uses the seasons outline portal. Knows search exists, but doesn't use
 * 30) * Does not find ToC button
 * 31) * Likes that the translation/languages button/drop-down exists
 * 32) * UK user
 * 33) jlapidus
 * 34) * Comments on search bar and how big it is immediately
 * 35) * Expresses some confusion around the personal tools section, pokes around the watchlist
 * 36) * Finds and understands history page immediately
 * 37) * Instantly finds discussions page. Expects more of a forum or blog comment layout, confused at how it does look
 * 38) * Likes ToC section. Uses search instantly, no scrolling
 * 39) * Finds ToC header button after a moment, likes the pop-up/pull-down menu, really likes the button
 * 40) * Comments that it captures the existing site well but adds good features

Takeaways

 * Users take notice of the personal tools area.
 * Users really want something in the ToC gutter area if it exists.
 * Users may have issues figuring out that the task for the ToC button is regarding a new navigation tool - perhaps the target section name in the task needs to be changed to another without 'association' in the name, to avoid users looking into the content vs searching for nav tool as desired/goal.
 * Search box might not be as apparent as it could be. Search sometimes bypassed in favor of scrolling.
 * Users tend to like the font.
 * Users like the language translation button/drop-down.
 * Users sometimes comment on talk pages not being organized enough, or have different expectations of what it should look like.

Other Notes

 * User does not see questions at the end while working on tasks - keep this in mind for future write-ups. The last task was edited for tests 6A-D.