Page Previews

Jeetendriyam Dewangan The Hovercards beta feature solves the core problem of users needing to open multiple tabs to gain an understanding of a word in the context of the subject they are reading. With Hovercards, whenever a reader hovers over a link to another article, a short summary of the subject and an image (if available) is shown to them, so that they can decide whether they need to visit that subject more fully before continuing the current subject.

There have been many requests over the years for a similar feature; there are browser extensions (see list) and one heavily used gadget (the more editor-focused Navigation popups) that also exist to solve this problem.

The Hovercards settings panel enables logged-out users to disable it entirely, and enables logged-in users to choose between Hovercards, Navigation popups, or disabling both.

Intended users and objective
Our primary users are the casual readers who browse around the Wikipedia without any particular goal, simply enjoying reading the articles.

Hovercards will be of use to any user of Wikipedia who browses around the site frequently.

Our product will make it easier to get an overview of an article before you decide whether or not to browse to it. Consequently, the smoothness of the user experience for these users will be increased.



FAQ
This is a feature intended to improve the experience for any reader who normally would have clicked on a blue-link in wikipedia because they needed an overview (definition) of that entity.
 * Why Hovercards?

The theory of impact for hovercards is that they lower the cost of exploring a link. This should mean that users are less inhibited about exploring links. We should see that the overall links clicked + (non-accidental) hovers exceed the number of links clicked without hovercards. This is what success looks like. There are also some indications of failure that we will look for:
 * How do we measure Hovercards performance?
 * hovers can be accidental-we need to measure normal dwell time in a controlled condition to ensure that the likely rate of accidental hovers is not too high. To give an example, if a user must dwell on a link for 250 msecs before a hover shows, then we would want to make sure that there are not a large number of users who tend to dwell on a link for more than 250 msecs without clicking it.
 * hovercards could lead to fewer pageviews, because the user gets the information that they need from the preview- this is not a problem, but we want to make sure that the decrease (if any) does not result in significantly less editing or fundraising.
 * the % of hovers that result in a user continuing to the page is high - this would suggest that most people wanted to go to the page anyway. If this is the case, then the hover is likely just adding an unnecessary step.  We expect some significant % of click throughs for hovers.  It is ~60% on Android for a similar feature, but we expect it to be lower on desktop.

If you have Navpops enabled, hovercards are automatically disabled. You will have disable Navpops in order to experience hovercards. This is an intentional decision to ensure that Navpop users' do not have their preferences interrupted. Right now, hovercards will either be turned on or off by a user. The option to turn them off lives in each hover event. So at each hover event, a user can decide that they are no longer interested. In order for a user to turn hovercards back on, they will have to go to their user settings. This currently does not exist for logged-out users, so we are creating it!
 * What if I have Navpops enabled by default?
 * How many options do I have in Hovercards preferences?

At the project level, administrators can determine how long a user should dwell on a link before a hover is triggered. If a project wants to be conservative, the lag can be longer. WMF will have a recommendation as to optimum dwell time that provides the best user experience while minimizing accidental hovers. Unfortunately, there is no good way to tell users about a new feature. Central notice banners have been suggested, but running them for 2 weeks would not solve the needs of future users and we do not want to run them continuously. Notifying a user of a new feature is best done using a...hover-over. So that is why we are planning to show an initial hover on the users first hover event, followed by an explanation of what the user has seen. They can choose to turn them off in that dialogue. Please go to the hovercards discussion page, here: Talk:Beta Features/Hovercards
 * Why can't users just turn on hovercards if they want them?
 * Why can't users just turn on hovercards if they want them?
 * If I have hovercards feedback or if I have a suggestion for making them better, where should I go?
 * If I have hovercards feedback or if I have a suggestion for making them better, where should I go?

Key differences to Navigation Popups
Hovercards is styled to enable easy reading. Currently, Navpopups' type-size is small and the margins are tight.
 * Create a consistent visual style with the article type treatment.
 * Emphasize the lead images, so a user gets an idea of a term using both text and image.
 * The action set in the gadget feels out of context, hence we need to validate which of those actions are useful for readers and editors.
 * Include the last edited timestamp that we have tested on Wikipedia mobile web, but de-emphasize it.

Open/Close delay timing
You can change the timing of how fast Hovercards appear and disappear (open/close delay); see Extension:Popups for instructions.

Enable/Disable
If you turn Hovercards off via the cog-icon, you can turn it back on at the bottom of any wikipage with the "Enable previews" link.

Logged-out users can turn Hovercards off via the cog-icon (settings saved in cookies or localstorage). Logged-in users who have the Navigation Popups enabled can also turn Hovercards off.



Future priorities
Design mockups for:
 * Article titles
 * Reference Tooltips (T67114)
 * Cross-wiki Hovercards (T67117), and subsection links (T65792), particularly for Wiktionary
 * User names
 * Integrate with Navigation Popups

Code

 * Analytics and instrumentation
 * Hovercards are currently instrumented to measure many aspects of usage. Please see here for details and feel free to ask any clarifying questions

2016 a/b tests
In 2016 we enabled a/b testing on hovercards so that we could look at hovercard behavior and see how that impacts learning, by comparing the sessions of users with hovercards auto-enabled and the sessions of users without hovercards auto-enabled.

Here is how we intend to interpret the data:

Are Hovercards good for our users:

 * 1) Rate of hovercards being valuable v. disruptive
 * 2) Do the number of links hovered per page increase or decrease?
 * 3) Ideally we would measure if hovercards make a user to spend more time with us (where time is a proxy for learning) by measuring impact on time per session or sessions per user, but this might not be possible.
 * 4) What is the rate of erroneous triggers?
 * 5) * Estimated by taking the dwelledbutAbandoned time length in control group and seeing how many are above the trigger time.  Apply # of these per page to the number of hovers in experimental.  That’s at least my best guess.
 * 6) What % of people dislike hovercards?
 * 7) % of sessions where people disable hovercards?
 * 8) ^ over time
 * 9) Distribution of # of hovers before someone disables?
 * 10) Impact on pageviews and fundraising
 * 11) What is the impact on links or hovers clicked of being in the experimental group?
 * 12) What is the impact on session depth of being in the experimental group?
 * 13) What is the impact on $ donated of being in the experimental group?
 * 14) Getting in the way of people who just want the next article:


 * 1) If 100% of the time, people continue onto article after seeing a hover, this is bad.  We want to keep the ratio below 70%.
 * 2) If 0% of the time people continue onto the article, that is also a sign that we are making it too hard.  We want to keep this ratio above 10%
 * 3) * Of hovers where user continues onto article

Descriptive:

 * 1) How long do people spend hovering before they click?
 * 2) How long do people spend hovering before they dismiss the hover?

Diagnostic:

 * 1) Is it ever the case that “perceived” wait (amount of time before hover shows) exceeds “popup delay” (amount of time to trigger) by more than 200 ms?  What % of the time?
 * 2) Frequency of error states (per hover)

2015 Greek and Catalan Wikipedia test
''We ran a test where hovercards were turned on by default for Greek and Catalan Wikipedia for a period of 4 months. We have both quantitative and qualitative data for this experiment.''

Here is a list of major bugs reported.

Known & documented issues:

 * Cutting sentences randomly
 * Info in parenthesis ignored (https://phabricator.wikimedia.org/T91344 )
 * Period problem with TextExtracts  (send email to Gergo and Prateek)
 * Hovercards not going away

Unknown and need more information:

 * Text alignment issue in certain scripts
 * Disambiguation text showing in some cases - this is when a page is just using plain indented/italic text instead of a template (e.g.). Or, if the hatnote template doesn't include.
 * Half sentences are annoying, people want more text
 * People are unhappy with animation speed, some want it faster, some slower (replicating the existing Reference Tooltips options, would be best.

Known and hard to solve for
 * Issues with PageImages (…)
 * Map shown without dot (This should be fixed once Max and Yuri are done with the tile server)
 * Reference hovercards (user and special pages too) (Prateek and Bene* are working on a restructure)
 * Sub-section links (TextExtracts)
 * Make it work with Wikidata (https://phabricator.wikimedia.org/T69434 )

Qualtrics Survey Results
In April of 2015, hovercards were rolled out to all users on Catalan and Greek Wikipedias. They also continued to be a beta feature on english and other wikipedias. During this time, a survey was shown to all hovercards users using a banner recruitment. There were over 1,700 responses, heavily dominated by non-logged in users on Catalan and Greek and logged in users (probably mostly editors) on English. Below are the aggregated results

Questions 2-6 are the qualitative text answers to whatever the user chose for open feedback. We cannot share these answers on wiki, as they have not been vetted to remove potentially personally identifiable information.

Here are some random, contiguous sample rows from those answers:
 * bug:
 * other problem:
 * positive feedback:
 * suggestion for improvement:

Project-level rollout discussions
Here is where the potential rollout of hovercards is being discussed:

English Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Proposal:_Enable_Hovercards_by_default