Page Previews

"Vatreni kolektiv (Vatra) osnovan je1.12.2015 godine u Gradačcu.Sviraju Rock,Pop-rock."

Denisa Drndić -prateći vokal


Logged-out users can turn Hovercards off via the cog-icon (settings saved in cookies or localstorage). Logged-in users who have the Navigation Popups enabled can also turn Hovercards off.



Future priorities
Design mockups for:
 * Article titles
 * Reference Tooltips (T67114)
 * Cross-wiki Hovercards (T67117), and subsection links (T65792), particularly for Wiktionary
 * User names
 * Integrate with Navigation Popups

Code

 * Analytics and instrumentation
 * Hovercards are currently instrumented to measure many aspects of usage. Please see here for details and feel free to ask any clarifying questions

2016 a/b tests
In 2016 we enabled a/b testing on hovercards so that we could look at hovercard behavior and see how that impacts learning, by comparing the sessions of users with hovercards auto-enabled and the sessions of users without hovercards auto-enabled.

Here is how we intend to interpret the data:

Are Hovercards good for our users:

 * 1) Rate of hovercards being valuable v. disruptive
 * 2) Do the number of links hovered per page increase or decrease?
 * 3) Ideally we would measure if hovercards make a user to spend more time with us (where time is a proxy for learning) by measuring impact on time per session or sessions per user, but this might not be possible.
 * 4) What is the rate of erroneous triggers?
 * 5) * Estimated by taking the dwelledbutAbandoned time length in control group and seeing how many are above the trigger time.  Apply # of these per page to the number of hovers in experimental.  That’s at least my best guess.
 * 6) What % of people dislike hovercards?
 * 7) % of sessions where people disable hovercards?
 * 8) ^ over time
 * 9) Distribution of # of hovers before someone disables?
 * 10) Impact on pageviews and fundraising
 * 11) What is the impact on links or hovers clicked of being in the experimental group?
 * 12) What is the impact on session depth of being in the experimental group?
 * 13) What is the impact on $ donated of being in the experimental group?
 * 14) Getting in the way of people who just want the next article:


 * 1) If 100% of the time, people continue onto article after seeing a hover, this is bad.  We want to keep the ratio below 70%.
 * 2) If 0% of the time people continue onto the article, that is also a sign that we are making it too hard.  We want to keep this ratio above 10%
 * 3) * Of hovers where user continues onto article

Descriptive:

 * 1) How long do people spend hovering before they click?
 * 2) How long do people spend hovering before they dismiss the hover?

Diagnostic:

 * 1) Is it ever the case that “perceived” wait (amount of time before hover shows) exceeds “popup delay” (amount of time to trigger) by more than 200 ms?  What % of the time?
 * 2) Frequency of error states (per hover)

2015 Greek and Catalan Wikipedia test
''We ran a test where hovercards were turned on by default for Greek and Catalan Wikipedia for a period of 4 months. We have both quantitative and qualitative data for this experiment.''

Here is a list of major bugs reported.

Known & documented issues:

 * Cutting sentences randomly
 * Info in parenthesis ignored (https://phabricator.wikimedia.org/T91344 )
 * Period problem with TextExtracts  (send email to Gergo and Prateek)
 * Hovercards not going away

Unknown and need more information:

 * Text alignment issue in certain scripts
 * Disambiguation text showing in some cases - this is when a page is just using plain indented/italic text instead of a template (e.g.). Or, if the hatnote template doesn't include.
 * Half sentences are annoying, people want more text
 * People are unhappy with animation speed, some want it faster, some slower (replicating the existing Reference Tooltips options, would be best.

Known and hard to solve for
 * Issues with PageImages (…)
 * Map shown without dot (This should be fixed once Max and Yuri are done with the tile server)
 * Reference hovercards (user and special pages too) (Prateek and Bene* are working on a restructure)
 * Sub-section links (TextExtracts)
 * Make it work with Wikidata (https://phabricator.wikimedia.org/T69434 )

Qualtrics Survey Results
In April of 2015, hovercards were rolled out to all users on Catalan and Greek Wikipedias. They also continued to be a beta feature on english and other wikipedias. During this time, a survey was shown to all hovercards users using a banner recruitment. There were over 1,700 responses, heavily dominated by non-logged in users on Catalan and Greek and logged in users (probably mostly editors) on English. Below are the aggregated results

Questions 2-6 are the qualitative text answers to whatever the user chose for open feedback. We cannot share these answers on wiki, as they have not been vetted to remove potentially personally identifiable information.

Here are some random, contiguous sample rows from those answers:
 * bug:
 * other problem:
 * positive feedback:
 * suggestion for improvement:

Project-level rollout discussions
Here is where the potential rollout of hovercards is being discussed:

English Wikipedia: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(proposals)#Proposal:_Enable_Hovercards_by_default