Reading/Web/Projects/Improve in-article language switching on mobile web

The problem
This is intended to improve the reading experience of users who speak more than 1 language, primarily for those users whose primary language has limited content. It is quite common for such a user to want to see if the article in another language has more content or is more understandable.

See also: Interlanguage links/Implementation comparison.

Goal Visibility
This is a Readership Goal that is committed externally here: https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q3_Goals#Reading

Rationale
The current language-switching button is buried at the bottom of the article and still gets more clicks than the hamburger menu see mobile report card (table name is 'page-ui-daily').

We have also identified some usability issues with the overlay that appears when the language switcher is opened. It is this latter UX issue that we have started with.

User story 1 : As a user I want to see what other languages an article is available in without scrolling to the bottom of the article.

User story 2: As a user interested in switching the language of the article, I would like to find the language I am interested in as quickly as possible

Success criteria
Our initial goal was: Increase interaction with language switching by 5% on mobile web.

However, simply raising the prominence of the tool could do this simply by showing the tool to so many more users who might click out of curiosity. The revised goal is to "increase language switching (clicking on button and then selecting a language) by 5% on mobile web,without dropping the conversion rate (# of language switches/ #clicks on tool) to below 40%. "

Logic: This should filter out the impacts of simply raising the profile of the tool, based on the assumption that uninterested users will not actually select a language and switch wikis. We want to protects against simply driving more people into the tool and artificially creating higher engagement at the expense of bothering all the people who click, but are not interested. We do expect that more people will open the tool and then exit out. This will lower the 'conversion rate' (# of language switches/ #clicks on tool), but if it stays above 40% AND raises the overall number of language switches, we feel that this tradeoff is appropriate. The current conversion rate is ~60%, so this will mean that we are willing to accept 50% higher rate of people clicking on the tool and not using it, in exchange for 5% higher rate of language switches. This sounds like a bad tradeoff in %'s, but the cost of an extra click, is much lower than the benefit of discovering and reading a version of the article that is better suited to the reader.

Overall feedback
This task is being tracked on Phabricator and we'd love to hear your thoughts. The voting here is a good exercise to help us discuss and understand about the different points of view. By the end of the voting period, we should summarize each of the points raised about each proposal and move on from there, meaning, this is a qualitative, rather than quantitative analysis.

[historical] Help us choose the solution
'''The feedback period has CLOSED! Thanks for your feedback!''' We will evaluate all the points made, consolidate them and link to the final design when we have one.

We are making strides to get community feedback earlier and earlier in the process. We are excited to be asking for your advice at the solution stage, before we start building, and are really excited to see how this works.
 * Duration: For productivity, we have set a timeframe for feedback collection, start as soon as the designs are up (no later than Tuesday February, 23rd 1:00 UTC) and will end Saturday, February 27th 1:00 UTC).

The user story we are trying to solve for is:
 * User Story 1 from above: As a user I want to see what other languages an article is available in without scrolling to the bottom of the article.

Below are 3 possible solutions developed by our designers are going to be shown. Under each one, please vote with an optional explanation of why you prefer or dislike that treatment over the others. Voting options:

Description:
Sticky footers do a great job at increasing the access to actions. We can create an action bar for articles. in this case, changing language and watchlist seems to be good candidates for "doing something no matter where you are in the article"


 * Live prototype:. This can help you check the feature in a live demo.

Known Pros: Known Cons:
 * Access to changing language no matter where you are
 * Add articles to watchlist while you are at the bottom of the article, no need to scroll all the way up
 * Creates a space for future article actions
 * Already tested on native apps
 * Mobile browsers who don't support fixed position or poorly support will not have this feature
 * Sticky footer will take up vertical space on small screens
 * Time to implement is more

Solution 2: Top of the article
Description: We would also like to align article actions into a concise toolbar under the title of the article. These are actions that can be taken on an article.

Three actions
 * Change language
 * Add to watchlist
 * Edit the article

Known Pros: Known Cons:
 * Easy to implement
 * Provides easy access to changing language without scrolling
 * Aligns article actions in a better way than what we have
 * User has to scroll to the top to change the language in the middle of the article
 * Might be coming in the way of casual, monolingual reader [#] (because it pushes start of the article further down the screen??)

Solution 3: Top of the article with different treatment
Description: This is same as Solution 2 but [#] to tackle the one of the Cons from Solution 2, we can try a different treatment.

Final Design Based on Feedback
Thanks to everyone who gave feedback on the initial treatments for improved language switching on mobile web.

We reviewed the feedback given in the above section and synthesized it. The outcome was that it seemed the top of the article without the gray background was preferred. In addition, it was pointed out that users already know what language they are reading and writing out the full language name would take up valuable space on every single page. It was also pointed out that most users are not familiar with ISO language abbreviations, so we decided to stick with the button. After all, it is no more confusing than "star = watchlist".

Here is the final design, pending any unforeseen circumstances or challenges:

Any updates will made here: https://phabricator.wikimedia.org/T128350

Pre-implementation analysis
Refer to the following for some pre-implementation analysis.

T125127 [Release] Confirm language overlay sampling on production working as expected

On 12-February-2016 before changes to the language switcher modal or introduction of a new button placement, the data on the stable channel suggested that globally the "Read in another language" button was tapped about 1.3% of the time when there was an impression of it. This was heavily influenced by enwiki traffic, where the preponderance of traffic existed at the time. A few examples where the button impression sample size exceeded the magical 30 number: On arwiki tapthrough was about 2.5%, hewiki it was about 3.1%, jawiki 0.5%, and eswiki about 0.8%. This was based on the following queries.

Analysis on one week with new overlay in stable channel
The new overlay has been released into the stable channel on the mobile web. A comparison of one week of data with the new overlay fully rolled out versus the previous overlay (looking at a week window two weeks prior) is promising.

New overlay base data
The following query helps to produce "Read in another language" button usage, which is as of 8-April-2016 what's in the stable channel for mobile web users (this is the existing old approach, a new affordance is being tested in beta at this time).

The following produces tapthrough events from within the overlay.

New overlay interaction
The following produces a count of "Read in another language" events tied to the new overlay using the data from the first query's output.

The following produces a count of language selections from within the new overlay using the data from the second query's output.

In other words, there was about a 66.7% language selection rate with the new version based on a week of data.

The following represents the number of dismissals.

The dismissal rate with the new overlay was about 4.26%.

Old overlay base data
Here are the queries for looking at the language overlay with the old modal two weeks prior (one week prior should not be used, ironically, due to some translation issues associated with a late breaking code revert; these issues did not impact the new overlay, incidentally).

"Read in another language" button events:

Tapthrough events from within the old overlay:

Old overlay interaction
The following produces a deduplicated count of "Read in another language" events tied to the old overlay using the first query's output.

Notice the number is lower than for the corresponding count of initiations of the new overlay. That's because there was an A/B test of the new versus the old going on at this time. As a side note, we decided based on the A/B testing at that time to update our language sorting mechanism in the new overlay in part to be similar to the old overlay, while keeping the new look and feel and keeping a keyboard language heuristic we implemented in the new overlay.

And here's the tapthrough count.

In other words, there was about a 65.8% language selection rate with the old version based on a week of data.

The following represents the number of dismissals.

The dismissal rate with the old overlay for this observation period was about 5.81%.

Analysis
One feature of the new language overlay is use of a keyboard language detection heuristic that floats the user's keyboard language option to the top of the screen, with the hope that users can scroll less and can also see the translated term for the title of the page at a glance. Although the latter case of term lookup should tend to reduce the tapthrough rate, the tapthrough rate appeared to hold up relatively well with the new overlay, in this case outperforming the old overlay for the observed windows of comparison. Additionally, dismissal rates were relatively lower with the new overlay compared to the old overlay for the observed windows. We anticipate fluctuation in these numbers as with any natural system, and a period of adjustment to the new overlay (e.g., for users who have cleared their browser history and are effectively first time users of the new overlay).

As hypothesized, first time usage of the overlay suggested that keyboard language detection, although imperfect (sometimes there is no corresponding translated title, sometimes the keyboard language is a poor indicator of the user's preferred language, and so on), appeared to improve the user experience for users who tapped through to select an alternative language.

The average position for the new overlay was about 6.90 whereas the average position for the old overlay was about 8.88, representing about 22.4% less scrolling. As a point of reference at the 80th percentile the language tap position with the new overlay was 8 whereas it was 12 with the old overlay.

Given these encouraging signals, the next step should be, pending user research analysis, to roll out the updated language switcher affordance, which presently is a new icon in the beta channel at the top of articles (as opposed to a text labeled button at the bottom of articles).