Topic on Talk:Reading/Web/Projects/Mobile Page Issues

Jump to navigation Jump to search
This is a permanent link to a single version of this post. This version is from 3 months ago. You can see the differences from the previous version, or view other versions on the post history page.

@WhatamIdoing As noted in the comments you linked to, those were not yet the final results for the entire "four-week study", but preliminary data quickly queried before the end of the experiment, and we'll publish a report soon with the fully vetted results for the entire timespan, including an assessment which of the changes were statistically significant.

"It is not unusual with a UI change for everyone to click on it once or twice to figure out what it is, and then to ignore it afterwards" - yes, that's called a novelty effect, and we anticipated that to occur (see e.g. the task description of T200792). However, we also usually assume that for such a fairly simple feature, they don't last longer than a day or two (for the individual user). This is supported by results from the page previews A/B tests. Still, this is a good point and I'll be plotting the time series of the clickthrough rates for the 4+ weeks we have, to see if they show a decay over more than just the initial few days.

BTW, are the details of your Facebook pageviews analysis published somewhere? I'm curious how you controlled for other factors and trends that may have influenced the traffic to those pages in October 2017 and October 2018.

Retrieved from "https://www.mediawiki.org/wiki/Topic:Ughiujdu59it47bc"