Article feedback/Call to action

This page presents a high-level summary of the latest findings on the effects of call to actions (CTAs) and discusses recommendations for future design.

Current CTAs
As of AFT v.4 three different CTAs have been implemented and analyzed: Edit, Join and Survey.



Edit this page
The Edit CTA invites users to edit an article after rating it. The CTA is non-conditional, i.e. it doesn't depend on the actual rating values submitted by the user. The analysis of this CTA across different AFT versions suggests that on average 1 out of 6 users who see this CTA accepts the invitation (click-through rate: 15.1% (v.3) - 17.4% (v.4)). This makes Edit CTAs the second by rank after the Survey CTA in terms of click-through rates. If we consider the average completion rate, 15.3% of users who accepted the invitation successfully completed an edit, which corresponds to a net conversion rate of 2.7% of all users who saw the CTA (v.4 data). These rates show that a relatively high number of users are potentially interested in contributing but only a small fraction actually do so.

Discussion. Given (1) the current positioning of the AFT at the bottom of the page, (2) the fact that it's not actually loaded until users scroll to the bottom of the page and (3) the fact that we know that rating conversions (i.e. the number of people rating an article after reading it) decay very rapidly with the page length, it's highly unlikely that rating may be distracting people from editing the article (people interested in editing would have done so using the main edit tab or section edit links) even if we currently do not have data to test this hypothesis. If this hypothesis holds, though, we can reasonably assume that edits produced via the Edit CTA, however small in volume, are additional to regular edits. A second issue that needs to be considered next to the edit volume is the quality of edits generated by the CTA. We currently do not have data to measure how good these edits are compared to regular edits and in particular if they have a higher or lower chance of being reverted.

Create an account
The Join CTA invites people to log in or to sign up after rating an article. The CTA is non-conditional, i.e. it doesn't depend on the actual rating values submitted by the user. The click-through rate for Join AFT is the lowest among the CTA we implemented, with 4.7% of users on average accepting to sign up and 3.5% of users accepting to log in (v.4 data). Completion rates are not available for this CTA. The click-through rates by themselves show that a small number of people are interested in creating an account as a result of rating an article.

Discussion. The low click-through rates confirm that people do not feel a compelling reason to sign up after rating an article. This is likely to be the case because of how decontextualized this CTA is with respect to the action of rating an article. It's also plausible to assume that part of the log in clicks may be target errors, so the actual CTR for signup is between 3.5% and 8.2%.

Take a survey
The Survey CTA invites people to take a short survey after rating an article. The survey is both related to the article and to the feedback tool. The CTA is non-conditional, i.e. it doesn't depend on the actual rating values submitted by the user. The click-through rate for the Survey CTA is the highest among the CTA we implemented, with more than 1 out of 3 users (37.2% (v.4) - 40.6% (v.3)) on average accepting to take the survey. Completion rates are also the highest we obtained among all CTAs with 64.8% (v.4) to 66.3% (v.3) of all users who clicked on the CTA successfully completing and submitting the survey: this corresponds to a net conversion rate of 24.1% of all users who saw the CTA (v.4 data).

Discussion. The high completion rate indicates that people who rate an article are on average happy to provide further feedback.