Wikimedia Apps/Short descriptions/Research

From MediaWiki.org
Jump to: navigation, search

To understand how users of the Android app used this feature to contribute, and for an initial assessment of its effect on Wikidata content and on the workload of recent changes patrollers on Wikidata, we analyzed data from two sources: The (publicly available) edit history on Wikidata, and an EventLogging instrumentation (Schema:MobileWikiAppEdit). Below are the results from the former.

Wikidata description edits from the Android app.png
Wikidata description edits from the Android app, total vs. reverted.png

The feature has been available in the alpha version of the app since late 2016 (and there were earlier test implementations, also on iOS). On February 10, 2017, it was enabled in the beta version on the Russian, Hebrew and Catalan Wikipedia, and on February 28 rolled out in the production release, still restricted to these three languages.

From the time of the language-limited beta rollout on February 10 until April 12, 5891 description edits were made using the app, by between 20 and 57 distinct users (including IPs) each day. For the three languages enabled in production, edits from the app already made up a substantial amount of all manual description edits (i.e. edits from the standard web interface, measured by excluding edits from easily detectable bots and edits made via external tools such as Quickstatements or reCH): 28.9% in Russian, 60.8% in Hebrew, 28.1% in Catalan (referring to the time from March 1 to April 12). In late April, the feature was activated in all but the ten most popular languages. By June 21, 20193 description edits had been made using the feature since the beta rollout.

4.6% of the app edits made between February 10 and April 12 were reverted, which is higher than the rate for manual Wikidata description edits in general (e.g. 1.2% in the week from February 1 to February 7 2017,). On the other hand, this revert rate is considerably lower than e.g. on the English Wikipedia (8.1% for all edits including bots, and 29.0% for anonymous edits, per data from November 2015). Here is a language-specific breakdown for the three languages that were enabled in the production version:

language Total edits Feb 10-April 12 % reverted cf. % rev on WP (all incl. bots) cf. %rev on WP (anon)
Russian 2854 7.7% 4.9% 14.9%
Hebrew 1963 0.7% 5.9% 29.2%
Catalan 616 3.7% 2.0% 19.5%

Some notes and caveats:

  • As always, it is important to note that not all reverted edits are vandalism.
  • Conversely (like for all Wikidata edits), not all vandalism may have been detected and reverted
  • Members of the Reading team (in particular Dmitry and Tilman) were monitoring the edits from the app frequently, to make sure the feature was not becoming too disruptive, and on that occasion reverted many vandalism edits themselves (in a volunteer capacity). This drove up the revert rate compared to the baseline of normal description edits that did not receive such extra scrutiny. Specifically, among the 4.6% of reverted edits, 1.4% were reverted by members of the Reading team.
  • The Wikipedia revert rates are useful for an informal comparison, which however needs be taken with various grains of salt (e.g.: the cited total revert rate for WP do not exclude bots and are thus likely lower, they cover all namespaces instead of just a particular content type, etc.).
  • Data sources, calculations and further details are documented in two public PAWS notebooks: 1(currently slightly incomplete due to technical issues), 2.