Extension:ExternalGuidance/Impact

Automatically translated versions of wiki pages can be accessed [https://www.blog.google/around-the-globe/google-asia/google-for-indonesia2018/ from Google search results] and other translation services. The External Guidance feature customizes the wiki pages delivered to these translation services to make sure the content is labelled as an automatic translation, and it includes a way for users to contribute.

We want to make sure that External Guidance generates new contributions that are sustainable for the editor community. In order to do so, we’re providing mechanisms for tagging and measuring the editing activity generated. In this way, the editor community can know about and control the contributions that start from an automatic translation of a page. More detail about these mechanisms is below.

Tagging contributions
Contributions that start from a translated page will be marked with an edit tag named “external-machine-translation” when they are published. This allows editors to use their usual tools for quality control. Editors can focus on this particular group of contributions by filtering them in the Recent Changes page. For example, you can access [https://id.wikipedia.org/w/index.php?hidebots=1&hidecategorization=1&hideWikibase=1&tagfilter=campaign-external-machine-translation&limit=500&days=30&title=Istimewa:Perubahan_terbaru&damaging__likelybad_color=c4&damaging__verylikelybad_color=c5&urlversion=2 the contributions recently produced for Indonesian Wikipedia].

In case of an unexpected spike of problematic contributions, a community can use edit filters  (also known as “abuse filters”) to limit the creation of content that starts from translated pages. An edit filter can be defined to prevent the publication of contributions with the campaign-external-machine-translation tag. Filters can be set for particular type of users based on their user role, their number of edits, the time since account creation, etc.

Apart from the tags, contributions originating through External Guidance are treated as regular contributions, following the same rules regarding blocked users and protected pages. Thus, communities have their usual tools they are familiar with to deal with potential problematic users and contributions that may originate from externally translated pages.

Measuring the impact
External Guidance activity is captured in [https://analytics.wikimedia.org/datasets/external-automatic-translation/impact%20of%20external%20automatic%20translation%20services.html an analytics report] to give an overall picture of how much content is read and created from externally translated pages.

In particular these are the aspects measured:


 * Reading to contribution funnel. How many people are accessing translated pages, how many of them decide to contribute, and how many of those finally complete their contribution.
 * Content creation. How many edits and pages were created as a result of people coming from an externally translated page.
 * Content survival. How many contributions have been reverted or not, which gives an idea of the quality of those contributions.

These measurements can be compared to the same activity of regular wiki pages.