Reading/Readers contributions via Android/Outcome

From mediawiki.org

Intro[edit]

From December 2016 through February 2017, we held a consultation on MediaWiki.org.

The purpose of the consultation (the third of this nature) was to identify how the community felt about readers (low-context users) contributing to Wikipedia via mobile devices.  We had some examples to share to start the conversation, but the hope was that members of the community would identify workflows, backlogs, or missing content that they felt someone with low context and a mobile device could reasonably perform.

The prompt was here: Reading/Readers_contributions_via_Android

And the discussion took place here: Talk:Reading/Readers_contributions_via_Android

Between the outcome of this discussion and the results of launching wikidata description editing on the Wikipedia Android app, the hope is that we will have enough knowledge to successfully find ways to help more people contribute to our movement.

Next Steps[edit]

After a period of at least 14 days during which we hope members of the community will participate in reviewing and commenting on this, particularly the Take Homes below, we will put out some choices for how we move on from this point. The proposal will be derived from a combination of the finalized results of this consultation, the results from the editing Wikidata description experiment currently being rolled out on Android (Wikimedia Apps/Short descriptions, RFC) and what best lines up with organizational imperatives (such as resourcing, what the Editing team is working on, the current state of the Android app, etc). Current thinking is that the proposal might be a set of alternatives. Unless there is strong feedback at this stage or changes to the other factors listed above, the current thinking is along the lines of: adding existing images to an article, more Wikidata description editing features, or confirming data from an article (for Wikidata).

Please review this and let us know if we're missing anything crucial here. This review ends on March 31st. There will be another chance to chime in at the proposal phase, but the earlier we get feedback about the basic premises the better.

Take-homes[edit]

  1. While some users advocated a complete overhaul of moderation mechanisms, the most pragmatic thing to do is require any contribution to show up on recent changes to re-use existing moderation schemes - recentchanges, or logs and desktop viewable -- in addition to any other place it shows
    1. Step 2 is to allow readers to add signal, by upvoting or downvoting…
    2. Step 3 is to allow certain reader/generated scores to count more
  2. Especially initially, we do not want to target “all” readers, but somehow find the readers who are most likely to provide valuable content.  If we assume a reader knows nothing, then we have very limited options.  We should find ways to target those users without making the UX the barrier.

Definitions for diagram:

Moderator: someone who actively participates in the governance of Wikipedia (admins, arb com, checkuser, stewards, new page patrollers, anti-vandalism, certain bot creators, chapter management)

Editors: “Encyclopedia writers”.  There are obvious subsets here, but these are people who can and want to write encyclopedic content.  

Contributor: This is a broader class of user, the class being targeted here.  Interested Wikipedia users that can contribute in some way to the movement without having the time, formal education, or intellectual rigor to write long-form articles.  Contributors might eventually become editors or moderators, but the hope is to find work that is meaningful in and of itself.

  1. Ideas ready for next features:
    1. Adding images to article via wikidata or simple commons search
    2. Wikigrok
    3. Thank an editor
    4. Ideas not mentioned specifically by contributors but fitting within the concerns mentioned:
      1. Cropping/centering image for “lead image”
      2. Selecting lead image
      3. Wikidata description translation
  2. Needing additional work/time:
    1. Tag an image (probably dependent on structured data project + possibly image recognition libraries)
    2. Upload a new image (probably dependent on ^)
    3. Research items from
      1. w:Template:Welcome training
      2. (WereSpielChequers)
      3. http://www.wikihow.com/Special:CommunityDashboard (Bluerasberry)
  3. Ideas that are not well liked - forms of article feedback, such as fixing typos or voting

Selected Points[edit]

Note: in most cases we quote the username of the person making the point, but there were many comments by a very close-set of IPs, that we are presuming is the same user. This user is referred to as IP.

Comments on the premise[edit]

Who are we targeting[edit]

(https://www.mediawiki.org/wiki/Topic:Tjdf0pdg8uzha02b)

Questions the originally stated theory behind this. Editing relies on self-motivated, educated people contributing.  If we opening it up to “everyone” we risk making assumptions like “the user doesn’t know anything” that could significantly limit the value of their contributions, making them not worth the effort of moderation by editors.  Also, we need to be careful that we don’t create tracks that don’t progress and artificially steer would-be contributors into dead-end paths.  

Discussion of identifying a user set between “editors” and “Everyone else”, that is 5x larger than editors but 10x smaller than everyone else.   The notion of educating users upfront or having test questions so that we can then make certain assumptions about users intent, dedication, etc.

A commitment to exploring bringing interested people up the ladder of engagement, so that these tasks aren’t dead ends.  Alsee remains skeptical.

The importance of communication/discussion for anything real and some ideas around how to simplify the talk-page experience on apps.

Hit a button to reply, auto-signature, auto shorten the page...etc. Building on the work of “add a new section” button.

(Alsee, https://www.mediawiki.org/w/index.php?title=Topic:Tjdf0pdg8uzha02b&topic_showPostId=tjfv96dnxc3p83wd#flow-post-tjfv96dnxc3p83wd)

Liked ideas[edit]

Wikigrok/wikidata[edit]

People like the wikigrok idea, in other words, allowing users to structure data that is currently on the page via multiple choice for inclusion in wikidata.  See prototype.  

(Strainu, MKramer, bluerasberry, IP, https://www.mediawiki.org/wiki/Topic:Thn5ve4o9t6ey3xg, https://www.mediawiki.org/wiki/Topic:Tjfkrtwjtvxivod1)

Adding new images to articles[edit]

People love nearby images idea.  But see concerns below

  • IP, (https://www.mediawiki.org/w/index.php?title=Topic:Thn52ix6tw8epw64&topic_showPostId=thr2rxpnsf9lxof7#flow-post-thr2rxpnsf9lxof7)
  • Alsee
  • WereSpielChequers
  • Jim.Henderson
    • also if you happen to open an article of somewhere you are near...JK note: seems unlikely

Add existing images to articles[edit]

Add images from commons or other public domain repositories (IP, https://www.mediawiki.org/w/index.php?title=Topic:Thn52ix6tw8epw64&topic_showPostId=thr2rxpnsf9lxof7#flow-post-thr2rxpnsf9lxof7)

wikidata p18

(IP, https://www.mediawiki.org/wiki/Topic:Tjoagr7r6xsrw5xc, similar to: https://phabricator.wikimedia.org/T95026)

Concerns for new image uploads[edit]

From Commons, user: from: https://commons.wikimedia.org/wiki/User:LX

What have you learned from?:

Commons:Village pump/Archive/2013/04#Mobile Web Uploads turned off in stable

Commons:Village pump/Archive/2013/04#Missing author/source parameters on mobile uploads: fix coming

Commons:Village pump/Proposals/Archive/2016/08#Rfc: Should we request a configuration change to shut down cross-wiki uploads?

Commons:Mobile access/Mobile upload needing check#Background

Category:MobileUpload-related deletion requests

Concern about copyright for images:

(Strainu, https://www.mediawiki.org/w/index.php?title=Topic:Thn52ix6tw8epw64&topic_showPostId=thn52ix6u06gy04c#flow-post-thn52ix6u06gy04c)

“Thank” Editors:[edit]

Like!

(Bethnaught, https://www.mediawiki.org/w/index.php?title=Topic:Thr12asoxevtrd32&topic_showPostId=tjaektwmw5g0vlm4#flow-post-tjaektwmw5g0vlm4)

Promising new ideas[edit]

Image tagging[edit]

Have readers rate images for usefulness or  - is image a selfie or is image a bridge, (https://www.mediawiki.org/wiki/Topic:Thr2mevmmnsqm8mk)

Run a machine learning algorithm to identify what is in an image then let users confirm or deny (note:  the closed form nature of the question would limit vandalism)

Existing machine learning:

  • https://catchoom.com/documentation/on-device-image-recognition-sdk/android-on-device-image-recognition-sdk/
  • https://github.com/tzutalin/Android-Object-Detection

(IP, https://www.mediawiki.org/w/index.php?title=Topic:Tk9zjrabaddks2pv&topic_showPostId=tk9zjracujs98ber#flow-post-tk9zjracujs98ber)

Or via similarity analysis

Something that would benefit both commons, and editors of every wiki is some form of similarity analysis (perceptual hashing) for media:

  • Images - http://en.cnki.com.cn/Article_en/CJFDTOTAL-DZXU200807028.htm
  • Video - https://github.com/rednoah/VASH , http://iosrjournals.org/iosr-jce/papers/Vol18-issue6/Version-5/P1806058486.pdf
  • Audio - http://link.springer.com/article/10.1155/ASP.2005.1780

While most of these would require some research to evaluate and integrate it into an app or even commons. Perceptual hashing for images is already supported by the backend software that wikimedia currently uses:

  • Imagemagick - http://www.imagemagick.org/discourse-server/viewtopic.php?t=24906

(https://www.mediawiki.org/wiki/Topic:Tkenxcprkclztpie)

Add Links[edit]

Work on dead-end articles[edit]

Work on disambiguations[edit]

If a link on a given article points to a stub, ask readers to choose between the different homonym links when they are reading. The example given is "the trick is to have something in the app that lets people choose whether to link to Dallas, Texas or Dallas, Scotland." (added by Trizek, post consultation)

(https://www.mediawiki.org/wiki/Topic:Tjesvfbv9nfzeemq)

Add headers[edit]

Quote from WereSpielChequers

“One task that would be useful to promote to readers is adding Headings. This could be targeted at articles with more than a few paragraphs of content but without sections. It is one of the newbie exercises in EN:Template:Welcome_training, and it is a classic entry level task - no referencing required! Hopefully an android App could prompt people to add some of the obvious headings, on English that would include See Also, External Links and References.

As well as an uncontentious and useful intro to editing, putting articles into sections reduces the risk of edit conflicts - one of our biggest problems for new editors.”

(https://www.mediawiki.org/wiki/Topic:Tjet5y03aa0kn89d)

Other sources of inspiration to look at:[edit]

https://en.wikipedia.org/wiki/Template:Welcome_training (WereSpielChequers)

http://www.wikihow.com/Special:CommunityDashboard (bluerasberry)

Moderation[edit]

Readers moderating readers[edit]

“Great idea. I'd say that this is the most important concept.” (IP, https://www.mediawiki.org/w/index.php?title=Topic:Thn5jofh98tul45p&topic_showPostId=thr496qlra7wqzo5#flow-post-thr496qlra7wqzo5)

But wary of bias (IP)

Need for desktop wiki community oversight[edit]

There is a well-established intolerance for any reader seeing anything that does not also appear in desktop recent changes (or at least logs). See Riskers list: https://en.wikipedia.org/wiki/en:User:Risker/Risker's%20checklist%20for%20content-creation%20extensions

Possibly queue new mobile images so they don’t show until editors on desktop have a chance to review, an open-ness to readers potentially up or downvoting the images which could alter how they appear in the moderation queue.. Then , if it works out, we could start using the reader moderation by itself

(BethNaught, https://www.mediawiki.org/w/index.php?title=Topic:Tj907bb5jf6kbdwf&topic_showPostId=tjad3cuhe0tc4ic0#flow-post-tjad3cuhe0tc4ic0)

Audio- somebody in community would have to care about this

(BethNaught, https://www.mediawiki.org/w/index.php?title=Topic:Tj907bb5jf6kbdwf&topic_showPostId=tj907bb73jm7nkm7#flow-post-tj907bb73jm7nkm7)

Build a new, generic moderation feed[edit]

TheDJ:

  • Please built a generic list system, where you can have queues of tasks, where people mostly just get to click: Yes/No, or small input fields. Then use multiple passes to do first line validation of the tasks (and to build a hidden 'trust score of the user' (yes, this 'duplicates' the workload)). Make the highest confirmed changes automatically or whatever, dump the rest in the next queue (editor review). Not everything has to create an edit (statistics can also be a result).
  • Facebook places editor, and translations, as well as Foursquare, wikiHow etc have this down. It works.
  • Keep the interaction simple, use multiple passes of the data, have multiple trust levels. Combine Magnus Manske's his Autolist, Mix 'n Match, and ORES and you are basically there.

(https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tjdmr7yorc2rhub8#flow-post-tjdmr7yorc2rhub8)

Use a common backend system for moderation so that it can be applied to any contribution type

(IP, https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tja8o8canxsldbjx#flow-post-tja8o8canxsldbjx)

IP rephrased this as a ticketing system:

Replace complex template system for moderation with ticket system like the one proposed here: https://www.mediawiki.org/wiki/Requests_for_comment/Tickets

https://www.mediawiki.org/wiki/Topic:Thr2mevmmnsqm8mk

https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tja8o8canxsldbjx#flow-post-tja8o8canxsldbjx

While DJ does not want this targeted at n00bs or apps, IP disagreed.

Miscellaneous tips[edit]

Onboard users  with examples

(IP, https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tja8o8canxsldbjx#flow-post-tja8o8canxsldbjx)

Confirm age before review of image (https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tja8o8canxsldbjx#flow-post-tja8o8canxsldbjx)

Gate process on login, captcha or review of other edits

(https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tja8o8canxsldbjx#flow-post-tja8o8canxsldbjx)

Image moderation ideas[edit]

Machine learning could also help with moderation.

Use existing image libraries to block selfies (IP, https://www.mediawiki.org/w/index.php?title=Topic:Tk9zjrabaddks2pv&topic_showPostId=tk9zjracujs98ber#flow-post-tk9zjracujs98ber)

Use images as captchas for moderation (have to get known one right in order to continue (https://www.mediawiki.org/wiki/Topic:Thr2mevmmnsqm8mk)

Get wikigrok answers and then use those as captchas….

----

Ways to moderate image uploads, quoted from Nick (a commons editor):

We think allowing a user to take photographs and to then obtain information about the photographs they select to upload/submit to Commons would be more helpful than having users wade through pages of copyright, freedom of panorama, personality rights and trademark data before they can take photographs/videos.

We thought after images have been taken, getting some basic information from the user would then be easiest - so the user selects if the photograph they've taken a picture/video of is of a building, a statue, if there are other people clearly visible etc. This will aide correct categorisation as much as helping us screen for copyright issues.

The app then does some automatic categorisation for 'us' on Commons based on what simple choices the user has selected - for example if they take a picture of Cloud Gate ('The Bean') in Chicago, during upload they tap that the photo is of a 3D artwork (or sculpture etc) and it's automatically categorised into a holding category such as Category:Possible non free 3D artwork images (USA) whilst similar works from the UK would go into Category:Possible free 3D artwork images (UK) or something similar.

I'm sure Commons would be quite happy hosting a protected page with settings for the maintenance categories and relevant data on freedom of panorama etc, which the app could use during the upload process.

Images are really easy to moderate relative to articles or video, but commons is already backed up!

(https://www.mediawiki.org/wiki/Topic:Tj90l4tsn0lsx5ml)

---

IP Pasted this table to illustrate difficulty of moderating videos and articles compared to images

Content Type number Average Size Type of work time taken
Articles 20 300 words Proofreading 400 minutes
Video 20 120 minutes Check for usefulness > 2000 minutes
Articles 20 300 words Copyright checking > 600 minutes (less with automation)
Video 20 120 minutes Copyright checking > 2500 minutes
Images 20 500KB Check for usefulness < 30 minutes
Images 20 500KB Check for copyright > 60 minutes

Article feedback[edit]

Overall: "What have you learned since last time? How will you avoid making the same mistakes again"

Structured feedback[edit]

So one of the best ways to design such kind of software is to force people to indicate a section (strictly limited range) of the document with the problem , and only allow structured annotation, e.g. spelling problem, grammar, inaccurate assertion (with reference url), bad lede, bad representative image, etc. (https://www.mediawiki.org/w/index.php?title=Topic:Thr12asoxevtrd32&topic_showPostId=thr4u8nqhgsg1neg#flow-post-thr4u8nqhgsg1neg)

Solving OTRS overflow with article feedback.[edit]

Much of OTRS is article feedback that we then can’t post on the talk page, due to email licensing issues. “To complement OTRS, there should be a suggestion queue which seems like an email form, but also includes an option for the user to cross post their comment to an article talk page.”

(Bluerasberry, https://www.mediawiki.org/wiki/Topic:Tjfkh94d9o9h9eo2)

Report an issue[edit]

It might also be a good idea to review how Extension:UIFeedback works (IP)

Concerns about article feedback or report an issue:[edit]

  • Garbage
  • Shouldn’t require a sign-in
  • Diverts people from being useful (Spielcheckers)
  • Don’t include a screenshot! (bethnaught)
  • More specific is better
  • More vague is better! (NaBUru38)
  • Downvoting/wars based on topic
  • Post-Article Feedback v5 trauma with communities
  • From Trizek (WMF) post consultation:  Other feedback we hear is concern over who will moderate the queue

Advice[edit]

Allow annotations for specific parts of articles

(IP, https://www.mediawiki.org/w/index.php?title=Topic:Tja8o8cantuj57lp&topic_showPostId=tjdn0kqyapajk354#flow-post-tjdn0kqyapajk354)

Miscellaneous concerns[edit]

Strainu’s concern that contributors would incur costly non-Wikipedia zero charges without realizing

Also that Wikidata Query Service wasn’t going to scale

Think big and go outside existing moderation structures - get away from templates (https://www.mediawiki.org/wiki/Topic:Thr2mevmmnsqm8mk)

Contributors[edit]

WereSpielChequers

TheDJ

BethNaught

Bluerasberry

Alsee

NaBUru38

IP 197.218.XX.XXX

Sänger

LX

Steinsplitter

Strainu

Jim.henderson

とある白い猫

Nick

Trizek (WMF)

RHo (WMF)

MKramer (WMF)

Tbayer (WMF)

Melamrawy (WMF)