Talk:Growth/Personalized first day/Structured tasks/Add an image

Jump to navigation Jump to search

About this board

MMiller (WMF) (talkcontribs)

Hi everyone! Thank you all so much for participating in this discussion. We've learned a huge amount, and made some progress and decisions on this project that I'll summarize here. I hope some of you can review the materials I link to and react with any questions, concerns, or ideas in this thread.

  • Summary of notes: we summarized the notes from the discussions on this page and on other local-language wikis. I hope this is an accurate summary and not overly optimistic. Please let me know if you think otherwise, and I will correct it.
  • User tests: we completed 15 user tests with our prototype and put the notes and full results here.
  • Statistics: we posted statistics on the coverage of the algorithm (how many images it can recommend) and statistics on non-English metadata. These help us understand the viability and value of this project.

Taking all of our learnings together, our team is optimistic that a good "add a link" feature can be built. But you have helped us see how many potential pitfalls there will be. Therefore, here's what we're doing: the Android team is going to build an MVP (minimum viable product) of this workflow just for the Wikipedia Android app. It won't save any edits -- it will only record the judgments of the user as the try to do image matches. Users will know that their effort will only be used for us to learn, to improve the algorithm, and to improve our design. This is a very low-risk way to generate important data, and will be something that we all (including community members) can play with as we continue to figure out this idea. The Growth team won't be building anything until we learn from this simple Android version over the next couple months.

I'll continue to post updates as this progresses.

HLHJ (talkcontribs)

Would it make sense to, instead of an Android app, build a cross-platform Flatpak app that will also run on the new Gnux Linux phones? Technical details. It seems Google is wanting to shift to Fuchsia, and [informal poll of local Android users on how they feel about Android], so it might be a bit more future-proof. Sorry, this is a bit belated, maybe for next time. HLHJ (talk) 15:51, 6 March 2021 (UTC)

Reply to "Updates 2021-02-04"

Pinging community members

41
MMiller (WMF) (talkcontribs)

The Growth team has started thinking through a potential new project: "add an image". I'm pinging some community members who participated in a discussion this past May around the general idea of "structured tasks". That was a really productive discussion, and resulted in the Growth team deciding to build the "add a link" structured task workflow. As of now (December 2020), the Growth team is in the middle of engineering that feature, with plans to release in February. But we are also looking ahead to what might be other future structured tasks that would be more valuable to our wikis than simply adding wikilinks.

I've put up a project page around this "add an image" idea, writing out the thinking we've done so far. We know this is more ambitious, and with more pitfalls, than just adding wikilinks -- but we think there is potential here, to make use of the valuable media in Commons. That's why we definitely need to discuss with community members, who know more than we do about Commons and about illustrating Wikipedia. On the project page, you can see why we want to pursue this idea, and you see how we're thinking about the algorithm and trying to validate the idea with newcomers. There is an interactive prototype to try out, mostly geared toward showing the algorithm rather than the user experience.

The page also has a list of open questions, which is what I'm hoping community members can speak to. If you can weigh in, please add a comment or thread about any of the open questions -- or bring up your own open questions and ideas. Especially for community members who are active in non-English wikis, we want to hear how you think this idea might or might not work, given that most metadata about images in English.

And if you know of others who we should ask to weigh in, who have expertise that we need, please feel free to ping them! We'll be conducting similar conversations on Arabic, Czech, Vietnamese, and Bengali Wikipedia to gather perspectives from other communities.

Pinging @John Broughton @Sdkb @Zoozaz1@NickK @Nick Moyes @Galendalia @Barkeep49 @Pelagic @Czar @LittlePuppers @HLHJ

Czar (talkcontribs)

Catching up on pings and had some initial thoughts:

  • In case you haven't already, have you tried asking WP editor communities directly what common tasks would (1) benefit from being semi-automated, (2) be friendly for new editors? I would be surprised if adding images was one of the initial suggestions. I would personally rate adding existing images to be an intermediate+ activity since the easy images have already been attached to articles, leaving harder or unclear matches, and we wouldn't want new editors automatically adding the wrong image to an article, leading to one of the press's favorite rags on Wikipedia: the wrong image showing in Google's Knowledge Graph.
  • Relatedly, I would be curious if developing wikis would rather have newcomers add images or instead just set their infoboxes to import the default image from Wikidata. In my experience, I've seen a fair amount of the latter on smaller wikis and it is a more elegant solution than manually entering the file name in wikicode. Also relatedly, adding metadata to Wikidata à la meta:Mix'n'match could be interesting for newcomers... especially if it would automatically show up in multiple Wikipedias at once too.
  • "50% of articles have no images" Just want to point out that this is often by design. I wouldn't expect a new editor to understand that we're not looking for any related decorative element, only for visualizations that illustrate something we want the reader to understand about the topic itself.
  • "'To add an image' is a common response newcomers give on the welcome survey" I'd be curious if there is follow-up data on this. In my experience, new users generally are looking to add non-free images and don't yet realize that it's against WP ethos. I'd be surprised if they just wanted to add images in general.
  • re: the open questions,
    • "Will newcomers think this task is interesting?" I think I said something similar last time, but do new users want to associate random images? It requires a lot of cognitive load. I'd be curious if this was instead posed as a suggestion when the user browses to an article without an image. I.e., "we think you can contribute to this article—would you have time to review images?" That way they'd be entering having read the article, knowing the context, and would only need to review the image's metadata.
    • I tried your prototype and felt like I couldn't confidently associate images and articles by the single view alone. Most of the details could be hidden. I mainly needed to see the target article and the file name (not even the description). The categories should have been useful but they're hard to parse in paragraph form. Another major difficulty was translation. If the title or category is in another language, usually I'd translate it on desktop and play around with it to make sure my translation was correct—that would just add complexity in this prototype. My two takeaways: (1) from being a Commons admin, I don't really trust Commons metadata for these associations, (2) it doesn't feel like a good fit for the format. I want to see so much more context from both the article side and the image side before committing an image to the infobox. Perhaps smaller wikis might be less stringent but I'm not sure what they'd gain from doing so if it creates more cleanup/inaccuracy work for their cohort of active editors.
    • Adding images to a section can be just as rewarding as adding images to an infobox
    • If I were doing this as a new editor, I'd only want high-quality matches. I'd be frustrated if I had to think hard/compare multiple pages for each image. Or more likely, I'd probably move on to something else with less friction unless I'm committed to completing a task.
    • For judging quality, if you're able to detect whether an image has artefacting/pixelation or is low-res and exclude those, I wouldn't pass the quality test to a new user
    • I wouldn't trust new users to write captions in WP house style en masse. More of an intermediate user thing.
    • The question of accessibility for non-English readers is a really good one. I wonder how non-English editors currently fare on Commons for desktop web, even before it's reduced to a mobile view for someone with less experience.
    • re: bias, it's going to reflect Commons bias, the images that were most accessible for upload and the images that have the best metadata (i.e., maintained by a user or a GLAM) are more likely to be paired as ready for linking. I think that's beyond your scope, though it could be interesting to run your prototype on a single data set, e.g., a certain type of media (photos, paintings, etc.) released from a specific region as another alternative to random image suggestions
  • I went to follow along with the image trials but the Google Sheets do not appear to have public access, e.g.,

Okay—hope that's helpful! :)

MMiller (WMF) (talkcontribs)

Hi @Czar -- thank you for these thoughtful comments. It's really great to hear from a Commons admin. Here are my replies and follow-up questions:

In case you haven't already, have you tried asking WP editor communities...

Yes, this was something we talked about in the original discussion about structured tasks, particularly in this thread. A type of task that several communities mentioned as being particular desirable would be some sort of copyediting/spellchecking/grammar task. We talked a lot about how one of the challenges is to find spellchecking or grammar-checking engines in all languages. This is an idea that I'm still learning more about and figuring out what's possible.

In choosing which structured tasks to build, we're trying to take several things into account, including (a) whether the task is useful, (b) whether newcomers can do it successfully, and (c) whether we have a technical approach to building it. The technical approach part is what helped us decide to prioritize "add a link" and "add an image". For both of those, the WMF Research team already had ideas for algorithms that could get us there. But for a task like copyediting, the path is not as clear.

Relatedly, I would be curious if developing wikis would rather have newcomers add images or instead...

Thanks for bringing this up. We've heard about ideas and concerns around Wikidata infoboxes from @Mike Peel and others. It would be super helpful if you could check out the questions I posted for Mike, and add any of your own thoughts.

"50% of articles have no images"...

Good point -- we already saw in the user tests we've done so far that users are unclear on what makes an image appropriate for an article. They're not sure whether they are strictly judging on relevance, or also on quality, or on other things. We would need the feature to include a little bit of onboarding to give them guidelines around how to decide.

"Will newcomers think this task is interesting?"...

In terms of whether users will want to go through a feed of articles missing images, we think there are some users who will really like it. In the fifteen user tests we've run so far (we'll post the summarized results in the next couple weeks), several users have said things like, "This is fun" or "I could do this all day." And in building this task, we would also allow users to narrow the articles to their topics of interest, which we do for existing newcomer tasks, and we think that would make the work more engaging.

But the idea you're talking about is different. We call it the "entry point in reading experience": if someone is just reading Wikipedia articles, how about something tells them something like, "This article has 4 suggested edits you can make." Then they could discover the link suggestions or image suggestions, or any other kinds of tasks in the future. And presumably, they would be interested in the content, because they navigated to that article on their own interest. Yes, we think that would be a great thing to build, and we definitely want to in the future. This workflow actually exists in the Wikipedia Android app for the "article description" and "image caption" tasks. But we're starting out with the "feed" approach for two reasons: (a) this gives us more control over how many tasks we make available and which users can do them, and (b) it is technically less ambitious to build the feed because we only need to generate and store suggestions on a sufficient number of articles to populate the feed, rather than on all articles to prepare them to be discovered.

I tried your prototype and felt like I couldn't confidently associate...

Regarding the format, I definitely agree that the context of the article is really important to have. Although our prototype shows one image match at a time on one screen with a small amount of information, it may make more sense to adopt a design in which the user actually navigates to the full article in order to complete the image match. Our team calls this a "Concept A" design, as opposed to the "Concept B" design that you saw in the prototype. We're actually building "Concept A" now for "add a link" (in which the user goes to the full article to complete the task. To read more about Concept A vs. Concept B, see this section.

Could you tell me more what you mean when you say, "I don't really trust Commons metadata for these associations"? What goes wrong?

If I were doing this as a new editor, I'd only want high-quality matches.

I think this could be a good idea. Perhaps we could show only the highest confidence matches for a user's first tasks, so that they can quickly grasp the objective of the task and be productive. Then they could advance to lower confidence matches as they gain skills.

For judging quality, if you're able to detect whether an image has artefacting/pixelation...

The WMF researcher who works on the image algorithm does plan to add some computer vision elements in the future to filter out low quality images, or perhaps NSFW images.

I wouldn't trust new users to write captions...

I agree that writing captions might be the most challenging part of the task for newcomers. But I think, at least on English Wikipedia, that images in articles should almost always have captions, unless they are an obvious image in an infobox. Is that right? If so, I'm trying to think of how else the images could be appropriately captioned, if not by the newcomers who make the match.

I went to follow along with the image trials...

I don't think I can share the spreadsheets publicly, but the prototype you were playing with actually does contain 60 actual matches from the algorithm. I hope that can give you a sense of the sorts of matches that it makes. What do you think of them?

Czar (talkcontribs)

In reverse order

prototype ... contain 60 actual matches from the algorithm. ... What do you think of them?

I couldn't confidently associate many of those images and articles. I often couldn't ascertain exactly what connection the image had, whether its details were correct, and whether it fit in the article (before viewing it).

I think, at least on English Wikipedia, that images in articles should almost always have captions, unless they are an obvious image in an infobox. Is that right?

Ideally, yes, but not as a rule. Editors reviewing edits look at every edit and ask, "Is this an improvement?" Most editors would see an uncaptioned, relevant image as an improvement the same way that the most minor of typo fixes is an improvement. But there's nothing editors despise more than automated edits that do not have wide consensus as being an improvement (i.e., irrelevant images or irrelevant captions added en masse). As for how else they'll be captioned, at least on enwp, it's fine to leave that to another editor. :) If the tool was restricted to infobox images, captions would appropriately be completely optional.

Could you tell me more what you mean when you say, "I don't really trust Commons metadata for these associations"? What goes wrong?

(1) It's hard to tell the provenance of the image. Did a new user just upload this from Google Image Search and claim it as their own (fairly common problem with understanding copyright)? If it's public domain (PD), what was the evidence of prior publication? (If it just "looks old", it's reasonable that it was created recently to look that way. I had this question with the Josias image in the prototype—depends whence it came.) Copyright is difficult! With the possible exception that trusted uploaders and uploaders with photo EXIF data of what claims to be their own work is likely okay. (2) Categories (and even descriptions) tend to be added piecemeal over time. Is this the right Josias? Could this have been some other historical Josias with a similar name? Who added the description and when? Are the "facts" in the description right, or did someone just infer about the image what I'm about to infer myself? Usually I just need a lot more info to make a concerted decision if the image is appropriate. One category in the prototype is "Promptuarii Iconum Insigniorum"—do I need to know what that is before I decide its connection to the proposed article? Etc. (3) The prototype's fourth and fifth images have similar connection issues. This Apollo and Artemis plate thing is somehow connected to "Polemos" (a lesser known deity) but for me to know how, I'd have to visit the article that the app says is linked to see its context, because what if the Commons description just listed the wrong deities? And once I view its usage on the Lithuanian WP, it's used in a transcluded navigation template, not the article itself and now I'm down a rabbit hole far more complicated for mobile user than simply clicking yes/no/unsure. Similarly, on #5, knowing so little about whether this monument is representative of Aizpute Parish, there's no information on whether it represents the area so even if it is related, is it good for the page? Even as I do the same queue on desktop web (i.e., added context), image confidence tends not to be absolute and more like good enough, but on mobile without that context, doubt makes it even harder to take confident action on this queue.

With Concept A vs. Concept B, it's hard for me to visualize how that would work with images. In-line image placement within the article I can see with Concept A, potentially, but to compare an image's context, I would need to see more metadata, its placement within the recommended comparison, possibly refer between the two, etc.

A type of task that several communities mentioned as being particular desirable would be some sort of copyediting/spellchecking/grammar task.

I absolutely love this. Yes, yes, a thousand times yes. I would suggest, in addition to your a/b/c criteria, (d) the community wants or will tolerate it, because it could be a beautiful tool but if the community has no will to adopt it, they will just vote to ignore it, and unfortunately this happens with a number of well-meaning WMF-developed tools. Typo correction is the sweet spot of what new users already do as their first edits (I see new IP editors change typos as their first edits all the time), what is actually helpful (no one will say no to this, unless the new user just doesn't understand the difference between British and American English, but that's an edge case), and what suits the mobile context. I see how it misses the technical approach—your (c)—but something like typo correction with "entry point in reading experience" is :chefskiss: To most editors, I'd wager, the editorial value of a newcomer who fixes even just one typo while wikiwalking down a rabbit hole is far more valuable than an editor who adds 15 or more questionable images to articles, potentially creating more work for someone to track down and clean up.

Indulging my fantasy for one more minute, would spell check be necessary? MVP could just be asking users if they'd like to know how to correct typos, single-screen onboard them to double tap on the word, edit and click save, and all good! In my experience, newcomers don't necessarily want or need a queue—they just need an easy opportunity offered within context of something they're already doing. The editors we want (or at least one type) are those who read on the app, are bothered by a typo, and are given an easy recourse for doing something about it. Then it feels good, you get immediate reinforcement, might do it again, and have an entry point into more advanced edits. And hell, even if they fall out of the funnel, that one typo edit has a far better likelihood at helpfulness and stoking curiosity than pushing users to a queue, no? My take, at least :)

MMiller (WMF) (talkcontribs)

Hi @Czar -- thanks for getting back to me.

About the prototype: when I went through the 60 matches, I ended up considering about half of them to be "Yes" matches, with maybe another 10 being borderline (even without digging deeper into the article or Commons or anything.) I generally consider that rate of accuracy to be appropriate for a task like this. For example, some of the matches that I considered "Yes" were for these articles: "Alexander Nikolayevich Engelhardt", "Kollel", "Raymond J. Barry", "Original Vampires (The Vampire Diaries)", and "Gloria (given name)". I'm saying this because I want to make sure that I am on base here with my consideration of the strength of this algorithm, which is the lynchpin of this whole feature idea. You've already invested a bunch of time into helping us with this idea, so please feel free to say no -- but if you are interested and have time to go through the 60 images in the prototype, and mark down your judgments on them, it would be an invaluable data point for us to judge the accuracy of this algorithm.

About the Commons metadata: yours is a really interesting perspective of not trusting the Commons metadata. That makes sense, since you have deep involvement with Commons and you're aware of all that can go wrong with the metadata. Newcomers are probably not equipped to think critically about provenance. I was more thinking that if an image has survived in Commons for a while, and especially if it is used as a P18 or is used on another Wikipedia, then it is likely trustworthy, and therefore other users could just trust the metadata to decide what to do with the image. Sort of like how a scholar cites another's work and builds on it. I also think our design and the text in the feature would encourage the newcomer to be conservative, saying something like, "Only add images when you have high confidence -- when in doubt, skip!" What do you think of all this?

About Concept A vs. Concept B: we'll likely make a mockup of the Concept A version soon, and I'll post it and we can discuss.

About copyediting/spellchecking/grammar: I'm glad to hear you think this is a good idea. We actually talked to User:Beland about this, because he's built a bot to spellcheck all of English Wikipedia for Typo Team. The notes from that talk are here. For your idea of an MVP, what would initiate the workflow for the newcomer? When someone is reading an article, and they notice a typo, they already have the Edit button, but nothing explicitly invites them to tap it at that moment. I think that's where having some sort of spellchecker comes in -- it allows us to highlight or otherwise draw their attention to the typo with a call-to-action, making them realize they can join in. What do you think?

NickK (talkcontribs)

Sorry for the late reply, somehow missed the notification...

I would say that as a user mainly of non-English wikis, I would appreciate this task. In many cases it is not needed to speak any language to identify if the image is appropriate. It is a relatively easy task where a user needs more of a general knowledge than of Wikipedia experience.

I think there would be two major obstacles:

  • Images that are misidentified. For instance, someone uploaded a picture saying it depicts some particular species. An expert later came and corrected that is misidentified, but fixed only a part of description. That expert would be annoyed if this image appears in the article again.
  • Items which lack good quality images on Commons and where we have illustrations that really need context (e.g. a person on a group photo or a building on a large panorama). In the example of Nora Strømstad, it would be hard to understand whether Nora is on the left or on the right.

Other than these two (quite rare) cases the workflow should work pretty well.

To suggest articles we can look for some popular articles (pageviews, interwikis, incoming links) to avoid some obscure topics (and yes, this would likely mean bias, as an alternative we might at some point ask a user to suggest a region they want to work with). To select images, I think we need to define the concept of 'main image' (I would say infobox or top left / top right) and suggest images used in other wikis in this position, or images used on Wikidata. It is is very important to use this and not any other image: for instance, for a person an infobox image is most likely their portrait, but somewhere below in the text we can find something else, e.g. pictures of their works or of a house where they lived.

Overall I don't think it will be dull, some people would like it more than others (pretty much like not everyone enjoys copyediting) but I think it would be useful.

MMiller (WMF) (talkcontribs)

Thank for replying, @NickK; I'm glad to hear from you, especially with your perspective from a non-English Wikipedia.

About misidentified images: the point that metadata on Commons can be wrong has been brought up by some other community members. It sounds like the specific point you're bringing up is what if an image is removed from an article for a good reason, but then our feature suggests that a newcomer put it right back on? In that same vein, the problem might occur if we suggest an image in our feature, then a newcomer adds it, but then it gets reverted...and suggested again to some other newcomer. I think this is a very good point. Maybe there's a way for us to suppress suggestions like those by looking through the edit history.

About context: maybe our best way to address this is to make sure the users know they are supposed to be conservative in their evaluations, like, "If you aren't sure if this is the right image, or if it's unclear which part of the image is relevant to the article, do not push yes." Something like that?

I like your idea about asking the region the user wants to work in. I've heard from other communities as well, that they want to make sure users can focus their efforts on improving locally relevant content. Right now, the topics we use allow users to select regions like "Europe", "Africa", etc. They can be more specific like "Eastern Europe" and "Western Africa". But we don't have country-specific topics right now, like "Ukraine" or "Benin". Is that something that could make a big difference for your community?

NickK (talkcontribs)

Thanks for your feedback @Miller (WMF):!

Yes, my first point was on cases when good-faith editors keep adding a misidentified image to an article and experts keep removing it. Filter that the image was not removed sounds like a good idea, but please try to add a vandalism check.

I think there is actually a huge added value of a human review that humans can read. I would rather suggest adding a good caption if possible (maybe with a wording like 'if this image depicts two or more objects, use caption to identify which one illustrates the article').

On the last point, for example, I know user of Ukrainian Wikipedia who is an expert on Argentina. This is an unusual interest and I think we can use it. Not a high priority I think, 'South America' should also work in this case.

MMiller (WMF) (talkcontribs)

Thanks, @NickK. This all makes sense. What do you recommend we do for a "vandalism check"?

NickK (talkcontribs)

@MMiller (WMF): Sorry for the typo in your username (perhaps autocorrect on mobile). Perhaps checking whether an edit was reverted and/or if a user was blocked (either the one who added or the one who removed).

John Broughton (talkcontribs)

The ability to add "tens of thousands" of appropriate images to a language version of Wikipedia is certainly worth a fair amount of effort. And while this is only still in an early stage, you might think about a related structured task for editors who reach a certain point - for example, someone who has added 50 images. That add-on would be a structured task to add an image to the Wikimedia Commons. Because, obviously, at that point the editor knows the value of adding existing images; certainly there is a lot of motivation to take a photo, add it to Commons, and then add it to an article.

Other comments:

  • For logic, I don't know if you have data as to what order you follow to decide which of three methods to try first, to get an image. It seems to me that the third listed - "Look at the articles about the same topic in other language Wikipedias. Choose a lead image from those articles" - is the best of all.
  • It's not clear what happens if the user rejects an image as a match - do you keep offering images, or do you switch to another article? I hope that you give the user multiple tries - maybe three (four, at most), with different images.
  • If an image comes from an existing article, and has a caption, why not try a translation as a starting point?
  • Articles with no "File" wikilink have no images, yes?
  • If the image isn't going to go into an infobox, then it's probably best in the second section, rather than in the lead section. But if there is an infobox in the article (is this easy to determine), you could offer the user the option of putting the image there or elsewhere.
  • I don't really see the problem of "bias" in recommendations - I think you're going to run out of available images (per the logic you've set out) long before the issue will arise of the lack of images for topics unrelated to Europe and North America. And it's really the job of the community (and the WMF board) to address the lack of images related to topics in developing countries.
  • If the images are being fed by the algorithm in the workflow, there really is little opportunity to vandalism, if by that you mean adding inappropriate images. (I hope you tag these image additions; if so, it's easier to see if they are being reverted to any great extent.) If you're worried that a user may inappropriately reject a lot of images, you might put a limit on the number of negative responses that a user can do (in a day?); when that is hit, the user doesn't see this option any more.
MMiller (WMF) (talkcontribs)

Hi @John Broughton -- thank you for the quick reply! More than one person brought up your first point (helping users to upload their own images), so I'll make a separate thread for that. I'll also make a thread about what priority to give the different image sources (Wikidata, Commons category, other wikis). For your other comments:

It's not clear what happens if the user rejects an image as a match - do you keep offering images, or do you switch to another article? I hope that you give the user multiple tries - maybe three (four, at most), with different images.

This is a question we were thinking about -- whether to offer multiple image options for the article. For some articles, the algorithm can only offer one option, but there are articles where there are many options. One concern we had about it is if we give the user multiple choices, they might be inclined to choose the "best" one, even if the "best" one is actually not a good match. We would want them to feel free to reject all of them. This is also a design challenge, especially for mobile: how to let the user see multiple choices? It's good to know you think we should keep trying to figure this one out.

If an image comes from an existing article, and has a caption, why not try a translation as a starting point?

Are you talking here about the Commons caption, or the local wiki's caption in the wikitext for the article where the image is placed? It sounds like your idea is to use something like Google Translate to offer users something in their own language if nothing is available -- is that right?

Articles with no "File" wikilink have no images, yes?

The challenge of identifying unillustrated articles turns out to be harder than it sounds! Lots of images end up in articles through templates. Sometimes, we would consider those images "illustrations", like in an infobox. But a lot of the time, the only images in an article are little icons, flags, or symbols, such that we would not consider the article illustrated. Here's an example of an article in Arabic Wikipedia that has a bunch of tiny flags. Those images end up in the article through templates. It's looking like a smart approach would know how to filter out those sorts of little images, perhaps by seeing whether they are used dozens of times in a wiki. If it's used dozens of times, it probably isn't the sort of image that we would consider an illustration for an article.

If the image isn't going to go into an infobox, then it's probably best in the second section, rather than in the lead section. But if there is an infobox in the article (is this easy to determine), you could offer the user the option of putting the image there or elsewhere.

The logic of where to place the image is definitely an important question, and your comment caused me to do some more exploration of how this might work. I was thinking it would be something like:

1. If there is an infobox with an image slot, put the image there, with the caption written by the user in the caption slot. 2. If no infobox with image slot, put the image in the wikitext under all the templates at the top of the article, but above the body of the article.

Do you think that would work? Do you know of other cases that would make this more complicated? That second step would put the image in the lead section, not the second section as you recommend. But I was looking at this article, for instance. Do you think that if the article had an image, it would be better in the lead or in the "History" section? I think the problem is that the user would be choosing a single image to represent the article as a whole, but the first section might not be appropriate for the image, i.e. it might not be a "history" image.

If the images are being fed by the algorithm in the workflow, there really is little opportunity to vandalism, if by that you mean adding inappropriate images. (I hope you tag these image additions; if so, it's easier to see if they are being reverted to any great extent.) If you're worried that a user may inappropriately reject a lot of images, you might put a limit on the number of negative responses that a user can do (in a day?); when that is hit, the user doesn't see this option any more.

Yes, I think this sounds right. And part of the idea with structured tasks is that we're giving newcomers guardrails so that they can make productive edits in a way that can't be too damaging to the wiki. I think the worst damage someone could do with the "add an image" workflow that we're talking about here would be to choose "Yes" on a bunch of images that are bad matches, or maybe writing bad captions for them. We would definitely be able to tag these edits to see their revert rate. For the sorts of edits we suggest to newcomers so far, the revert rate is actually slightly below the revert rate for newcomer edits overall.

John Broughton (talkcontribs)
  • Regarding "whether to offer multiple image options for the article", I wasn't clear - I think your programming work would be much simplified - as would the task in front of the new editor - if you presented images (where more than one is possible) sequentially. If the editor says "Yes, good match" to the first, that's the end - they never see any additional images. If the editor says "No, not a good match", then the application says "Here's another, what do you think?" [And no, don't tell the editor this is "1 of 3"; if you do that, you have to allow the editor to go back, or say "Unsure"; that just unduly complicates things.]
  • On second thought, I don't think it's worth overthinking where the image goes. One of the good things about Wikipedia is there are lots of editors who like to do cleanup, which in this case would probably be to create an infobox and put the image into that. In other words, getting a good image into the article is 90% of the value; placement is only 10%. [But you do have to think about sizing.]
  • Regarding "does this article have an image", I would have thought (perhaps naively) that looking at the article wikitext (and specifically looking for the textstring [[File: ) would be straightforward. Apparently not.
MMiller (WMF) (talkcontribs)

Thanks for the quick response, @John Broughton.

  • Yes, I see how your idea for multiple images could be easy for the user. I suppose my concern would be that someone might pick the first image in the queue, but the second one (which they wouldn't end up seeing) is much better. I'll talk about this idea with our team's designer.
  • I like your point about how getting the right image somewhere into the article is 90% of the work. Could you tell me more about sizing? Would it not be sufficient to add all these images as "thumb"? What do you recommend?
John Broughton (talkcontribs)

Sizing as a thumb is a good default. I'd be really reluctant to introduce the concept of cropping, which involves alternative versions of images. There are lots of editors who care about images and can do cropping in their sleep, I'm sure (I'm not one of them).


I can see that "best of the bunch" seems tempting, but it's certainly going to complicate the interface, and if most of the time it makes little difference ... ? More to the point, this is really about (a) teaching new editors that they can add images, and (b) getting good images into articles; it's not a project whose goal is to get as many images as possible into Wikipedia articles.

Sdkb (talkcontribs)

Re sizing, the main times I adjust away from the default size are very wide images, which appear small as default (since thumb width is fixed) and often need to be made bigger, and very tall images, which appear big as default and often need to be made smaller.

LittlePuppers (talkcontribs)

Images aren't really my area of expertise, but there is definitely value to the idea.

Looking through the suggestions, there are a decent number of good ones, as well as some questionable ones; I think it would be beneficial to give some examples of what pictures are and are not good matches, and emphasize that it's fine to say no - if the system were perfect, there wouldn't be humans in the loop. I would like to note that in your prototype I came across one for a "given name" article, which per your guidelines should be excluded. As a result and some thinking I'm not sure exactly which articles should and should not have pictures - in looking through some given name articles, for example, I came across some without images, some with images of what the name comes from (eg. flowers), and some with images as maps of where the name is common. As such I do think it is difficult to make a call for what the best image for an article should be, or even if there should be one (I've been here for a bit, although I'm by no means an overly prolific editor, and I'm unsure on some of these - how are we to expect new editors to make the call? On the other hand, there's always the simple guideline of what you would want as a reader, which everyone has experience with.), but I'm not coming across anywhere where an image would be obviously harmful.

That's a somewhat unstructured stream of consciousness, but hopefully it's somewhat helpful. LittlePuppers (talk) 03:10, 16 December 2020 (UTC)

MMiller (WMF) (talkcontribs)

Hi @LittlePuppers -- thanks for these thoughts. I think you're right on with showing the users some examples. We hear something similar whenever we user test any feature: people say, "it would help me to see an example". That led us create an onboarding screen for the "add a link" feature that shows an example sentence about the moon (see image).

With respect to the question of whether an article (like a list of names) should have an image, I agree it can be a tough call. I think that something that could help is showing the user a preview of what the article will look like with their added image. Then perhaps they might have a moment where they say, "Oh, hmmm, this article is actually a list of a dozens of people named Jane, and so this image of one of the people at the top doesn't seem right."

Mockup of first of three onboarding screens for the "add a link" workflow, showing an example sentence.
Zoozaz1 (talkcontribs)

This is definitely task that I think has a clear benefit. I do have some suggestions and questions. As an example, how would it deal with adding an image to the article Transport in the Republic of the Congo (assuming no image exists)? The wikidata item linked to in the article does not contain the commons category, though going through P910 gets you there, which might be a solution. As a suggestion in cases like these or other cases where the wikidata item is not complete, it might be a good idea to check if there are any categories on commons with the same name of the article.

I also think a clear distinction should be made between adding an image and uploading it; I could imagine some newcomers getting confused at the distinction. If an article has an infobox with an image parameter, the image should automatically be added to the infobox; otherwise it should be added to the top. A lot of the other open questions are best solved through testing. Looking at the phabricator task, it might be a good idea to limit the task to specific types of articles where the task is most accurate.

Another suggestion I have is to give images on foreign language articles on the same topic before ones from the commons category. Linking to commons categories has a higher potential for error (such as if it is populated only by subcategories, or the images are fairly unhelpful or miscategorized, or even if it contains hundreds of possible images that someone would have to sort through) than getting it from foreign language articles, where another human looked over it and decided one particular image was suitable for the article.

MMiller (WMF) (talkcontribs)

Hi @Zoozaz1 -- thanks for quickly joining the conversation. Regarding the "Transport in the Republic of the Congo" article, I think you're using that as an example of an article for which it's hard to find an image via the algorithm I described, right? That's right -- for most articles, there is no clear way to grab an image for them. They don't have a Wikidata image, they don't have a Commons category on their Wikidata item, and no other languages have images either. So for now, it just means that we'll only be able to offer a minority of articles for users to address. And to be clear, it's not that the users would navigate on their own to articles of their choosing and try to add images, rather they would be provided with a feed of articles for which we know we have recommendations, and they have to review them.

That's a good point about making the distinction between "adding" and "uploading". We'll need to think about that when we write the copy: if we encourage newcomers to do this task, and we call "add an image", they might think they're about to take a photo. Perhaps it needs to be something like, "add images from repository" or "add suggested images".

About where in the article the image should go: I wonder if there are some wrinkles in the idea to "add it to the top". For instance, the top of an article frequently has a bunch of templates, like maintenance templates, before the body starts. So perhaps it would be more like "add it below the templates and above the body. Can you think of other pitfalls or wrinkles around that kind of logic?

I'll start a different thread for the thoughts about the order in which to prefer the logic in the algorithm.

Zoozaz1 (talkcontribs)

With all the markup and additions to many Wikipedia article, certainly adding it to the top can be challenging. I would say as a general rule of thumb doing it below the short description and any templates, but above the first word of the article, would work well enough.

Talking more about which articles the should be recommended, first of all an obvious approach would be not to recommend disambiguation pages. You've mentioned this, but there has to be a balance between recommending the articles with the highest pageviews without images (likely without images for a reason) and the lowest pageviews (with the least impact). It might be helpful to take a random sample of articles without images by pageviews to determine how much of a problem this is, and from that determine which pageview numbers newcomers should generally receive.

MMiller (WMF) (talkcontribs)

Thanks, @Zoozaz1. I agree about not recommending disambiguation pages. Other questionable types of articles might include:

  • List articles: for instance, for an article like Julius (name), one might add an image of a bust of Julius Caesar -- or is it bad form to include an image of just one item in the list?
  • Year articles: for instance, in an article like AD 112, one might add an image of Trajan, who became a Roman consul that year. I see that some year articles have illustrations, and some don't.
  • BLPs: perhaps because of the heightened sensitivity around BLPs, we should exclude them just to make sure the wrong images don't end up in them.
  • Featured and Good articles: people have illustrated these articles with care, and so we wouldn't want to casually change them (though most of them likely already have images).

What do you think of these? Can you think of other special cases?

Sdkb (talkcontribs)

There's a bit of a spectrum between disambiguation pages and list articles, and name lists are sort of in the middle. Disambiguation pages certainly shouldn't have illustrations as we want to get people off of them to their intended destination as quickly as possible without distraction, but nearly all featured lists do, and for Julius (name), it mentions Julius Caesar, so adding an image of him would probably be fine.

For year articles, there's certainly a potential to add images, but context is important—we'd want the Trajan image to have a caption that explains he became consul.

For BLPs, yeah, they're more sensitive, but they're also perhaps among the most straightforward, since if you find an image of the person, it should generally be added.

Excluding GAs and FAs definitely sounds like a good idea—if those are unillustrated, it's highly likely it's for good reason.

Sdkb (talkcontribs)

Thanks for the ping, Marshall! I have a bunch of thoughts on this which I'll throw out here.

First, looking at the broad picture of what's needed to improve the image situation on Wikipedia, I'd say the #1 thing is making it easier for people to upload images they've taken themselves. Despite the huge collection of images on Commons, it often happens that there's just no freely licensed image available. For instance, we've been seeking for months a decent image of the Huanan Seafood Wholesale Market (the Wuhan wet market), but the only image available is an uncompelling overhead shot of the street outside it. The Commons category has some images that look like probably copyvios (they're tagged as "own work" of a new Ukrainian user), and at one point someone uploaded a map that on second glance is promoting a conspiracy (that survived on en-WP for a few days and still survives on several wikis, from the looks of it), but there's just nothing else available. I'd guess that some Wikipedia reader somewhere probably has a photo of the market on their camera roll, and if they'd been prompted to upload it as aggressively as Google prompts me to add my photos to Google Maps we might have something on Commons. The point is that no amount of clever searching can find an image where one doesn't exist on Commons, and searching harder will only turn up problematic options. I'm not sure to what extent uploading one's own work could be made into a structured task, but that's where I see the greatest need, so perhaps it could get passed on to another team if it's not a fit here. Concretely, one thing needed is help revamping the local upload wizard for fair use files (until I recently implemented a stopgap, its workflow for anyone uploading their own work was to make them fill out the entire form and then say "actually, please restart at Commons", which is obviously terrible).

Zooming in a little, the main way that newcomers tend to interact with images is to come across an article on something they care about that's lacking one, say "I can find an image for that on Google!", and then try to upload something non-free (obviously not ideal, but also probably inevitable to some degree). I'm not sure it's safe to assume that this enthusiasm would translate to other pages that are just surfaced randomly. Emphasizing licensing needs to be up front, if only so that people understand why they're selecting from a limited pool rather than allowed to roam free on Google Images. And allowing people to choose a type of article rather than all articles would help, since I'm not going to find it that satisfying to upload images of subjects I don't care about.

Regarding the algorithm, it seems like it'll do a good job spreading images from one language to others, although from the long-term perspective that's just a stopgap to building a multilingual encyclopedia via Abstract Wikipedia. On the topic of globalization, I disagree on a wikiphilosophy level with classification -1. We're trying to document global knowledge, not just knowledge in our own language, so ko:나비 shouldn't prioritize an image of a Korean butterfly just because most Korean-language readers are from Korea. Maybe small wikis see this differently, and certainly de facto it tends to happen, but on en-WP we at least aspire to not give additional weight to English-speaking countries at pages on global topics (see w:Template:Globalize).

I agree with Zoozazz1 that Wikidata/other languages should be prioritized before Commons categories. As with the Ukrainian example above, though, there's some risk to this, as I suspect some of the smaller Wikipedias struggle with enforcing copyright, and we don't want to spread their errors into other languages. When there's desire for an image that isn't available, that creates a kind of pressure that tends to get filled with improperly licensed media, and that's hard enough to counteract even when it doesn't have an algorithmic boost.

Regarding the difficulty of the task, I think it could actually end up being fairly tricky depending on the quality of the metadata available. For instance, if it's a group photo that includes the subject of a page, then the editor will need to identify which person the subject is, something that's not always in the caption, and then crop to them, which is a whole additional component. And if the title/description/caption is in a different language, that massively magnifies the potential for error; I'd suggest avoiding that until this is up and running smoothly with non-translated suggestions.

Regarding determination of articles to surface, I'd suggest taking into account pageviews, as we want this feature to have impact for readers, not just be wasted on essentially unread pages. However, that gets tricky, since the most-viewed unimaged pages are presumably that way for a reason (this is again the problem of a lack of images creating pressure to add inappropriate ones).

I'll stop there as I've already written a lot, but I hope the above is useful, and feel free to ask any follow-ups.

MMiller (WMF) (talkcontribs)

Hi @Sdkb -- thanks for quickly posting your thoughts! It's all very useful. I'll start a separate thread to talk about the idea of how to help users upload images themselves, and another to talk about the algorithm. About your other points:

Emphasizing licensing needs to be up front

This is a good reminder that an important part of structured tasks is teaching newcomers about how the wiki works, so that they can graduate to more ambitious tasks.

And allowing people to choose a type of article rather than all articles would help

Yes! I think that we would want to use the same topic modeling for this task as we've used for other kinds of suggested edits, in which a user chooses from a set of 39 topic areas (Performing arts, Physics, Transportation, etc.) and then only sees suggestions in that area. 75% of newcomers choose some topics, giving us some signal that people like to do this narrowing to topical areas.

just a stopgap to building a multilingual encyclopedia via Abstract Wikipedia

I'm also excited about Abstract Wikipedia, and glad to hear that you're following along with that project. For what it's worth, we're in touch with the Abstract Wikipedia team, and hopefully that will help us see opportunities in the farther future.

On the topic of globalization, I disagree on a wikiphilosophy level with classification -1

That's good to know. I haven't heard that perspective as much. I think the globalization issue is one that we would think more about down the line, and so I want to defer figuring that out until a bit later. We also don't really know how often these "-1s" would come up in practice. In our tests, they were very rare.

For instance, if it's a group photo that includes the subject of a page, then the editor will need to identify which person the subject is, something that's not always in the caption, and then crop to them, which is a whole additional component.

I think you're bringing up an important larger point here: we need to be mindful of image policies on the various wikis. It sounds like you're saying that on English Wikipedia, images are expected to be cropped to the specific person? It we look at the example I posted in this section on the project page, the article is about an individual, but the suggested image contains two individuals. Would you say that needs to be cropped? Or might it still be a productive contribution without the cropping?

Regarding determination of articles to surface, I'd suggest taking into account pageviews

Yes, this is a question that has come up for our team a lot. On the one hand, we want to help newcomers to be impactful. On the other hand, they're new, and so maybe we should not be asking them to try their first edits on the highest-visibility articles. Perhaps there's an idea in here about preferring higher-visibility articles for users who have more experience.

One other thing I'd like to ask you -- I think that the people who usually follow along with Growth team conversations are engaged because they care about newcomers. I was thinking it might be good to hear thoughts from people who are image experts on the wikis. Would you recommend anyone I can ping to ask if they want to weigh in?

Sdkb (talkcontribs)

Thanks for those replies!

Regarding cropping, the w:MOS:IMAGES guideline includes A biography should lead with a portrait photograph of the subject alone, not with other people. This isn't strictly enforced (particularly outside of infoboxes), though, and a group photo would be better than nothing, especially if it's clear from genders who the subject is or if the caption identifies their position. Regarding the example, yes, given that there's enough space in that photo to crop it naturally, I would do so if I were adding it to a page.

Regarding image experts, Rhododendrites comes to mind as someone I respect who is experienced with images. They might have thoughts or know someone else who would. Many image experts hang out at Commons, so you might want to post to the Village pump there.

MMiller (WMF) (talkcontribs)

Got it -- thanks, @Sdkb. I sent a message to the user you recommended.

About cropping: perhaps rather than ask users to do the cropping, there could be a way to flag the image as "needs cropping", so that they could add it to the article, but someone with more expertise could come along later to crop it.

Sdkb (talkcontribs)

Yeah, that might be a good happy medium. Cropping isn't too difficult if one has the CropTool extension enabled, but since it creates a new file (see this wishlist item), it can get a little messy if anyone messes up.

HLHJ (talkcontribs)

Strong second to that, Sdkb, MMiller (WMF). See also Commons:Commons talk:CropTool#PDF quasi-extract for a detailed step-by-step description of what the tool should do. Wikisource has... thousands? of scanned books with transcribed texts but without extracted images. Where the images should be, it says "An image should appear at this position in the text". Scroll through Wikisource:Ten Books on Architecture/Book V for a varied example; the missing images are all implicitly tagged as needing cropping. Unextracted, still embedded in a scanned page of text, these images are unusable on other projects, and often unfindable.

I've spend many tens of hours manually extracting images, not only from scanned book pages, but from PDFs where the image files are already machine-extractable, and just need a human to check that the autogenerated name is sane. The latter are often academic articles. It's very frustrating.

So:

1. Cropping images out of scanned book pages

2. Guiding an automated extract of images in machine-readable PDFs

Both could readily be done by newbies. Both would be genuinely useful.

MMiller (WMF) (talkcontribs)

@HLHJ -- thanks for pointing out these ideas, which sound like good structured tasks for Wikisource content (but possible could be surfaced in the Wikipedia context). I'm going to add them to the full list I keep of structured task ideas.

HLHJ (talkcontribs)

MMiller (WMF), thanks. To some extent this is also a Commons task (and readily translatable). For instance, at Commons:Category:Japanese Homes and Their Surroundings (1885 book), you will find some of the line drawings from that book, which I began uploading because I wanted to use a few on Wikipedia. You won't find all of them, because I got too bored uploading them. I filed a request somewhere for automation, but as you can see, that was nearly a year ago. HLHJ (talk) 15:44, 6 March 2021 (UTC)

Pelagic (talkcontribs)

Thanks for the ping. I haven't read this whole thread yet, but did look at the page Growth/Personalized first day/Structured tasks/Add an image. My first thought on seeing the prototype screenshot was "oh, is that how you plan to do it?" I'd been thinking that add an image would involve showing 2–4 options, and the user chooses a preferred one.

I realise the final flow is still to be decided, but how would it affect the suggestion algorithm if you were aiming to pick four suggestions instead of one?

MMiller (WMF) (talkcontribs)

Hi @Pelagic -- thanks for joining in. @John Broughton also brought up the question of whether we would show just a single image or multiple options. It's true that the suggestion algorithm can sometimes suggest more than one option. Our first inclination was to only show one, because (a) it would be simpler to design/engineer/understand, and (b) we would worry that if the user has multiple options, they would be inclined to choose the best of the options, even if none of them are actually good enough. What do you think about that balance?

I just posted mockups of three potential designs along these lines. The first just shows a single image. The second shows multiple images all at once. The third shows multiple images in serial, and the user decides on a best one if they liked more than one along the way. Does one of these designs seem best to you?

Sdkb (talkcontribs)

It's a little hard for me to anticipate how likely people might be to choose an image even if none fit if the "multiple" option is chosen. You might have to do some user testing to figure that out. I do like the disclaimer text shown with the "single" option, but otherwise I lean toward multiple since image quality on Commons can vary a ton and it's a big plus to have a good image rather than a bad one. (In some cases, it could even be worse to have a subpar image than no image, since even if it's an improvement in the short term, long-term the existing presence of an image might discourage people from going to Commons and finding the better one, which might have happened otherwise.) I'm not really a fan of the "serial" option, since that seems like a bit clunky and would require more taps (when the goal is just to find one good image, asking people to rate multiple images is just extra work).

HLHJ (talkcontribs)

Linking Articles to Commons categories with a "commonscat" template might be more useful and automatable. I think we need to be clear that we are asking the user to judge whether the suggestion bot is out to lunch, or they will tend to assume that the suggested edit must be right because that's what Wikipedia's program says. So more "does this image category match this article?". HLHJ (talk) 02:50, 25 January 2021 (UTC)

MMiller (WMF) (talkcontribs)

@HLHJ -- thanks for joining in. Can you explain what you mean with the commonscat template? I'm not familiar with what it is and what it does.

I think you're right, though, about the user potentially trusting the algorithm too much. Some good news is that trusting this algorithm a lot is pretty reasonable, since it's only suggesting connections that have been made by other Wikimedians already via Wikidata or other Wikipedias. But we did notice in our user tests that some users would tend to give the algorithm the benefit of the doubt. For instance, they might have an article about a church, and then the suggested image might be of a church, but it might be hard to tell if it is the church from the Commons metadata. In a situation like that, the user might say, "Well, the image is of a church, so I'm assuming it's the right church." We definitely want them to be skeptical, and err on the side of "no" instead of "yes". This is something I think we'll be able to influence in the design and onboarding to the feature, by making it clear that that the user is not just supposed to rubber-stamp the algorithm; they're supposed to be quite critical. Does that make sense?

HLHJ (talkcontribs)

Template:@MMiller (WMF) en:Wikipedia:Template:Commons category. It makes the little box that says "Wikimedia Commons has media related to X" in the article. The Names of the article and the Commons category are often but not always identical.

Yes, that makes sense. "Is this correct?", rather than "Do this".

Why not make it a CAPTCHA? The CAPTCHA user-interface format makes it clear that you are being asked to make a judgement. The time is ripe; Google just started charging for their "reCAPTCHA" service. Now way back when dinosaurs roamed, reCAPTCHA was an academic project used to digitize public-domain books (it showed two non-computer-decipherable words, one of which had already been translated by a few humans, one novel, and judged correctness by consensus). When Google bought it in 2009 they gave the usual reassurances, but they used it to train proprietary algorithms; identifying address numbers on Google Street Views, and now training self-driving cars. Now they have also started charging larger websites to use it. A lot of the websites are not really keen on paying millions to Google. So if you can make an embeddable wikimediaCAPTCHA, you can get a LOT of people involved in editing. And once the CAPTCHA has concluded that they are human, it could thank them for contributing to WP and invite them to come edit an article related to the site they are on when they have time. Or invite them to make an account so they get their CAPTCHA work credited to it. This would get pretty much everyone on the internet saying "I contribute to Wikipedia sometimes".

MMiller (WMF) (talkcontribs)

Hi @HLHJ -- I think that the CAPTCHA idea is interesting, and I think it's come up a few times in our history. I found this example of when the idea came up a few years ago, and I think there are others. I think the broader idea you're getting at is whether the sort of "structured tasks" we're building now could be portable off Wikimedia properties. I think that's a potentially exciting future idea, and I'm going to bring it up with some of my colleagues.

HLHJ (talkcontribs)

Thank you; that Phab link is exactly my idea, much more succinctly expressed. Carpe diem! HLHJ (talk) 15:46, 6 March 2021 (UTC)

Pelagic (talkcontribs)

Hi @Marshall, sorry for the long delay in returning to this. I’m really impressed how you managed to fit the multiple thumbs into a single small screen for the Multiple mock-up. I seems more intuitive to me than Single and Serial options. The extra grey tile (x) tells one that no-picture is an option.

But if I was doing that in real life, I may have wrongly added the millefeuille picture to the vanilla slice article (similar shape and icing, different filling)! * The only thing that tipped me off was the file name. Would I have been more likely to notice that in the Single version and tap “not sure”? I don’t know.

If a user gets six cases in a row that are all skip, skip, ... skip, will they become discouraged? Could there be an option to “load more suggestions” before “skip”? The chance of surfacing something good with an alternate search strategy might be low, though.

( * Argh, there's a merge discussion about that right now. )

MMiller (WMF) (talkcontribs)

Hi @Pelagic -- thanks for the response! We've actually learned a lot since posting those different mockup ideas. We conducted user tests using a simple "single" image prototype, and one of the most important things we learned is that it is absolutely crucial to see the image metadata (filename, description, caption, etc) in order to make a confident decision. Just as you say, something might look like a vanilla slice, but not actually be one, and you need the metadata to tell if it really is a vanilla slice, or some other kind of cake. Going forward, I think we're going to be favoring designs that convey the metadata as a "first-class" citizen of the experience, as opposed to a design that implies that one could make a strong decision without it. When we do our next set of tests, I expect that we'll try the "serial" design as opposed to the "multiple" design for this reason.

We're also going to learn a lot in the next couple months from the Android MVP, which uses a "single" design, but will generate lots of data to inform our future designs.

Reply to "Pinging community members"

I'm not sure this is the right approach

5
Mike Peel (talkcontribs)

Hi all. While I understand why you are approaching things this way, I am not sure it is the best thing to do. It's good to see that you're using Wikidata as part of the input for these suggestions, but a lot of Wikipedias now are heading towards using P18 values directly in infoboxes (e.g., on enwp, telescope articles already do that; lots of eswp and frwp articles also do so; and of course Commons infoboxes display everything from Wikidata).

I suggest instead to focus on adding P18 values to Wikidata items - and to also think about adding other image values, like P3451 (nighttime view), there are also interior views, coats of arms, etc. That way, you will significantly increase the impact of what you're doing (the images will be used in multiple languages from the start), while avoiding adding extra clutter (when converting an infobox to Wikidata, do you use the existing image on Wikidata or the one from the article), and you're more aligned with what others in the community are trying to do (expand Wikidata and increase its usage), while your current approach may easily bring you into conflict with editors on the different Wikipedias.

(Also, note that P373 will hopefully be deprecated soon, you're best using the sitelinks to Commons instead.)

MMiller (WMF) (talkcontribs)

Hi @Mike Peel -- thanks for joining in this discussion! It is definitely great to hear from people who are experienced on Commons. We also just heard this same thought from users in Czech Wikipedia, when we brought up the idea there, because they use Wikidata infoboxes extensively. I'm trying to think about the right way to approach this. I could imagine altering the task so that newcomers add P18 values, but it brings up a bunch of questions for me. I hope you (and anyone else watching the page) can reply with your thoughts on them:

  1. In the case where the image would be recommended because it is used on another Wikipedia, should we be concerned about whether it would "clear the bar" to be the P18 for the item? In other words, my sense is that the bar for image relevance and quality is higher for P18 than for an image in an article -- because P18 is meant to be the image to represent the item. Taking a real example from the algorithm, perhaps the unillustrated article is 1991 World Championships in Athletics – Women's heptathlon, which has no P18. And the recommended image is this photo, which is used on the same article in German Wikipedia. The photo is of the stadium where the event took place, but taken when some other event was happening in it. I do feel like that image meets the bar of illustrating the article in an enriching way, but not the bar of P18 to represent the item on Wikidata. What's your take on situations like these?
  2. What would you recommend for articles that have a P18, but don't have an infobox in them? On the one hand, we could enable users to add the P18 to the article's wikitext. Or do you think that the best thing is to add a WIkidata infobox so that the P18 is drawn in (which might be a task out of the reach of newcomers)?
  3. When adding an image as P18, I imagine that might immediately make it show up in infoboxes in many Wikipedias. Because we're thinking about newcomers as the primary audience for this feature, I'm a little worried about weak decisions from newcomers reverberating across many Wikipedias. How do you think we should think about this?
  4. Related to the previous question: as Wikipedias inherit information from Wikidata through these infoboxes, would that mean that Wikipedias are not able to patrol those changes, and they are only patrolled on Wikidata? Has that been working well for Wikipedias so far?
  5. If we enable newcomers to make Wikidata edits through this feature, I would be concerned that there are fewer opportunities for them to make connections to their Wikipedia communities. Their Wikipedia edit count wouldn't increase, their work wouldn't be visible on their own Contributions list, and local editors wouldn't be seeing their work in watchlists or Recent Changes. What are your thoughts on this?
  6. Is there anything I can read about the potential deprecation of P373? I hadn't heard about that, and I definitely want the algorithm's researcher to look into it.

Thank you for helping us think about this work!

Czar (talkcontribs)
  1. If you would be comfortable adding an image to the top of an article, I think it would be okay for wikidata:Property:P18. Per discussion on the property's talk page, usage is not heavily regulated. I don't think the stadium photo appropriately depicts the 1991 Women's heptathlon either for a main image or a P18. Lower in the article perhaps—even then it's a stretch but falls in the range of mediocre edits that are unlikely to be challenged unless someone happens to be watching that article.
  2. P18 but no infobox: Every community will have its own preference. If you'd be comfortable adding the image to the top of the article with the currently proposed tool, I think it's fine to start with the assumption that it would work in both P18 and as a thumbnail (captioned or not) at the top of the related article. I wouldn't necessarily add infoboxes automatically—that's at least a sensitive subject on ENWP.
  3. P18 automatically shows in multiple places: Per the thread below, I wouldn't worry about this. Wikipedias that have opted into automatically pulling P18 from Wikidata are aware that the quality is variable. If anything, it makes it easier to undo that addition if the tool had instead hardcoded the image into multiple Wikipedias.
  4. Wikidata quality has been a source of contention for ENWP and is a reason why its editors would rather hardcode their own short descriptions, mostly not use Wikidata metadata in infoboxes, and not want to outsource its Category feature to structured Wikidata. There is a way to view an article's related Wikidata changes within the WP's watchlist but I'm guilty of not really looked into it after turning it off when it was rolled out (and really noisy). Mike will have more to say about this but to my knowledge, it's something that's already in heavy consideration. I think it's widely agreed that Wikidata would need more quality checks and integrations before ENWP considers integrating more tightly. Again, if other language WPs have already bought into using Wikidata because they would lack the edits otherwise, I think they're already comfortable with using whatever Wikidata provides.
  5. I would doubt that newcomers would automatically receive encouragement whether adding images, adding links, or adding Wikidata metadata. Those are entry points to heavier edits. I wouldn't expect much of a "community" for someone making these edits by mobile. For comparison, check out new editors who join and use some of the desktop web antivandalism or other semi-automated tools. They need to hit a high volume before regulars notice. At most, newcomers using the proposed mobile features will get a "welcome" template, based on current ENWP practices. Some local editors tracking Wikidata in their watchlists might see, but I'd figure that community connection will be its own problem to solve with any of these structured task proposals. If you invite/build a contingency of volunteer editors from a WP to be invested in this project, they might be able to figure this out on their end without product support.
  6. Visit wikidata:Property talk:P373 to find wikidata:Wikidata:Properties for deletion#Property:P373—it hasn't been decided yet, it seems? Even if/when it is deprecated, it will just be linked through sitelinks when relevant.
MMiller (WMF) (talkcontribs)

Thanks, @Czar. Everything you've said pretty much makes sense to me (I'm sorry for asking similar questions on other threads before realizing you've already answered them here). This is all helping me understand what P18 and P373 are and are not for.

Also pinging @Mike Peel again -- I hope you have a chance to weigh in!

Mike Peel (talkcontribs)

Sorry for the really slow reply @MMiller (WMF)! I completely agree with everything that @Czar says. The only extra point I'd make is on (4), you can see Wikidata changes in your watchlist, and it can be useful on occasion (but there is a lot of noise). However, compare that with the use of templates and images in articles, which can be changed dramatically without any notice appearing in a watchlist/article history. It's more about trust than anything else (and, for enwp, requiring referenced data only - but obviously not for things like images). (And on 2, I'm generally a fan of adding infoboxes, but others disagree.)

I do think it's worth exploring the other Wikidata properties like nighttime view - perhaps these aren't as immediately visible very widely, so newbie mistakes will make fewer problems there? Also, consider suggesting featured or quality images for adding to other parts of the article, rather than just in the infobox.

Reply to "I'm not sure this is the right approach"

"How exactly should we determine which articles have no images?"

9
Pigsonthewing (talkcontribs)
Sdkb (talkcontribs)

In my experience, the presence of a requested image tag on an article usually means that someone has searched for an image on Commons and couldn't find one.

Pigsonthewing (talkcontribs)

Indeed; but that may have been months or even years ago, and an image may have since been uploded to Commons.

MMiller (WMF) (talkcontribs)

Thanks for that pointer. I was also wondering how that category gets applied. I see that the category is applied to about 4,000 articles, while according to a calculation made by a researcher we're working with, about 50% of English Wikipedia articles are unillustrated (I need to check whether that includes disambiguation pages).

Sdkb (talkcontribs)

Yeah, the presence of a requested image tag does certainly indicate an exceptionally high demand for an image, so if the algorithm is confident it's found one, it should probably be at the top of the queue. Many of the articles with this tag just don't have the desired image available, though.

John Cummings (talkcontribs)

It feels like trying to tag things manually would be a giant and unfun task and could be done automagically.

If there is no 'File:' in the wikitext for the page then can we assume there is no image? This would ignore images in the infoboxes I guess? Maybe also include .jpg and .tiff?

MMiller (WMF) (talkcontribs)

@John Cummings -- it looks like the way we are planning to identify unillustrated articles is actually by looking at the HTML of a page, as opposed to its wikitext. The reason is because images can appear in articles through templates, like navboxes or Wikidata infoboxes. Another aspect is that many articles include images that we would not consider an "illustration". For instance, this article contains many small flag images -- but we would still want to consider it unillustrated so that a real illustration could be added. We're planning on an approach that can that aspect into account.

Reply to ""How exactly should we determine which articles have no images?""

Will newcomers have sufficiently good judgment when looking at recommendations?

2
T Cells (talkcontribs)

Most newcomers have sufficiently good judgment when looking at recommendations that are easy-to-find, concise, and simple with little or no technical details.  Newcomers are not likely to search for recommendations that are hidden or look at recommendations that are not sufficiently clear or written in complex language with too many technical details. This impact on their judgment and they tend to do things in their own way as a result. This is based on behavioral observation of newcomers during the Wikipedia Pages Wanting Photos.

MMiller (WMF) (talkcontribs)

@T Cells -- thanks for joining this conversation. It's good to hear that you think newcomers can apply good judgment to images. When you say that we should avoid "technical details", what kind are you referring to? Do you mean that we shouldn't expose too much of how Wikidata and Commons work behind the scenes, so that the newcomer can just focus on the task?

Reply to "Will newcomers have sufficiently good judgment when looking at recommendations?"

Breaking the problem up into topics

4
John Cummings (talkcontribs)

One approach may be breaking the problem up into large topics (e.g nature, science, buildings etc) and possible things like area of the world, this might help in a couple of ways:

  • Allows new users to work on an area they have some working knowledge of a topic of e.g I would have a high confidence in adding images to articles about buildings but not about scientific concepts. This would hopefully increase accuracy of images added and allow people to contribute to an area they're interested in.
  • Potentially allows us to draw on the millions of high quality images collected by Wiki Loves competitions or images from the collection of a partner organisation who have donated sometimes millions of images e.g Wellcome Collection.
T Cells (talkcontribs)

Breaking the problem up into topics makes sense. This would help new editors to properly add caption to images and narrow their works to areas or topics they are familiar with or have expertise.

Trizek (WMF) (talkcontribs)

At the moment, the Growth Suggested Edits tool allows people to filter task by topics. Newcomers can filter articles about Transport needing copyediting. :)

MMiller (WMF) (talkcontribs)

@John Cummings -- thanks for bringing up this idea. As @Trizek (WMF) says, letting newcomers select topics of interest is already part of the Growth feature set, and would be part of an image recommendation workflow. Right now, we offer 39 topics to choose from, and we believe users like topics: about 75% of users who work on suggested edits choose some topics before they get started. Here's an image of what it looks like for newcomers. What do you think?

Reply to "Breaking the problem up into topics"

"Wikipedia Pages Wanting Photos" campaign

4
Pigsonthewing (talkcontribs)
T Cells (talkcontribs)

@Pigsonthewing, thanks for your comment. The Wikipedia Pages Wanting Photos campaign was a new experiment and a simplified campaign to recruit newcomers who may be interested in putting into use the millions of photos on Wikimedia Commons and other works that have been released for use under a free license. We are very proud that many of the new editors recruited from the campaign continue to contribute to various Wikimedia projects to date. This is something we find really interesting as the campaign demonstrated that adding images to articles could be a great way to recruit and engage new editors. It could also be a way to engage and retain existing editors who may want to do something entirely different from creating new articles or improving existing contents. However, the blocking of the user you cited was justifiable but that's not strange or unexpected for a new campaign involving new editors. We shouldn't judge an entire campaign based on some bad behaviour from a few new editors.

:We mop thousands of copyvios contributed to Commons during and after the WLX contest annually but we can't discredit those contests based on bad contributions from some new editors. That's usually not a smart way to evaluate the overall impact of a project. This is not to compare Commons to Wikipedia as the latter deals with live articles

:I'd also like to note that we cannot second-guess the motivation of other editors by simply assuming that all participants participated just to win prizes. We should assume good-faith as such assumption may be considered disrespectful, insulting and demoralising to our volunteers who have no such motivation. The WPWP campaign was not the first and only project that offer prizes. Almost all contests and campaign organized in the Wikimedia movement including the WLX offer prizes.

:That being said, we took the feedback provided by some communities notably, the English and French Wikipedia seriously as they are very valuable and would be useful for the international team to improving the campaign in the next edition.  And if the works on the "structured task" is completed before the next edition in 2021, that would really be helpful in taking care of some of the concerns raised particularly those that are related to poor contributions by new editors. Again, thanks for your comments and Merry Christmas. T Cells (talk) 15:52, 24 December 2020 (UTC)

MMiller (WMF) (talkcontribs)

Thanks for linking to that discussion, @Pigsonthewing. It definitely shows that while a lot can go right from urging people to add images, a lot can also go wrong. We should think about what we can learn and try to head it off in building new features. It seems like some of the main challenges were:

  • People adding totally irrelevant images. This would hopefully not happen often because this feature would be built on an algorithm which usually proposes images that are relevant. In other words, there would not be an opportunity for the user to search Commons and insert something random to the article.
  • People might be incentivized to add images carelessly just to rack up points. Right now, our features don't offer any kind of awards, but we might offer a good way to see your "stats", so you can be proud of your progress. I think one thing that can help with the incentives is to give the user credit for the times that they declined to add a suggested image: saying "no" to a suggestion should "count" as work, just like saying "yes" does. So instead of someone thinking about their "images added" number, perhaps we can refocus them on their "images reviewed" number.
  • People continuing to rapidly add images even as they are being reverted. It sounds like there were some scenarios in which users were adding irrelevant images so quickly that people couldn't clean up after them and block them quickly enough. In the Wikipedia Android app, they incorporated some automatic rules that remove a user's access to tasks where they have been frequently reverted. Maybe that's something we can think about.

How does this sound? Can you think of other pitfalls to plan for?

Another question I have is about image "quality" and how users should evaluate it. For instance, the only available image of an obscure historical figure might be a grainy or blurry photo (like this one). Or the only available image of a town might mostly be a big field with some of the town in the background (like this one). What do you think about cases like those? Is it good to include those illustrations in the Wikipedia articles, despite their drawbacks?

Sdkb (talkcontribs)

For both of those examples, I'd say they're better than nothing, but I could envision a circumstance in which an image is so bad that we'd prefer to have nothing. That situation comes up sometimes when we're deciding whether we should illustrate a particular section, but it's less common at the level of an entire page. On English Wikipedia, there's some philosophical disagreement about how strictly to interpret MOS:IMAGERELEVANCE—some editors want only images that have a clear instructional purpose, whereas others such as myself are much more lenient about images that fall more (but not completely) toward the decorative end of the spectrum.

Reply to ""Wikipedia Pages Wanting Photos" campaign"
MMiller (WMF) (talkcontribs)

@John Broughton @Zoozaz1 @Sdkb -- you brought up thoughts about how the algorithm works, and how to prefer the three main inputs to it:

  • Look at the Wikidata item for the article. If it has an image (P18), choose that image.
  • Look at the Wikidata item for the article. If it has a Commons category associated (P373), choose an image from the category.
  • Look at the articles about the same topic in other language Wikipedias. Choose a lead image from those articles.

Ideally, we would be able to generate sufficient data on images from each input to calculate the rank ordering of which inputs are the best and worst. Imagine if people went through hundreds of potential matches, and labeled them "Yes" or "No", and then we could see which inputs have the most "Yes" answers. Perhaps volunteers might help us do something like that. Or farther down the line, if the feature is in the wikis, we could look at revert rates for images from different inputs.

But absent that quantitative approach, it's good to have some expectations. It sounds like all three of you expect the Commons category input to be the least reliable. That makes sense to me, and aligns with what we saw when we evaluated a couple hundred of these matches. Because many images can be in a Commons category, there can be plenty of images that are only peripherally relevant.

I agree that the other two inputs both seem pretty strong. Usually a Wikidata item only has one P18 image, and so I would imagine the person applying it to the Wikidata item believes that it is an appropriate image to illustrate the concept as a whole. Is that your expectation as well?

For images from articles in other languages, this one usually seems to work well, but has some wrinkles to it. The main issue can be illustrated with the following example, in which the article where the image is drawn from is a lot more extensive than the unillustrated article. In English Wikipedia, the article "Economy of North Rhine-Westphalia" has no image. It's a pretty short article. Its counterpart in German Wikipedia is a long article with lots of images, which makes sense because that region is in Germany. Which image would we recommend? One idea is to choose the first or "lead" image in the article, which in this case is a photo of a factory. Perhaps that would work as the single image in the English article, but only with a good caption explaining why it belongs in the article.

But overall, I do agree that the "images from articles in other languages" approach has the great benefit of another Wikipedian having consciously decided that the image is appropriate for a Wikipedia article on the same topic.

John Broughton (talkcontribs)

Whether the first or third source, listed above, is best, is an empirical question; it should be fairly easy to do a comparison. I think we're also agreed that the second source, above, is the least desirable.

More generally, I think that you should be moving in the direction of presenting, serially, three (or four, maximum) images to the editor, if there are that many; if the editor says "yes" to the first, that's all he/she sees; if he/she says "no", then the second image is present, for the same yes/no choice. (That responds to "which image would we recommend", if an article in another Wikipedia has multiple images; you don't have to choose, if you present, sequentially, multiple choices. (I do wonder if the larger the image, the more likely it is to be useful; that's another empirical question.)

MMiller (WMF) (talkcontribs)

@John Broughton -- I talked with our team's designer about the question of presenting multiple images serially, and I just posted some ideas in the Design section. The first idea just shows the simple "single image" workflow. But the second two are ideas for how to present multiple image suggestions. In one of them, the user sees all the suggestions at once, and the other one presents the suggestions serially, with an additional step at the end for the user to decide between them if they ended up liking more than one. I know that none of these are exactly what you proposed, in which the series of images stops as soon as the user chooses one, but I was thinking that these workflows would give the user a chance to discover that an image later in the series is actually better than an earlier one. What do you think?

Edit: I see in another of your responses that you said that "best of the bunch" might add needless complication for little benefit. That's a good thing to keep in mind.

John Broughton (talkcontribs)

I don't have strong feelings about the three options. What I do have concerns about is the text (from the Commons) that accompanies the images you're showing. The new editor has no idea where that text came from, and the extent to which it should influence his/her decision. What if the text for one image is a fairly good match to the topic, while the text for another image is not - should that be a deciding factor?


I also think it's a serious mistake to show the image on top of the task window, with the article text below: (a) image, (b) topic information, and (c) question. I strongly suggest that the order be (a) topic information [which the editor saw in the previous task window, and therefore sets the stage, (b) image, and (c) question [which is immediately below the image, to which it is, obviously, closely related].

Reply to "Algorithm"

"Will newcomers who don't read English be equally able to make good decisions, given that much of Commons metadata is in English?"

2
Pigsonthewing (talkcontribs)

Use Commons' structured data, which is multi-lingual.

[Sadly, much of it is polluted with generic "tags", by another WMF-led project; see https://commons.wikimedia.org/wiki/Commons_talk:Structured_data/Archive_2020#Is_this_really_how_it's_supposed_to_work? et seq. I'm still waiting for RIsler (WMF) - or anyone - to answer the points raised in February, in Commons:Village pump/Archive/2020/02#Misplaced invitation to "tag" images. Those being the points he said were not being ignored; and including (but not only) "requests to show where there is consensus for the tool to operate, or to use depicts statements in the manner it is [and] requests to explain how the tool, or the invitation to tag, can be turned off."]

MMiller (WMF) (talkcontribs)

@Pigsonthewing -- yes, I was thinking about that, too. I think something we'll do in January is run counts on how much structured data exists in which languages. This will give us a sense of many image matches we would be able to propose that have captions in the local language, and we can see whether there would be sufficient volume. In our user tests so far, users are being pretty careful about making matches, e.g. they want to make sure the church in the image is the church being described in the article, not just a church. I will try to learn more about how often the depicts statements are specific enough to help with that. What's your take?

Reply to ""Will newcomers who don't read English be equally able to make good decisions, given that much of Commons metadata is in English?""
Pigsonthewing (talkcontribs)

Where the Wikidata item corresponding to a Wikipedia article has an image and the article has no image, then the image should usually be displayed in the article.

On Wikispecies, this is done with https://species.wikimedia.org/wiki/Template:Image which can be used even when Wikidata has no image, in which case it displays nothing, but if an image is later added to Wikidata, then it is displayed in the article. Some infoboxes can also do this.

MMiller (WMF) (talkcontribs)

Hi @Pigsonthewing -- thanks for checking out the materials! It sounds like you have high confidence in the Wikidata image -- that's good to know as we continue to refine the algorithm (which currently does choose the Wikidata image as its first priority). I do need to learn more about how templates can help with this effort, and also to be careful not to accidentally collide with images brought in through those templates.

Reply to "Use the Wikidata image"