Talk:Edit check

So glad to see!
@ESanders (WMF)@PPelberg (WMF), I'm very glad to see that this project exists! If implemented well, I think it will have a strong positive impact on the new editor experience. Please let me (and others!) know when you're looking for community feedback, and I'll be happy to share thoughts on specific ideas/prototypes! Cheers, &#123;{u&#124; Sdkb  }&#125;  talk 18:29, 24 January 2023 (UTC)


 * hi @Sdkb! I'm glad that you're glad to see the Editing Team prioritizing work on this ^ _ ^
 * Please let me (and others!) know when you're looking for community feedback, and I'll be happy to share thoughts on specific ideas/prototypes!
 * Thank you for offering as much; we're going to need it!
 * In fact, since we're here, I wonder: can you think of specific Senior Contributors or groups of Senior Contributors that you think we ought to prioritize reaching out to first? I've added a bit more context below about the project and what we're currently looking to learn at this stage.
 * All of the above aside, thank you for dropping by! You doing so was the reminder I needed to prioritize posting a status update about this work which I'll do on the project page before this week is over.
 * Seeking Input from Seniors
 * Okay, this is the "context" I referred to above...
 * In the coming weeks, I expect us to be ready to share an initial idea for the user experience of the first "check" we're prioritizing work on: a check that will present people who are attempting to add new content without a corresponding reference with a call to action to do just that. ''This design work is happening in T325711.
 * While we'll be keen to hear what Senior Contributors think of the user experience we're proposing to present to newcomers and Junior Contributors, we're thinking some of the most valuable things we have to learn from y'all in this moment are things like:
 * "In what – if any – ways do Senior Contributors think the proposed mobile UX flow could be adjusted/augmented to increase the ease and efficacy with which they can review edits people will be making with Edit Check?
 * "Which parts of the feedback system Edit Check is proposing do Senior Contributors would value being able to configure on a per-project basis?" [i][ii]
 * i. This "configuration" bit is particularly top-of-mind for the team as we're seeing Edit Check as having two distinct, and complimentary, dimensions: 1) the user experience newcomers and Junior Contributors will see and 2) the configuration tools/options Senior Contributors will have access to that will enable them to customize this user experience it guides people to take actions that align with project policies and conventions.
 * ii. For more details about how we're thinking about consulting with experienced volunteers, please see T327330. PPelberg (WMF) (talk) 22:33, 24 January 2023 (UTC)
 * ...thank you for dropping by! You doing so was the reminder I needed to prioritize posting a status update about this work which I'll do on the project page before this week is over.
 * ✅: Edit Check Update: 27 January 2023 PPelberg (WMF) (talk) 19:32, 27 January 2023 (UTC)
 * @PPelberg (WMF), thanks for sharing all that! I reviewed the project page, phabricator tickets, and Figma mockups for desktop and mobile. I responded on Phabricator to some of your prompts there.
 * I think you're on the right track with who to reach out to — w:WT:Teahouse and w:WT:Growth Team features are good options, and I'd add somewhere more general like w:WP:VPI to bring in some folks who might have good advice but who we might not have thought to ask.
 * Regarding your questions:
 * If I'm understanding this right, you're looking to make sure that the mobile implementation of EditCheck doesn't make mobile editing annoying for experienced editors. Well, when I need to make an edit on mobile, my workflow (and that of most other experienced editors I know) is to just switch to desktop mode, since the mobile mode is so deeply lacking in functionality it's preferable to suffer through the tiny fonts/buttons of desktop on mobile. I know has been interested in mobile editing, so they might be able to offer more.
 * There are lots of things we might want to configure. We care more about referencing for certain types of articles, such as those on living people and those on medical topics, so we'd likely want a more aggressive implementation for those than others. Of note for you all, there's currently a tag for new users adding unreferenced material to a BLP. Noting this comment you made on Phabricator — Iterate on the "Citoid" edit card to include some kind of guidance that prompts people to reflect on whether the source they're considering adding is likely to be one other volunteers deem acceptable — we'll absolutely want control over the source list there, so that we can modify it as RSP changes, and ideally we'll want to be able to provide context/specific conditions for (sometimes-)unreliable sources, as it's far from just a binary reliable/unreliable switch. And we'll want to be able to tweak all the guidance text that shows up at the different steps.Going more broadly, I think the true wiki way would be to allow us to come up with custom filters (just as we currently do abuse tags) and provide custom feedback. This would probably mean leaving out some of the more advanced functionality, but I think the community would be able to come up with many good uses that'd be more effective than our current approach when we notice a persistent error of trying to expand guidance (leading to creep).
 * One other thought (for now): Many editors who are contributing uncited material are providing original research, so we'll want to identify and provide guidance to them. The Upload Wizard comes to mind for its approach to a similar problem. When you're contributing a local file, it asks you whether it's a free work, in a fair use category, or doesn't fit either of the categories above. If you choose the last option, it comes back with Please don't upload it as it's very likely a copyright violation. For here, the question we want to make folks adding information answer is "How do you know this?" If it's from a particular source, we of course want the source. But if not, we want to give editors a way to express that by saying "it's something I know from my personal experience". And we want to then explain to them why that's original research and why they need to either find a source for it, or if it's private information that doesn't have a public source, why they shouldn't contribute it.
 * Cheers, &#123;{u&#124; Sdkb  }&#125;  talk 06:47, 28 January 2023 (UTC)
 * First off, thank you for investing the time and energy into thinking about these prompts and making this thinking easy for us to understand and engage with, @Sdkb!
 * I'm going to respond to the specific points you raised in discrete comments so that we can more easily explore the distinct points you are raising here.
 * Before that, thank you for being patient with me.
 * Note: I've also seen the comments you posted on T327330 (thank you!); I'll respond there directly. PPelberg (WMF) (talk) 00:57, 11 February 2023 (UTC)
 * I think you're on the right track with who to reach out to — w:WT:Teahouse and w:WT:Growth Team features are good options, and I'd add somewhere more general like w:WP:VPI to bring in some folks who might have good advice but who we might not have thought to ask.
 * Excellent. You affirming the instincts we had leads me to think something like, "Okay, we're on the right track." PPelberg (WMF) (talk) 00:58, 11 February 2023 (UTC)
 * If I'm understanding this right, you're looking to make sure that the mobile implementation of EditCheck doesn't make mobile editing annoying for experienced editors.
 * While we're open to hearing input on the user experience, at this stage, we're primarily seeking to learn things like the following from experienced volunteers:
 * How can you envision this general approach going wrong? What fundamental assumptions/constraints might this project be at risk of "running up against"?
 * What logic/heuristic do experienced volunteers think ought to cause the reference check to get triggered? Thank you for responding to this in T327330#8566397; I'm going to follow up with you there.
 * What kinds of editors do experienced volunteers think the reference check should be enabled for? E.g. All logged out editors? All logged out editors *and* editors who have made fewer than (insert number here) cumulative edits? Etc.
 * How might experienced volunteers value being able to evaluate/audit how Edit Check is performing and the impact it is having so that they can make improvements to it?
 * What other kinds of checks can experienced volunteers imagine being useful? I see you shared a collection of wonderful ideas in T327330#8581224 (thank you!). Similar to "2.", I'm going to respond in Phabricator.
 * Note: we do NOT anticipate the in-editor Edit Check user experience to disrupt experienced editors because we:
 * Assume experienced editors are unlikely to use the visual editor on mobile to edit. See data
 * Assume experienced editors are likely to intuitively add references when they are adding new content to the wikis and thus, be unlikely to trigger the reference check
 * Think we might start by only enabling Edit Check for people who are logged out and have made fewer than, let's say, 500 edits or something like that. Exact number TBD. Although, writing this out led me to realize we need a task to hold ourselves accountable for deciding this: T329340.
 * PPelberg (WMF) (talk) 01:01, 11 February 2023 (UTC)
 * Responding to the numbered questions:
 * I think there is some risk for false positives. E.g. if the tool tells people to add citations for the plot section of a film, that'll violate w:MOS:PLOTCITE. There is also some risk for false negatives. This could happen if newcomers become used to the tool, and then start expecting it to let them know whenever they need to add a reference. Then, when the tool misses an instance, they might falsely think that they don't need to add anything because edit check didn't flag it.More generally, I think the difficulty of this project is that it's trying to communicate Wikipedia's policies/guidelines succinctly at points when newcomers need them, but those guidelines are both inherently complex and not designed to be machine-readable. Because of that, identifying when they come into play will be a challenge.
 * (See Phabricator discussion)
 * We always want even the newest of newcomers to be incentivized to create an account, so I would not want to see it enabled only for logged-out editors. Once the tool is mature, I could see it being enabled for everyone if it's good enough about not having false positives — but of course there would be an option in the settings for anyone to turn it off. For the beta stage, I think something like fewer than 15 days or 100 edits might capture most of the target user base. I think you should also make it an opt-in beta feature, though, since some experienced editors like myself are going to want to try it out to provide feedback.
 * This could be tricky, as our normal way of checking things is to look at edits, but this tool of course intervenes before an edit is made. I think trying it out for ourselves will be very helpful, as we'll be able to see how accurate its suggestions are.
 * (See Phabricator discussion)
 * Cheers, &#123;{u&#124; Sdkb  }&#125;  talk 18:42, 11 February 2023 (UTC)
 * Well, when I need to make an edit on mobile, my workflow (and that of most other experienced editors I know) is to just switch to desktop mode, since the mobile mode is so deeply lacking in functionality it's preferable to suffer through the tiny fonts/buttons of desktop on mobile.
 * Understood. I appreciate you naming the differences in what functionality is and is not available within the desktop and mobile editing experiences.
 * While we are aware of this, the Editing Team is not currently prioritizing work on feature parity.
 * I recognize that me saying the above offers no relief. Tho, it is important to me that, at a minimum, you know what the Editing Team is and is not focusing on in the near-term. PPelberg (WMF) (talk) 01:03, 11 February 2023 (UTC)
 * Yeah, certainly. I definitely didn't mean it as an accusation that you ought to focus more on mobile editing. (And indeed, I'm not sure how fruitful that might be — there are some tasks just inherently not well-suited to mobile.) &#123;{u&#124; Sdkb  }&#125;  talk 18:46, 11 February 2023 (UTC)
 * There are lots of things we might want to configure. We care more about referencing for certain types of articles, such as those on living people and those on medical topics, so we'd likely want a more aggressive implementation for those than others.
 * Understood and I feel energized hearing that you anticipate people wanting to be able configure the checks themselves!
 * But back to what you shared, two follow-up questions for you:
 * Would it be accurate for me to think that you'd expect to be able to configure the check based on the category in which the article someone is editing belongs to?
 * Can you say a bit more about what you mean by "aggressive implementation" in this context? Asked another way: can you share the kinds of 'rules' you'd imagine wanting the reference check to operate on and that you think volunteers would value being able to 'tune' based on the category an article belongs to?
 * PPelberg (WMF) (talk) 01:03, 11 February 2023 (UTC)
 * On (1), yes. We might also want to configure the check based on project tags (which add categories to the talk pages) or even Wikidata information.
 * Another way we might want to configure is based on article quality, which would also be reflected in talk page categories. For instance, we might decide to tone down the suggestions on a featured article, which has already received extensive human review.
 * On (2), I'm thinking of it in terms of the confidence level of the tool. So, for example, if the tool thinks there's an 80% chance that adding a reference is needed, we might decide that that's high enough that we want to display the check at a BLP page, but not if it's a normal page. &#123;{u&#124; Sdkb  }&#125;  talk 18:51, 11 February 2023 (UTC)
 * Of note for you all, there's currently a tag for new users adding unreferenced material to a BLP.
 * Oh, great spot. I've added this tag/filter (Special:AbuseFilter/history/686) to the Edit Check project page. Although, please boldly edit that page if I've linked to a tag/filter that differs from the one you had in mind. PPelberg (WMF) (talk) 01:05, 11 February 2023 (UTC)
 * Noting this comment you made on Phabricator — Iterate on the "Citoid" edit card to include some kind of guidance that prompts people to reflect on whether the source they're considering adding is likely to be one other volunteers deem acceptable — we'll absolutely want control over the source list there, so that we can modify it as RSP changes, and ideally we'll want to be able to provide context/specific conditions for (sometimes-)unreliable sources, as it's far from just a binary reliable/unreliable switch.
 * Great call; I'm glad you named the nuance and configurability such feedback would require.
 * I've added a note to T276857 (the ticket where we'll explore adding feedback of this sort) so that I can hold us accountable to revisiting this conversation when we prioritize work on it. PPelberg (WMF) (talk) 01:05, 11 February 2023 (UTC)
 * And we'll want to be able to tweak all the guidance text that shows up at the different steps.
 * Understood. PPelberg (WMF) (talk) 01:06, 11 February 2023 (UTC)
 * Going more broadly, I think the true wiki way would be to allow us to come up with custom filters (just as we currently do abuse tags) and provide custom feedback. This would probably mean leaving out some of the more advanced functionality, but I think the community would be able to come up with many good uses that'd be more effective than our current approach when we notice a persistent error of trying to expand guidance (leading to creep).
 * I feel inspired reading the above! Primarily because it sounds like you and us (the Editing Team) are seeing similar potential in this work…
 * Where – to double confirm – "potential" in this context refers to the ability for experienced volunteers to write custom filters/checks (e.g. similar to those you've listed in T327330#8581224) in ways that:
 * New and inexperienced volunteers will intuitively understand and be equipped with the tools/workflows they need to apply them
 * Experienced volunteers can audit and iterate upon
 * I'd value knowing if you sense any gaps between the potential you articulated above and what I responded with.
 * Note: I still think there is a lot for us to collectively execute and validate before I'd feel comfortable "promising" the potential we seem to both see, but yes, the Editing Team is definitely thinking about this first check as a proof of concept for an open-ended system that we can collaboratively shape and expand. PPelberg (WMF) (talk) 01:08, 11 February 2023 (UTC)
 * Yep, that's correct! &#123;{u&#124; Sdkb  }&#125;  talk 18:52, 11 February 2023 (UTC)
 * One other thought (for now): Many editors who are contributing uncited material are providing original research, so we'll want to identify and provide guidance to them.
 * Before inquiring more deeply about the Upload Wizard, would it be accurate for me to understand what you're describing above as the following?
 * In the event that someone declines to add a source, the edit check user experience ought to present them with next steps (that experienced volunteers at individual projects ought to be able to configure) that align with the best practices outlined in WP:No original research.
 * In the meantime, I've filed T329406. PPelberg (WMF) (talk) 01:09, 11 February 2023 (UTC)
 * Yep, that's correct! &#123;{u&#124; Sdkb  }&#125;  talk 18:53, 11 February 2023 (UTC)
 * @Sdkb, are you saying that if someone adds material without adding a source, then it's likely impossible for anyone to find a reliable source that could support the new material? That hasn't been my experience, but perhaps it varies by subject area.  For example, consider this edit that turned up in my (volunteer) watchlist the other day.  Would you call that "original research"? Whatamidoing (WMF) (talk) 21:11, 14 February 2023 (UTC)
 * Note: I still think there is a lot for us to collectively execute and validate before I'd feel comfortable "promising" the potential we seem to both see, but yes, the Editing Team is definitely thinking about this first check as a proof of concept for an open-ended system that we can collaboratively shape and expand. PPelberg (WMF) (talk) 01:08, 11 February 2023 (UTC)
 * Yep, that's correct! &#123;{u&#124; Sdkb  }&#125;  talk 18:52, 11 February 2023 (UTC)
 * One other thought (for now): Many editors who are contributing uncited material are providing original research, so we'll want to identify and provide guidance to them.
 * Before inquiring more deeply about the Upload Wizard, would it be accurate for me to understand what you're describing above as the following?
 * In the event that someone declines to add a source, the edit check user experience ought to present them with next steps (that experienced volunteers at individual projects ought to be able to configure) that align with the best practices outlined in WP:No original research.
 * In the meantime, I've filed T329406. PPelberg (WMF) (talk) 01:09, 11 February 2023 (UTC)
 * Yep, that's correct! &#123;{u&#124; Sdkb  }&#125;  talk 18:53, 11 February 2023 (UTC)
 * @Sdkb, are you saying that if someone adds material without adding a source, then it's likely impossible for anyone to find a reliable source that could support the new material? That hasn't been my experience, but perhaps it varies by subject area.  For example, consider this edit that turned up in my (volunteer) watchlist the other day.  Would you call that "original research"? Whatamidoing (WMF) (talk) 21:11, 14 February 2023 (UTC)

Commenting since I was pinged (thanks !). , if you're interested in my thoughts about mobile editing, I wrote a piece for the Signpost last month. There's some interesting comments on the related talk page from other Wikipedians. If there's anything you want to ask me, feel free. Clovermoss (talk) 15:20, 28 January 2023 (UTC)

I responded at the ticket and have found this discussion, which is perhaps a better place for it. Mathglot (talk) 00:13, 12 February 2023 (UTC)