Topic on Talk:Edit Review Improvements/New filters for edit review

For accessibility see also : https://phabricator.wikimedia.org/T91201

I don't use Recent changes, so I'm speaking just from the perspective of using this new functionality on my Watchlist, though the functionality I'm going to suggest would be equally useful for Recent changes.

I'm not going to switch between a bunch of different filtering options every time I visit my Watchlist. I just want one stable view that I use almost all the time. There are a ton of new options that I'd like to use, but since I'm not going to turn a bunch of different options on and off every time, I need to use them as highlighting, not exclusion filters. However, there are only 5 preset colors that can be used for highlighting (and the ability for these 5 colors to be mixed across filter categories does not solve the problem).

There needs to be a different (or additional) way for all these useful new items to be marked. To the left of the edits, where the colored bullets now appear, I'd like to see "Q:1" through "Q:4" (possibly outlined with a box), corresponding to "Contribution quality prediction: Very likely good" through "Very likely have problems". I'd like the Q:num (and border box, if any) to be colored as per my selected highlight colors. If something were Q:4, there wouldn't be additional subset indicators for Q:3 and Q:2 — that's an implementation detail, not a useful UI feature.

Then, if enabled, next to the Q indicator, there'd be "I:1" through "I:4" for the User intent predictions, again colored by my selected highlight colors.

Then, if enabled, next to the I indicator, there'd be "X:e", "X:l", "X:n", and "X:u", for (eXperience:) Experienced, Learners, Newcomers, and Unregistered, again colored as selected ("Registered" would not be useful except as a Boolean alternative to the first three categories).

For the highlighting of the text itself, what would be most useful to me would be if that instead of doing confusing color-mixing, the text were highlighted with the single color from the "worst" of those three categories. So if I had green, yellow, orange, and red selected for each of those (and probably a single color picker across all three would make sense if you're not going to offer a larger palette and/or arbitrary RGB picker), a Q:1 I:1 X:e edit's text would be in green, but Q:1 I:1 X:u would be in red.

Of course, unregistered users' edits always having their text in red, overriding quality and good faith predictions, wouldn't be very useful, so what I'd like to see would be a checkbox on each of the those three categories as to whether them having the worst value would override the text color or not. I personally would turn off the text effect for X, so Q:1 I:1 X:u would be green, but Q:2 I:1 X:n would be yellow. Highlight colors would still be applied to the X:val text (and outline box, if any), however.

Thank you for your feedback, Dan Harkless.

The filters have been designed to filter, and then highlight the filtered results. Filtering helps to focus on a particular type of edits, or to focus on a particular user group, to increase efficiency.

You can define your personal default set of filters on the Watchlist, using bookmarks. Bookmarks can also help you to switch quickly from one saved set to another.

The alternative you propose sounds more like an advanced tool, for a very specific need. Maybe you should create (or ask someone to) create gadget that would add the indicators, by checking CSS classes used on the results? I don't know how familiar you are with CSS and how far will it work, but, as an example, that code will add "X:u" before all edits made by unregistered users:.mw-changeslist-user-unregistered:before {content: "X:u"; margin-right: 20px;} You can copy/paste it in your common.css to try it.

Thanks for the reply, Trizek. Yes, I understand the purpose of filtering and highlighting. My default Watchlist setup using the beta UI uses the filters Unseen changes, Changes by others, Human (not bot), Page edits, and Logged actions to limit it down to just what I need to see.

And yes, I had already discovered the filter bookmarks, and tried to use them to set up two views, a "Group results by page" version and a non-grouped version (since there is not yet an option to expand all groups, and since the non-grouped view is missing the "cur" diff links), but unfortunately I discovered the bookmarks don't save the settings from the drop-down boxes.

However, even the ability to quickly switch between different filter presets does not address the serious limitations of the highlighting system. I want a view of my Watchlist in which I can see all the information that I'll utilize to determine which edits I'm most motivated to examine first, including the edit summary and related info, and the three different predictive indicators of edit quality. Having to switch back and forth between a bunch of different views would not work for this purpose, since it'd require me to memorize what's on one view in order to compare it to information on another view. And I'm not going to deal with my Watchlist in a robotic fashion, first always looking at all "Very likely bad faith" edits, then switching filters so I can see "Very likely have problems", then back for "Likely bad faith", and so on.

And I disagree strongly that the ability to see all this information at a glance only makes sense as "an advanced tool, for a very specific need". The ability to see the different edit quality indicators would be potentially useful to everyone. Indeed, less advanced users would probably benefit even more than advanced users, since they would not yet have learned what sorts of edits the machine-learning system tends to qualify as "May have problems" vs. "Likely have problems" vs. "May be bad faith" vs. "Likely bad faith", etc., nor would they have started to memorize what certain color combinations look like when mixed.

Another example where a UI like I describe would be useful to non-advanced editors would be for individuals with colorblindness (between 7% and 10% of the male population in many ethnic groups). Two or more of the 5 available colors are difficult or impossible to distinguish from each other in several types of colorblindness (e.g. see http://www.color-blindness.com/coblis-color-blindness-simulator/), and that's not even considering the case of when the system mixes multiple colors together.

As for your CSS suggestion, it hadn't occurred to me that a usable display of the multiple edit quality indicators could be achieved that way; thanks for the example code. I've never had a need to delve into the user stylesheet feature on Wikipedia, as existing preference settings and gadgets always allowed me to achieve what I needed. However, if functionality like this doesn't end up getting added for everybody, I may look into that.

I discovered the bookmarks don't save the settings from the drop-down boxes.

To which settings are you referring to?

I disagree strongly that the ability to see all this information at a glance only makes sense as "an advanced tool, for a very specific need".

I may haven't been clear enough (and my English is not always used accurately): I don't consider view all changes at a glance as a feature for advanced users - that's the basic use of Watchlists. I was describing your solution using additional tagging as "an advanced tool, for a very specific need".

We have tried to take most cases we know into consideration (based on experiences of Watchlists users we have interviewed and user feedback). How did you handled all the edits in you Watchlist before the filters and the highlighting option exists?

Concerning colors used for the highlights pass the AA color definition of the WCAG 2.0 at minimum. The palette is here: https://phabricator.wikimedia.org/M82. Concerning combinations of colors, the most challenging one (blue links on a blue background) is AA compilant. I'm looking for more information now. Highlights have a small dot as a reminder on the left, which is not concerned by color combinations (if two highlights match for one entry, you will have two dots).

As I said, I was trying to make bookmarks for two versions of my standard Watchlist view, one using the option "Group results by page", and the other not. Poking around a bit after that failure, it appears that none of the settings in the drop-down boxes below the "Filter recent changes (browse or start typing)" box are saveable in bookmarks; instead, those are all "global variables".

Normally, non-advanced users don't use things that'd be described as "an advanced tool, for a very specific need", which is why I generalized from "advanced tool" to "advanced editor".

Before the beta version, I found the Watchlist to be a bad UI that needlessly wasted a lot of my time. There was no way to hide edits that'd I'd reviewed, so it was necessary to scroll through the entire history to search for things that were still bold. IIRC, there didn't use to be an option to group changes by page, so I found the "Expand watchlist to show all changes, not just the most recent" option to be unusable, and thus had no "cur" diff links and always had to view the history of every single page to see whether the change mentioned in the Watchlist was the only one I hadn't seen. I didn't use options like "unregistered users" or "probably good edits", because as I've said, having to keep changing filters around to visualize this data is not a usable approach, in my opinion.

Thus, the new version is a big improvement, but one major problem remains unfixed. The introduction of the ability to highlight rather than filter out promised to allow me to finally use the (now much expanded) edit quality predictors, but unfortunately, the 5 preset colors (+ mixing) is entirely insufficient to visualize the three independent, four-valued predictors.

And yes, I figured that some attention had been paid to accessibility when the 5 colors were chosen, but not enough, it seems to me. Per http://www.color-blindness.com/coblis-color-blindness-simulator/, 2 or more of the 5 primary colors are difficult or impossible to distinguish for users with Protanopia (~1.3% of males), Deuteranopia (~1.2% of males), and Monochromacy (~1 / 30,000 males and females). And again, if you consider the mixed colors, people with other types of colorblindness would be affected as well.

While it's good that the mixed colors problem is partially addressed by the left margin circles, a single 5-value display is not enough to visualize the 64 possible combinations of the different edit quality indicators.

While some portion of Watchlist / Recent Changes users would no doubt find a "Q:2 I:1 X:n"-type indicator to be too much information (or "an advanced tool", as you put it), that's fine; none of those would be displayed / highlighted by default, and each indicator would be individually enableable. On the other hand, for the not-insignificant percentage of affected MediaWiki site users with colorblindness, for users who are partially or completely blind and must use screenreaders (~3% of the population), and for normally sighted users who don't see why they should be arbitrarily forced to pick just one of the three (partially related but) independent edit quality predictors to visualize, I think implementing individual text-based indicators (with colored text, or for better readability, black text with a colored outline box and/or background) similar to those I propose really is called for.

Seems interesting, but the complexity doesn't seem worth it. Recent changes and presumably watchlist pages are incredibly cluttered , with massive amounts of overlinking and text. The result list probably desperately needs to be considerably improved before any extra options are added to it.

Also, not sure this is an issue or bug, but the filters aren't really that straight forward, e.g. it contains stuff like "class =mw-changeslist-damaging-likelygood mw-changeslist-damaging-maybebad mw-changeslist-goodfaith-good", for a revision like [1]. This means that it would need to be something like "Q2Q3i1"or "Q2-3i1" because there might be overlap in the scores. A user As for user experience, the best place to place that information is probably right next to the username, e.g. something like:

.mw-changeslist-user-learner .mw-userlink:after { 
    content: " X3 ";
    background: #0E0A0A;
    color: white;
    border-radius: 25px;
    font-size: .83em;
    vertical-align: super;
}

Even if users are color blind, they can still select individual highlights rather than several at the same time, and even if that is deemed cumbersome, hovering over the bullets or the highlighted filters themselves reveals a tooltip with the applied filters.

A color picker would fix some issues, but would add more complexity to the interface and would potentially require setting up the different colors for different wikis for those who use many. As an example, the highlighting requires excessively bigger dropdowns than previously needed.

---

Nothing developers can create will ever fit every use case, e.g. blind people can't even see any colors to start with, and X1Q1 is unintelligible to them, and certain cultures deem certain colors offensive, so an extreme view might be that those would need to be completely hidden from all interfaces.

Agreed, developers can't address every use case. However, I believe I've provided credible arguments and statistics to illustrate why a UI like I propose would be important to a significant percentage of Wikipedia users, e.g. for accessibility reasons. Yes, I'm aware that blind people can't see colors, which is one of the reasons I'm suggesting a text alternative. No unexplained "Q:2 I:1 X:n"–type strings would suddenly be read out by screenreaders. Just like the colored highlighting, these indicators would be disabled by default, and blind users would be just as capable as sighted users of reading accompanying help information before enabling them. I'm not aware of any cultures that deem certain colors offensive in all contexts (including web UIs), but in any case, comparing such an extreme view to what I'm proposing is a straw man argument.

With regards to your "Q2Q3i1" or "Q2-3i1", as I stated in my original feedback post: "If something were Q:4, there wouldn't be additional subset indicators for Q:3 and Q:2 — that's an implementation detail, not a useful UI feature." And yes, I had discovered the tooltips on the circles, but having to hover over every set of circles in succession to find out what they indicate (and having to memorize that information in order to compare different edits' quality predictors) is a poor design from a data visualization standpoint, a waste of users' time, and fails the goal of letting editors see at a glance all the criteria that are useful in deciding which edits to review, and in what order.

However, I believe I've provided credible arguments and statistics to illustrate why a UI like I propose would be important to a significant percentage of Wikipedia users, e.g. for accessibility reasons. 

Yes, there are certainly credible arguments. But in the end what this thread is asking is for an additional option. Additional options mean additional interfaces, additional complexity (for end-users and developers), and potentially additional software maintenance difficulty, issues and bugs. As an example, there are probably more than 10000 (if not millions) of possible combinations of filters.

 that's an implementation detail, not a useful UI feature

Not really, it just points to the fact that much like humans, machines can't completely tell if an action is perceived as unwanted. It would also disregard that the other use case of filters is to make it easier for editors to find and help new users who may be having problems making edits. As noted previously, in some cases this might be a result of a software issue, rather than expected functionality. Only developers would know for sure.

poor design from a data visualization standpoint

It is worse design to continually clutter an already cluttered interface with added symbols, especially if it is made optional, as explained above.

Probably most people who have looked at the Mediawiki preferences would be utterly appalled at how many options there are. It likely started out innocently with someone asking for just "one more optional feature".

The great number of filter combinations seems to me like a reason to allow those combinations to be visualized quickly and meaningfully, not the reverse. I certainly won't dispute that it would add interface and code complexity, though.

I don't think my suggestion "disregards" the use case of editors looking for new users that need help with their edits; indeed, it would make it easier for them to do so. And yes, the very fact that machines can't identify problematic or bad-faith edits with the accuracy of a human is why I want all of those predictors of quality to be available side-by-side, so editors can use that information as they best see fit, on a case-by-case basis.

I can't agree with your assertion that adding more options is worse than forcing an interface to have accessibility problems for a significant percentage of the population. The genie is already out of the bottle on the "lots of options" thing, and most of what I've proposed could be relegated to the "Advanced" box or the Preferences → Watchlist settings rather than being part of the main filter interface.

I've never understood the position that too many useful options is harmful, especially since options can always be segregated into Basic and hidden-by-default Advanced categories (and since it's easy to implement the ability to search for options whose associated text contains a user-entered string). I think a significant portion of users who are likely to use Watchlists and/or Recent Changes are the sort that aren't scared by lots of options. I would hope the MediaWiki authors wouldn't go the route of the Mozilla developers and decide against the wishes of a core subset of their userbase that plentiful options and ease of customization are inherently a bad thing.

I can't agree with your assertion that adding more options is worse than forcing an interface to have accessibility problems for a significant percentage of the population

Moderate accessibility should not be optional it should be default. Any options beyond moderate accessibility would certainly benefit from being general and impacting many pages with few exceptions for unique cases. There are alternatives, e.g. there could simply be different types of bullets so color blind users wouldn't have to guess that the colors are different.

I've never understood the position that too many useful options is harmful, especially since options can always be segregated into Basic and hidden-by-default Advanced categories

It is like attempting to shove many more tools into a swiss knife, it can't work without considerable re-engineering. In fact, the new recentchanges is already suffering from options, the page is slower, it can't even work without javascript, and has serious performance issues.

Anyway, perhaps the developers and designers of the tool will feel differently.