Thanks so much @Excirial for your feedback. It’s essential we hear from people like you so that we avoid just the type of errors you’re describing. I wanted to start this thread to discuss one of the specific issues you raise. You write:
“For me using Huggle equals looking at the displayed diff while using keyboard shortcuts to navigate the edit queue. I wouldn't take note of any added queue icons or the ORES score….I am manually checking each edit so a machine evaluation is redundant.”
So it sounds like you generally examine edits in the queue sequentially; you don’t pick and choose based on Huggle’s icons or other clues. This is good feedback. You don’t say which filtering option you use for the queue. I’m going to guess it’s “Filtered edits,” which sorts by Huggle score? Is that right?
If so, then at the top of your queue you're seeing the edits Huggle's internal algorithm thinks are the most likely vandalism, followed by less and less likely candidates. So that would explain why going in order works for you. But what if I said there is research to suggest that ORES’s quality predictions (based on its “damaging” test) provide more reliable predictions? (I think this is true; @Halfak (WMF) can comment).
Assuming that’s so, then a queue sorted by ORES score could help you to more quickly eliminate the worst edits, putting your efforts where they'll make the most difference. What might work for you, I'm thinking, is if we simply offer one or more alternative sorting options based on the ORES scores. Then you could try them and decide which options you prefer. Would that be interesting, do you think?
It sounds like you find Huggle's queue icons irrelevant. I wonder if other users feel differently? There must be a reason why this complex system evolved? I’d be interested in hearing from users who do look at these icons for clues. What is useful, what not? If there are such users, I’m inclined to believe that implementing the ORES icons could still provide valuable data for some. (Though I assume we’ll let users turn them on or off as they prefer.)