Topic on Talk:ORES

Is it possible to rebuild a 'reverted' model to finetune it or does it finetune itself and with the help of new labeled edits? Thanks.

It gets fine-tuned with new edits. Is there a problem with one of the reverted models?

Thank you for your reply. Not a problem but "roughness" sometimes, so to speak (which is to be expected as the docs says) . But I was not aware that it gets finetuned itself with new edits. In any case I hope we can finish the damaging/good faith labeling on eswiki soon to have better heuristics. Regards.

So it doesn't automatically get fine-tuned with new edits, but we can always retrain the model with new data. If you'd like us to give that a try, we can add it to the backlog. It's not too difficult to do.

I'm leaving that to @-jem- because his bot, PatruBOT, is using the currently avalaible data from ORES at eswiki. If he thinks it'd be worth requesting a retrain of the reverted model for his bot to be more accurate until the damaging&good faith campaing is not active, his choice, as long as the labelling campaign do not reset and all our job gets lost.

Thanks, @MarcoAurelio. Well, as the other campaigns may still be some weeks or months ahead, and we still have some false positives in the reverted model, if the model can be improved with not so much work, I (and the eswiki community) will appreciate it. Or maybe you can give me some hints about fine-tuning the bot with ORES information beyond the use of the reverted probability. I'll keep on reading. Regards.

As long as we don't loose the already-done tagging of edits (we're at 70% now) I have no objections to rebuild the reverted model. Maybe we can exclude es:User:PatruBOT from the rebuild so its false positives do not "contaminate" the results?

When PatruBOT commits a false-positive, is the good edit generally restored via a second revert? If so, we're likely already excluding them from the model. Whenever a reverted edit is later restored by someone other than the original author, we exclude that edit as a revert example for the model.

I think most of those good edits are restored when people check their watchlists, but pages with no active watchers will remain without the edit. But the same can happen with human mistakes, so I think things are acceptable as is. And if I can help you by explaining (privately and away from vandals) how PatruBOT works, I'll be glad to do it.