Topic on Talk:JADE

Jump to navigation Jump to search

Judgment Bold-Revert-Discuss vs. optional consensus

Adamw (talkcontribs)

@Halfak (WMF) and I identified a design assumption in IRC last week, which has been implicit in a few of the other threads here, especially in "Judgments, Endorsements, and Preference" below, but is worth surfacing as its own issue. I'd like to open this for more comments.

In the current design of JADE, there can only be one "correct" judgment per wiki entity. Most entities will simply have one editor making one judgment, but in the cases that multiple editors supply a judgment, the second, third, etc. editors will be overwriting each other's judgments as the "correct" answer. This follows the BOLD,_revert,_discuss_cycle on English Wikipedia. @Halfak (WMF) has pointed out that this is the best process for the problem domain, that we want to encourage consensus.

An alternative workflow would be something like the Wikidata model, in which multiple judgments on a wiki entity default to having equal "rank", and it would be optional to go through a consensus process. Any editor could mark one or more of the judgments as "preferred", with or without discussion, or the editors could leave all judgments as "normal" rank and we would consider the set of judgments to be unresolved and equally valid.

I think there are two questions here: what is the preferable workflow for the various use cases in which editors add JADE judgments, and what are the best requirements on the data in order to support algorithms and reasoning using the judgments?

EpochFail (talkcontribs)

I think this discussion of the "Wikidata" model misses that it has BRD as well. In this case, "rank" is part of the data model itself. There's still one true representation that is identified via a consensus process. But with "rank" a more accurate reflection of a messy reality is represented in the data model itself. If we wanted to represent messiness like this, it might be better captured in JADE's schema itself -- depending on the type of judgement.

E.g. if a user decides that one judgement should be ranked "preferred" while another should be ranked "normal", they are essentially marking one as best and another as "safe to ignore". You might show up and disagree with how they have ranked and revert them. Preferably, you'd then discuss your differences in ranking.

I think when it comes down to it, I want to know why having ranks makes any sense. What information does it capture that we need to capture here? In what case, do we want one judgement to be "preferred" when another is simply "normal"?

BTW, I'm Halfak (WMF) accidentally made this edit through my volunteer account.

Keegan (WMF) (talkcontribs)

Is there a compelling user story that would have a ranking/preference system? How likely is this to be encountered in the wild?

I think the simple BRD approach is fine, with the one "true" judgement that can be revised further by others. If I could think of an imperfect analogy, it'd be kind of like File History on Commons. The latest upload is always displayed as the canon version, but previous revisions are still visible/available. Does that make sense?

EpochFail (talkcontribs)

Good Q. Here's the thread where I originally brought up the proposal for "preference": Tzw0uv2bucrdprm4 It discusses a use-case.

I think that generally, we should allow people to disagree with consensus -- much like what happens in a straw poll.

Adamw (talkcontribs)

I can buy this argument. I was thinking it would be more simple to a user to "make a judgment, ignore what else is there as you wish" rather than "make a judgment, yours will be the new authoritative judgment and will overrule the others". But I know I haven't had my fill of wiki Kool-Aid yet ;-)

Keegan (WMF) (talkcontribs)

In that case, I still think limiting it to one preferred judgement is what we'd want to go for. People are welcome to discuss it, but splitting it off into rankings and separate judgements seems unnecessary IMHO.

Reply to "Judgment Bold-Revert-Discuss vs. optional consensus"