User:Jeblad/subjective voting

Subjective voting is an idea to use subjective logic (Dempster–Shafer theory; probabilistic logic, probabilistic soft logic, Markov logic networks, subjective logic, evidence-based subjective logic) to make a kind of voting system for deletion candidates. The method use a set of criteria, preferably as a set of prepared claims or propositions. These are the "set of possibilities under consideration". Users will then cast votes on whether these claims are true or false, by choosing among other prepared claims or opinions. These are a "subset of the frame of discernment".

The propositions and the opinions may or may not point to evidence for the correctness. Usually it would be a GoodThing™ to use such evidence, but the method as such does not depend upon it. It could be interesting to use evidence as a means to add prior belief to opinions.

Propositions and opinions
Assume we have a proposition. Then we have (at least) two opposing opinions about this proposition, we have a belief and a disbelief. This is called a binominal opinion. It is not obvious, but it is also a third opinion for those that does not commit themselves to any of the two previous beliefs. This uncommitted belief change how much belief is necessary to reach a previously set threshold. It is not enough that the belief is just slightly bigger than the disbelief, it must be bigger that the disbelief and uncertainty combined. How much bigger depends on the different models for subjective logic.

It is tempting to make a system where the voters only mark the propositions they like, but such a system would miss the uncommitted belief, thus tend to choose a single winner on raw count.

Now assume we have two propositions. Claim (A) has a sufficient condition, and claim (B) has a necessary condition. For simplicity assume they are opposing claims. Each of those two has their own pair of opposing opinions. If the voters put too little belief in claim (A), then claim (B) might win. If the voters put enough belief into claim (A), then it does not matter what claim (B) is, claim (A) wins anyhow as (A) has a sufficient condition. See Necessity and sufficiency for details.

Within specific domains sets of propositions could be prepared, like the guidelines from w:Category:Wikipedia notability guidelines, and then just included on a talk page or other discussion page by referencing the set somehow. One way to do this is to make a JSON structure, and then reference the page. Another way to do it would be to make templates.

Trust and reputation
The votes don't have to be given equal weights, new editors can be given less weight than an ordinary user, and an admin even higher weight. This is interesting as on-ramping of new editors, but it can also give a wrong signal to the community.

One interpretation of non-equal weights are as trust. For example the admins are assumed to be more knowledgeable than other users, thus an admin as higher trust. During voting we can use this and give the admin a higher weight. If we give the user too much weight, then the voting process will start to fail as other users starts to distrust the outcome.

One quite interesting method for assigning trust for a specific user is to calculate covariance between previous votes and previous outcomes. If the user has a high covariance with outcome on his previous votes, then assign the user high trust. Note that covariance will be a covariance matrix for multinominal cases, with no entry for uncommitted belief. That is the user votes are compared to the outcome.

Another method is to use a reputation system to calculate a trust metric. The reputation could be calculated as part of another voting process, for example the votes cast during admin nomination, or a gild system. A gild system is based on users joining a thematic guild, and thus being assumed to have special knowledge about the given theme. On Wikipedia there are projects, like w:Wikipedia:WikiProject Astronomy, and the projects has members. The members within a project is a guild. On English Wikipedia there is one named guild and it is w:Wikipedia:WikiProject Guild of Copy Editors, it is somewhat different from a guild in this context.

User interface
The user interface is kept as simple as possible. Radio buttons in a row for each proposition, and the opinions as headers above the buttons. This works well as long as evidence isn't included. The action is given as part of the proposition, and the sufficiency is shown by bolding. It should probably be additional flyover help in a real interface. A real user interface could be very different.

Propositions should be falsifiable, that means that for deletion processes it is very difficult to formulate a condition that imply deletion. To get around this there can be a default action to delete if all propositions fails.

One way to show evidence is simply to list them below each table row.

For users without Javascript to be able to vote there must be a submit button somewhere on the page.