Thread:Talk:Article feedback/Please stop/reply (60)

Hi Wasbeer,

rating bias for/against the topic of the article is definitely a major issue with any open rating system. It's important to note that the dashboard is a very particular query on the data, i.e. the 24 hour time window with a 10 rating threshold, which is going to produce its own odd biases. I'm not convinced yet that the current dashboard queries are useful.

One could cut and weigh the rating data in different ways to correct for bias. For one thing, it may be the case that within the submitted rating data, certain rating behavior is more or less predictive of rater bias (e.g. all 1s vs. a more balanced rating; self-identified expertise vs. no self-identified expertise). It may also be useful to take into account the overall behavior of the rater (e.g. just one article vs. multiple articles).

Pretty much every community-driven rating system applies additional algorithms to produce the total rating, and so far we're taking a pretty simplistic approach in surfacing this data. For a project like Wikipedia, we think making the data as openly available as possible will be the best way to gain a collective understanding of what the strengths and limitations of the current approach are, and to continue to iterate on the overall system based on what we learn.