Thread:Talk:Article feedback/Justin Bieber article not complete?/reply (3)

Yes, that's an interesting point partly related to my previous comments on trending topics vs. quality. We definitely need to think of how to cope with these effects. I am persuaded the solution should not apply to the rating collection itself but to how we calculate aggregate scores: for example we should filter out suspicious patterns that indicate that an article is being rated as a function of the topic's popularity in the media. Having more data on multiple ratings by the same users or IP addresses should also allow us to study the reliability of ratings. Unfortunately, because we are still using a small sample of articles, we have very little data on people who rated multiple articles.