Is the pilot over? We still have the category on en:WP and people are still adding items to it.
the AFT can still be activated manually on individual articles via the AF category. Ratings for the 100K articles are tracked independently. When the test phase is over activation via AF categories will be disabled.
Hi, I was asked by a wiki site owner to change the text in his article feedback. By default the article feedback shows the text headings: Page Trustworthy Objective Complete Well-written but the wiki site owner wants different text than these. I searched the ArticleFeedback php files but couldn't find anywhere to edit these. Is this done through toolserver or the dashboard?? Please advise because the instructions at are not clear on this. If you know how to do this, please reply with detailed instructions. Thank you.
Sorry, your intentions are good, but this article feedback tool is, to put it simply, a bad idea. Please remove it.
Could you elaborate on why do you think it is a bad idea? I believe a lot of people like the tool, and some people are also requesting it to be enabled for Wikipedias in other languages.
I will try, but English is not my native language, sorry. I hope I do not come across rude, that is not my intention.
The system is based on the false assumption that people's opinions are actually valuable, even if that opinion is simplified into a rating of 1-5 stars. Their opinion would be more valuable if they just wrote the pro's and con's on the talkpage instead of using the pro's and con's to determine a score via an undisclosed process and then give an appropriate amount of stars.
The system discourages users from using the talkpage. Giving a bad article a 1-star rating is easier than fixing it yourself, tagging it or explaining what should be improved on the talkpage, but far less useful. People who see a bad wiki article have two options: fight or flee. This box is just another way for people to complain and leave, and even gives them the impression they did something useful.
Basically asking for a rating is asking the wrong question. Compare the answers that you might receive with these two questions:
- How many stars would you rate this article on a scale of 1-5?
- What made you decide to rate this article in this way?
The answers to the second question are much more interesting.
Usability/interface problem: the location of this box is wrong. It should not be shown in the articles themself (do not trust me, ask an usability expert!). The box is much bigger than it needs to be, and very ugly.
You made some good points about the hit'n'run. Not to mention that only IMDb has a working rating system (calculating the average technically, ignoring beginners, etc). I left some thoughts on a proper rating extension here. --Subfader 21:13, 7 June 2011 (UTC)
"You made some good points about the hit'n'run. Not to mention that only IMDb has a working rating system (calculating the average technically, ignoring beginners, etc). I left some thoughts on a proper rating extension here. --Subfader 21:13, 7 June 2011 (UTC)"
I'm one of the people who finds the tool (and its results) completely useless. As I've said in another thread - I've seen well written, well-sourced articles rated very low because 10-year-old girls don't like a character an actor plays on TV, and I've seen mere stub articles consistently given high ratings because people love and remember their work from 50 years ago. However, I wanted to chime in on the discussion I see emerging here about the IMDb rating system. This is something I've thought about before. If people absolutely INSIST on keeping this useless feature, then there should at LEAST be a 10 vote minimum before the votes show up (like there is on IMDb). This would at least provide something of fighting chance of there being a somewhat impartial vote result before the votes show up on the page. As it is now, the ratings tool is just another (and now sanctioned) way for vandals to operate anonymously on Wikipedia.
Since the excessive size of the rating box was mentioned also on English Wikipedia (see this topic), I opened the following request on Bugzilla:
- Bug 29303 - Improve layout of ArticleFeedback tool
@Subfader: Thanks! I agree; IMDb has had a long time to figure out which rating system would work best, and instead of inventing the wheel again we should learn from the mistakes they have made in the past.
@Helder: Good work! Because you included a link to this discussion in the bugreport we can try to give suggestions for a 'compact' version of the design here.
Maybe it is a good idea to move the ArticleFeedbackTool to the talkpages. The "normal" article feedback should also be on talkpages, not on the articles themselves...
People who give 4 stars only send the message: "I like the article this much". Instead we should ask them to tell us why they decided to choose that rating: what did they like about the article, what did they dislike, what should we improve to turn the 4-star rating into 5 stars? If we know that most people rate an article 4/5 for "completeness" that fact alone does not help us. We want to know what they felt was missing from the article, but we force them to use a 1-5 star system without a textbox where they can leave a comment.
I think we lose valuable information if we ask the people who rate the article to turn the pro's and con's list they have in their head into a rating of 1-5 stars: an actual textbox where they could express their opinion would be far more flexible and give them the opportunity to be a lot more specific and helpful. But we already have that, its a talkpage. So I suggest moving all the boxes to the talkpage. Ofcourse, this means there will be a bit less input and a lot less eyeballs on the article feedback tool. On the other hand I think you might get a bit less vandalism and more people who take the time to make a serious and informed rating.
Another problem is that it slows down page loading quite a bit. Maybe we can test it with software like Yslow.
The sentence "Your ratings have not been submitted yet" appears in green, a color that tells me the exact opposite of the message.
That sentence and the arrow icon in front of it should be removed imho (if we can not move all the article feedback boxes to the talkpage) because they make the box way too big and I think they do not have the positive effect they are supposed to have: the people who fail to see the huge blue button will also be unable to see the small green sentence, probably because they need to scroll down.
Its better to put the stars and the submit button next to eachother if that fits so that people see them at the same time when scrolling down.
The redesigned interface is less cluttered because I removed the trashcan icons (they should be added back in) and the sentence that warns people they need to submit their rating with the green arrow (because you need to have scrolled down enough to see the submitbutton in order to see the warning and the warning is smaller than the submitbutton) and the icon behind "View page ratings" (no longer necessary because I changed the position of the text).
After I made the redesign I realised its probably better to swap "Objective" with "Trustworthy". Ofcourse you can move the "expert" checkbox to the bottom and the submitbutton to the top if you prefer that. Is it possible to replace "What's this?" with "Help" or "Info" or "How to rate?" or "Info about rating"?
Ofcourse, this is a very ugly and messy image, but you get the idea.
The endresult looks something like this. The fontsize and styling of the links on the lefthandside has changed slightly.
I think we can use this as an example:
The image contains a mistake, the text on the right side should be black instead of gray.
At the bottom I made two extra versions of the "expanded" version for "experts", with or without a solid line in between, I prefer the one on the bottom.
After I made the redesign I realised its probably better to swap "Objective" with "Trustworthy".
Why? Because of the lenght of the words? (If so, consider this comment about the translation to other languages).
I completely agree with Wasbeer's assessment, and reiterate that the tool provides readers a lame excuse for not posting an actual feedback comment on the talk page. The box could possibly be made much more useful if an extra comment field were available; at least it would provide encouragement to justify a rating.
I want to add that I feel that most casual readers do not care for citations at all, so if you are asking whether the page is trustworthy but by that mean to ask whether the article is well sourced you will most definitely get the wrong or a random answer. It also does not consider that outside readers are usually not familiar with our strict sourcing policy, and what they might find well sourced may not be so for us – in fact, we as editors will know best whether an article is sufficiently sourced or not.
The box could possibly be made much more useful if an extra comment field were available;
Thank you Wasbeer - you echo my thoughts exactly on article rating, but if this is going to be pushed on us at least it's better than the thumbs up/thumbs down style "assessment" that other websites provide. This being said, it was a great idea to plug the "did you know you can edit this page" message in at the end of the review. Hopefully this will encourage new editors. Themfromspace 20:12, 31 August 2011 (UTC)
It would also be fair to say that an equal amount of people dislike the tool.
Nope, the people who dislike this tool outnumber the people who like this tool afaik.
There is no data to support either of those conclusions, guys. Sorry to put a damper on this, but we're still collecting and collating the information.
While I appreciate and hear your opinions, we are beholden to the will of the entire community - who has charged the Foundation with increasing the overall quality of the articles on Wikipedia. Part of completing that task is to develop a mechanism whereby quality can be measured - and this is one of our experiments.
True, I just made that up, hence the "afaik". But as long as you have not proven < 50% of Wikipedians like this tool we can assume that the actual percentage is a lot lower than 50%.
The intention of developing a mechanism whereby quality can be measured is an excellent idea, but this articlefeedbacktool on every article is not the way to move forward.
That is why I try to change the direction now, before everything is set in stone.
Is it possible to reach consensus about the location of these articlefeedbacktool boxes? Can we please move them to the talkpages?
If we move all of them to the talkpages the majority of Wikipedians who does not want to use them has a lot less to complain about and the people who want to use them will still be able to.
Given that you have worked on this tool, and that you list it on you talk page as a feature you have worked on, I believe you should recuse yourself of any discussion of this feature. You have, I assume, invested much time in it, and thus this is something more emotional for you than logical. Please recuse yourself from discussion of this feature due to this conflict of interest. (Copying to talk page). --Timl2k4 06:11, 8 July 2011 (UTC)
"... beholden to the will of the entire community ... " ... that's interesting. Of course, you will never know the will of the entire community, you will only know the will of a very small sample -- which you already have. I suggest you go with it. Move that rating box as you "community" has suggested. It really, really does NOT belong on the article page button the talk pages.
Anyway, good for you, for trying!
"While I appreciate and hear your opinions, we are beholden to the will of the entire community"
Exactly. And given that other communication channels do not seem to be working very well, I am now using that other "feedback tool", the one that appears at the top of every page a couple times a year with Mr. Wales' mug on it.
I feel sorry to have donated actual money in the early years of this project, seeing how this is now going to financing the Wikibureaucracy's pet projects and other nonsense "the community" have neither asked for nor supported in any significant manner.
I will not be making further donations in the future. It will not make any impact on the path this project is going to take, but at least my conscience will be clear.
Just so you know.
I agree. This article rating things are starting to become annoying and they are cluttering up the encyclopedia. It's absolutely unclear if they actually help improve articles. Good in theory, well executed, but bad in practice. Get rid of them. 184.108.40.206 05:22, 17 June 2011 (UTC)
Earlier I wrote: "I am not very optimistic about the value of the ratings in general but especially those of users who did not log in, I fear people will base their ratings on the subject of the article and not on the article itself (e.g. people who like Bieber a lot will rate his article high and people who hate him give 1 star)."
If I look at the dashboard I see this:
"Born this way" is an album by Lady Gaga. Bieber and Gaga are probably the two most hated artists on this planet. Now look at the other article, the German guy called "Dirk". He was in the news because he won a championship-thingy in basketball (I am not a fan), which means some people hate him and some other people love him.
The articles are rated the lowest not because the articles are shitty but because people base their rating on the subject of the article and not the article itself. Go see for yourself.
Now look at the best articles. Among them is en:Lol@souffs, an article that is currently considered for deletion and obviously inferior to the 3 worst articles. Another one is United States, because in the U.S.A. nationalism and patriotism are considered to be positive things. The third is José Rizal, a Filipino national hero. This is no coincidence of course, today he will be commemorated.
UPDATE: It's 19:12, 19 June 2011 (UTC) now and suddenly Bieber is among the best articles!
Pay attention to the numbers.
rating bias for/against the topic of the article is definitely a major issue with any open rating system. It's important to note that the dashboard is a very particular query on the data, i.e. the 24 hour time window with a 10 rating threshold, which is going to produce its own odd biases. I'm not convinced yet that the current dashboard queries are useful.
One could cut and weigh the rating data in different ways to correct for bias. For one thing, it may be the case that within the submitted rating data, certain rating behavior is more or less predictive of rater bias (e.g. all 1s vs. a more balanced rating; self-identified expertise vs. no self-identified expertise). It may also be useful to take into account the overall behavior of the rater (e.g. just one article vs. multiple articles).
Pretty much every community-driven rating system applies additional algorithms to produce the total rating, and so far we're taking a pretty simplistic approach in surfacing this data. For a project like Wikipedia, we think making the data as openly available as possible will be the best way to gain a collective understanding of what the strengths and limitations of the current approach are, and to continue to iterate on the overall system based on what we learn.
I wish I was eloquent in English! In my native language discussions like this one are much easier for me.
I think you are right about the current dashboard queries, they should be improved. To be honest, designing algorithms is not my expertise, but I will spent some time thinking about it and if there is any result worth mentioning I will post it on this page.
It is probably worthwhile to look at other high traffic websites that have had a rating system for a long time, we can learn from the mistakes they made in the past and steal their ideas.
Subfader wrote about this and used the IMDB as an example: Extension_talk:Rating_Bar#Adapt_IMDB_rating_system
I'm broadly supportive of the Article Feedback Tool, but I think we might be able to tweak it. Wasbeer raises a good point about the potential for gaming - whether it's the Justin Bieber "haters" downvoting the article because they don't like Bieber, or "Beliebers" upvoting the article because they like Bieber, the problem is that people are voting based on their preferences for or against the subject of the article, not on the quality of the article itself.
One simple fix for this would be to make the display of the voting panel much more random. Make it so that there's only a 1 in 20 chance of it turning up, so you don't get people trying to ballot stuff it for or against particular people or things that they like or dislike. Here, Slashdot is the model - on Slashdot, you only get a certain number of moderation points every so often. It is hard to use those randomly assigned moderation points to downvote the stuff you don't like because getting them is such a random thing. This seems like an easy fix for the more popular articles to prevent ballot-stuffing and other concerns that Wasbeer raised.
One of the most important things in designing an interface - any interface - is that it should behave consistently, every time. This falls into the Principle of Least Surprise.
While I see your point about randomizing the access to the voting panel (to avoid gaming situations), doing so would clearly make the tool unpredictable, which would be a rather bad user experience.
There are better ways to detect and handle gaming (mostly done on the server side, by tweaking the calculation maths). This is something that I and Dario are interested in a great deal, and if anyone has any suggestions as to how modify the calculations, we're all ears. I must confess that gamification problems are low on the list of issues, since (as with all things in Wikipedia), they end up solving themselves over time (like spates of vandalism or errors in the text).
An article may be gamed for a short period of time (say, a couple months, or even a year) but eventually people will stop caring about that and move on and the ratings will adjust back to normal.
Wasbeer, your comments have ceased to be constructive in any way.
If you wish to continue to have a dialog about this, I am willing to do so but you must decide to be a useful part of the process.
Sorry, I do not understand you. Please remember this is not my native language.
Was I impolite to you in any way? If so, I apologize in advance, but please do tell me what the problem is.
I think my comments are highly constructive, and I even give a lot of useful suggestions that can be implemented if my advice to stop with the AFT is ignored.
Example of completely biased data is not helpful? Maybe because it is showing why it is bad idea? Is it possible for WMF to stop wasting time and money on wikilove extension, abandoned projects (LT) and useless surveys and provide WYSIWYG editor?
The article feedback tool is a cancer on Wikipedia. At this point it ought to be abundantly obvious that is not a good idea but, like with Flash welcome pages of the 90's web, it takes so long so some people to recognize the obvious. A) These things are ugly and detract from the aesthetics of an article. B) They are huge and especially distracting on smaller pages. C) It's clear even from a quick glance here that there's a strong percentage opposed to them, so why are they still around? D) The proponents are using arguments against the opponents and are failing to apply those same standards to themselves. That's annoying. E) I do not think that the information collected will be properly used and interpreted by editors. Interpreting surveys is hard and naive attempts are more likely to misinterpret the data, F) Small sample size will lead to incorrect conclusions being drawn and acted upon. To all those involved in its creation, swallow your pride and abandon this blight on Wikipedia. We appreciate your effort but the solution is worse than the problem. Jason Quinn 02:04, 21 June 2011 (UTC)
- B) They are huge
For the record: This was reported on Bug 29303 (improve layout of ArticleFeedback tool).
Jason made a huge mistake by mentioning cancer, its like Godwins law, please refrain from using analogies that are incorrect and offensive. We, the people who oppose the AFT, should try to be way more reasonable and polite than the people who support the AFT at all times.
Remember: this is not a debate we are trying to win. We already have. Now we need to gently inform the supporters of the AFT that even though their efforts are appreciated we do not want the AFT. It is much easier for them to accept this if we are nice to them.
I acknowledge the fact that the people who support this idea are acting in good faith and I think certain parts of the AFT are certainly well thought-through. A lot of time and effort have gone into this idea, and even though Jorm and I are not always on the same page I am sure he has the same goal as we do (to improve Wikipedia), even though we disagree about the way to achieve this common goal, and is not responsible for spreading any deadly diseases.
I hope Jason will understand that I agree with the rest of his message, but that I think it is important to say something about that single word.
If we want to use analogies we need something that looked like a good idea when it was under development and turned out to be not so great in practice, but in general it is better to refrain from using analogies in situations like these.
When when did using the word "cancer" metaphorically become "incorrect and offensive"? I never got this memo. I consider these article feedback surveys a harmful addition that's spreading throughout the encyclopedia worsening its quality. According to you, "cancer" cannot be used to describe that? At worst I was being blunt, not offensive. For that matter, I don't even think "blunt" applies. I made a run-of-the-mill comment. Has PC'ness really degenerated to such a point? Quite frankly, I felt like I was reading an Onion-article when I read your comment.
Incidentally, due to the annoyingly garbled nature of... I think this is LiquidThreads? Anyway, the point is I'm really not sure if anyone can see the message I posted yesterday. So... here's a fresh link.
If the WMF doesn't heed feedback, why would editors? DragonflySixtyseven 13:56, 7 July 2011 (UTC)
"maybe your parents can explain this", That is a very nice retort isn't it.... >.< I have never known anyone to be offerended or disliked when they saw cancer being used to mean "unwanted", so perhaps you can expand on this.
(Oh, this is also coming from someone that has had a tumour removed their body)
I don't know the context behind this thing, and this is the only place I found where it is discussed besides some threads at the Village Pump.
First, and maybe I am not very well informed here, but why the hell is AFT forced on the English Wikipedia without a clear community consensus? Why is it discussed at this obscure wiki and not where it matters most?
Second, I echo Jason Quinn's comments above. Good effort and all due respect to those who thought it out and what not, but seriously, I cannot see a valid use for this thing besides being disruptive on the (seemingly random) pages where it's placed. At the very least, make it into a template or something, so that editors can remove it from pages and discuss on specific talk pages whether it's needed or not.
Please remove this ASAP until community consensus has emerged. CharlieEchoTango 03:48, 21 July 2011 (UTC)
As mentioned here, it is already possible to remove the tool from specific pages by adding those pages to a category which is used as a black list. On Wikimedia projects, the category is set as "Article Feedback Blacklist" by default.
I have to agree that this doesn't seem to be working. Whether there's a sudden spike in "pro-Bieber" or "anti-Bieber" ratings is just a symbol of a deeper problem: readers can't tell you if an article is of high quality. They can only tell you if they like it.
Trying to "quantify quality" is one of the most impossible things you will ever try to do as a scientist. You may as well ask people to rate the articles for how delicious they are, or how nice they smell.
Much better to make it easier to "qualify quality". If you gave readers a red pen, they'd tell you far more than a numerical rating.
I think it's too early to tell where the quantitative ratings are or aren't useful. There's definitely a fair amount of "like/dislike" based voting, as well as attempts to manipulate/game the ratings in various ways, which is entirely expected. A big question is whether, when looking at the trends in the overall rating data or on specific articles, it's possible to translate these ratings into meaningful action (e.g. articles that may need tagging/cleanup, undetected vandalism, articles that are candidates for promotion, etc.), or whether there's too much noise in the data to do that.
To answer that question, we'll have to let the ratings play out for a while, and look at the data from different angles.
But, I do agree that soliciting more specific feedback and action (other than direct edits) is likely to ultimately have more impact. What do you think of the ideas in Article feedback/Extended review?
How long is a "while"? How long do you estimate it would take to get enough data that would actually be sufficient to translate it into meaningful action? For the lesser frequented articles that may actually need the meaningful action I anticipate it would take years!
There is no way in which we can increase the volume of ratings for articles that receive a small number of visits unless we change the positioning of the AFT to make it much more prominent , which is not planned. The ramp-up of AFT to the English Wikipedia was completed yesterday so we are now able to collect and analyze data across a much larger number of articles than the previous 3% sample. We expect to have new results ready to be published in the coming weeks. If you have specific questions, please post them in the AFT research talk page or on specific research subpages: 1 - 2 - 3 - 4.
I don't like the feedback tool. It reeks of insecurity or hubris. Maybe both.
I don't read Wikipedia to pat or slap its authors on the back, really I just don't care that much about it. I understand that the authors/owners/whatevers really care a lot, but I don't.
I'm trying to articulate what I don't like about the feedback panel. Perhaps its because it's an advertisement/promotional style thing which is stylistical masquerading as content. Perphaps if it was flashing yellow/red "You're the 1,000th visitor to this page..." I might pay less attention.
In short it'd be great if the survey thang had an option [x] Please Don't Bother Me Again On This Computer With Surveys and Begging Notices.
Yet another superflous feature that only makes the system more complex, makes the articles uglier, wastes screen space (lots), wastes work, consumes more bandwidth, and diverts the readers's attention. (Actually, the whole article evaluation machinery is a huge waste of resources.) Please junk it. Wikipedia must be made simpler, nor more complex.
I am user Cazort on Wikipedia. I gave a detailed explanation of why I think this tool goes against the spirit of Wikipedia's policies, and how I would like to see it removed, here: w:en:Wikipedia_talk:Article_Feedback_Tool#I_believe_this_tool_conflicts_with_the_spirit_of_Wikipedia_policy.2C_and_would_like_it_removed
Hello! I respect your work, but have a problem.... Often is difficult for me to understand your code. It is necessary to implement the rating system on the wiki site . The question is - how can indefinitely store user ratings( $wgArticleFeedbackSMaxage >> unlimited, ), and how to show on the dashboard page list of all articles with rating for all period ($rating_set_threshold = 0 ?, dates? ) . I hope for your help. Sincerely, Roman. ( PS not commercial, /*Sorry for my english*/)
The "Rate This Page" box is too big. Maybe:
- have it collapsed and/or across the width of page rather than what seem to be a fixed 50% width
- it should be after categories since it is not article content
- makes article look silly if it is a stub and there are three templates, e.g. w:Hudson and Halls
- maybe it should be a new tag on the top of the page (like the talk, history etc tags)
After spending a lot of time browsing as a logged off reader rather than logged in editor the "Rate This Page" box became a real annoyance.
I like the rating feature and I think it's well implemented. I knew this information must be aggregated somewhere, and I was happy to finally find the Article feedback dashboard. What I found was surprising.
At the moment that I looked, the Forbes list of billionaires (2012) page was ranked number 1, and the Plastic page was ranked number 2. Not that individual rankings matter so much, because it's not a contest. But I think it is of tremendous value to see examples of what the community considers to be among the most valuable content on Wikipedia. And I think the Forbes list, while not a bad page, does not rise to that level. So looked at the number of reviewers.
The Forbes article had about 20 reviewers at the time. The Plastic article had around almost 500. I believe that a 4.85 with 500 reviews definitely beats a 4.86 with 20 reviewers. The question is, what is the right way to factor in the count in the algorithm. I'll look into whether there's a standard way to do this in the world of statistics, but maybe somebody here knows. But I think it's a conversation worth having. I did a quick look through this Talk page to see if this thread has already been started, and I apologize if I missed it. I didn't see one.
After further consideration, it might make sense for there to be a threshold before appearing on the list. There may already be, lest you end up with the highest-rated article being one that was given all fives by a single user. The question is, what should the threshold be?
I submit that a "statistically valid sample size" should be determined. To do that, you need to have a denominator to start with. I think that denominator should be based not on the entire number of Wikipedia users, but perhaps on one of the following:
- The total number of distinct users who have rated any pages in that particular instance of Wikipedia
- The total number of points awarded in ratings on all pages of that instance.
By "instance," I mean a language or locale. For example, en.wikipedia.org.
So, to summarize, I mean that if I put up a page on trapeze cats, and a few people rate it as five stars (perhaps including me -- I don't know if that's possible), it doesn't show up on the list. But when that 600th user rates me, then my votes have become statistically valid, and I can appear on the list.
Happy to talk more about this with anybody who is interested. I think the ratings are really useful and really interesting.
Is there community consensus for full deployment on all English Wikipedia articles 31 May 2011?
I must ask; Is there a community consensus for the planned full deployment of this on all English Wikipedia articles on 31 May 2011? I see comments scattered here and there on enwp and on this wiki. Some comments are positive and some are negative. Does the community want it, or is it just the Foundation that wants it? Has the community been given ample opportunity to properly give feedback about the AF-project?
I think it is fairly clear from various discussions at enWP including w:en:Wikipedia:Village_pump_(technical)#Article_Rating_appears_immediately_after_creation.3F that there is not consensus for this deployment, at least in the present form of the tool. This is a major content related issue, not a narrow technical one, and I think a very broad consensus would be needed. there are unfortunate similarities to the earlier premature pre-emtive actions on article protection with a tool that did not meet the needs and expectations of the community--and has consequently been totally rejected when it might have been improved and used productively. If the developers wish to show any respect at all for the project they are serving, this should be postponed. (I am not sure there would be consensus for its broader use at all, considering some of the comments; I am fairly sure that would be consensus that it certainly must not be used on new short articles.
It should be mentioned that the Software deployments was updated and the entry of full deployment on all enwp articles 31 May was removed but the question of community support for deployment of the tool is still unanswered.
Last edit: 01:10, 2 June 2011
- Merged post from "Could someone open a discussion page for this on en.wiki?"
Any chance anyone could open a discussion page for this on en.wiki? It would be nice to see the results of any discussion about the trial and as I'm sure everyone here knows there are mixed feelings about both the concept and the design. Thanks. 220.127.116.11 09:25, 1 June 2011 (UTC)
Thanks, but I think we need a dedicated page to discuss this, neither of those is appropriate for what is a content related discussion. 18.104.22.168 04:57, 2 June 2011 (UTC)
Everyone is free to start a discussion on Wikipedia. Those I linked to are the two I've found that currently talk about it. If you don't want to join any of them you are free to do so too. :-)
I was hoping maybe someone from the Article feedback/Public Policy Pilot/Workgroup  might start a page, or at least someone directly involved. That would give it more 'official' standing and make sure someone involved with it was involved with the page. Dougweller 09:52, 2 June 2011 (UTC)
I searched all over but couldn't find the way to pull the results of "Trustworthy, Objective, Complete, Well-written" for a specific article to my page. In short i want to display AFT Dashboard (Special:ArticleFeedback) with added parameters of my own on page other than Special page.
|Name||Website URL||Content Rating||Reviews|
how can i achieve the above? Thanks
There's probably not much point, since these categories of ratings are scheduled to go away soon, when version 5 is rolled out. Before long, no articles will be rated for being trustworthy, objective, complete, or well-written.
President Barack Obama was listed in the past as one of the American presidents to visit the Bilderberg meetings on you webpage, and if I am not mistaken, it was in 2009. He is now not listed. We all know he was there.
This is possibly a problem with Wikipedia-en's implementation, but I am skeptical.
I wanted to learn more about the survey, which led me eventually to this dashboard, which put the race condition article in the 'Recent lows' category, saying its average ratings were 2.06, 1.88, 2.41, 2.44, and 2.20, respectively. I thought, "wow, even with my good ratings, people don't like that article?" So I went to view the averages on the actual article, and saw the following averages: 3.5, 3.5, 3.7, 2.2. This is all currently viewable.
People base their ratings on the subject of the article and not on the article itself.
The two pages use different data sets. If memory serves, the article tells you the average of all ratings recorded during a variable time period (however long it took to have 30 edits to the page, which could be anything from 30 seconds to more than a year, depending on the level of activity at the article). The dashboard tells you the average of all ratings in the last 24 hours.
Feedback is only worthwhile if it's used as the basis for future actions.
despite all the feedback saying that the AFT is a bad idea, the WMF is not actually doing anything. It's collecting lots of feedback that says "BAD IDEA, GET RID OF IT", but nothing is actually being done. As above, so below: the AFT will simply squat on our articles like a sewage plant at the edge of a reservoir, and ratings people leave will never make any difference.
If people had to be logged in to rate articles, that might make a difference. But as is... no. It's pointless. DragonflySixtyseven 16:42, 6 July 2011 (UTC)
I agree, there are too many folks making decisions about rolling this out who have created it. This project is some person (or persons) baby and they are ignoring negative feedback, IMO.
Isn't it still in a pilot? If so, then feedback during the pilot should be gathered for the length of the pilot and not be a reason for killing the pilot.
I also believe the axiom that more people complain than praise. Without being a statistician, I wonder what the numbers would show about the numbers of individuals using the AFT against those complaining against it. Would that be a data point worth considering by both sides?
Yes, the feedback about the feedback thing isn't really going anywhere. Some people don't like the real estate it takes up, how it's not centered, etc. I also think it's a cop-out to make people post here in order to (possibly) get responses from the dev(s). Killiondude 23:27, 20 July 2011 (UTC)
Some people don't like the real estate it takes up, how it's not centered, These feature requests were reported on the following bugzilla tickets:
The people dropping in on this page are the random users we want to get feedback from
Unfortunately the data gathered so far is useless. The only conclusion you can draw from looking at it is that the current version of the AFT is a total waste of time.
Of course the data is available. See FAQ question #16:
- Is data generated by the Article Feedback Tool publicly available?
- We are releasing regular weekly dumps of the data we collect via Article Feedback Tool as well as making anonymized data available on the Toolserver. Further information on the data dumps can be found on this page.
The data is basically weekly raw data dumps, which you can analyze however you want.
This tool, which I've begun to loathe, is the most intrusive experiment on Wikipedia to date. And yet outside of this page, I cannot find much discussion about its present status and future. I must (I hope) be missing some deep insightful discussion about this tool. 22.214.171.124 06:51, 8 July 2011 (UTC)
No, unfortunately you are not missing any "deep insightful discussion" because it's creators (e.g. User:Jorm_(WMF) have no interest in creating or maintaining such discussion.
Gaming the system, is there a way to have both useful ratings and conversion of readers to editors?
According to the Article Feedback Dashboard stats on the 11th of October, the article with the highest average rating in the entire encyclopedia was w:Rangers F.C. (with a 5.0 rating) whilst the lowest rated article is - surprise surprise - their crosstown rival w:Celtic F.C. (with a 1.1 rating). This is possibly the most obvious "gaming the system" of the AFT system there is but even the other articles at the "top" and "bottom" show evident gaming. Other "top" include a series of South Asia related articles including w:List of Nalanda College Colombo alumni, w:Nalanda Maha Vidyalaya Colombo, w:2011 Champions League Twenty20, w:2011 FIBA Asia Championship. Other "bottom" ranked articles include w:Justin Bieber, w:Barack Obama, and w:United States.
The Article Feedback tool seems primarily designed to try to convert readers into editors. However, in the mean time, the feedback that it is generating is *actively* unhelpful. I'm all for converting more readers to becoming editors but the tool as it currently is does not seem to be generating useful ratings information. Is there a way to have both the cake and eat it too? Wittylama 04:45, 11 October 2011 (UTC)
You'll always have this kind of rating bias in the data -- one common way to deal with it is to apply smarter algorithms, e.g. to predict individual rater reliability based on past rating behavior.
We've now also started digging more seriously into alternative models for feedback systems that we'll want to experiment with. We've established a contract with Fabrice Florin (of newstrust.net) as product development consultant for this purpose, and you can see some wireframes and ideas in his October 10 presentation. I've encouraged Fabrice to start posting here as well.
One of my questions earlier on at this talkpage was about the RFP for the "extended" article feedback tool: Talk:Article_feedback#Extended_aticle_feedback_RFP_7581. Does this mean Fabrice is the person who has been given the project?
With regards to the presentation, linked above, I really like the idea of a new "reviews" tab in the skin especially if it will enable all the functionality identified in the "extended AFT" wireframes and in the presentation :-) I may have overlooked it, but is there any intention of integrating the AFT with the existing Article Rating system in use on talkpages (the FA, GA, B, C, Start, Stub system), or will it be kept separate? I think it would be a shame to have two completely independent ratings systems as that would merely perpetuate what we currently have now - one rating system for readers (that is scorned by users) and one rating system for users (that is unknown to readers). Wittylama 05:19, 13 October 2011 (UTC)
This is the point in which you make such suggestions, I believe. Fabrice is (afaik) managing the project, with a lot of input from us at the WMF and the community. We really want this to be transparent in design, with input from the community, so opening a thread with your thoughts is probably good.
Personally, I'm not a fan so much of a whole "tab" devoted to the reviews. I'd prefer to have the reviews more tightly coupled with the tool itself. Currently, the tool has two modes: "Submit Rating" and "View Ratings"; I could see a third option for "Reviews" or "Suggestions for Improvement" (which is why I argued for a more 'tab' focused format in the current design. This did not manifest, however.).
I do not believe that there is any intention to integration with the current system of article assessment. Article assessment is done by people who are devoted to it or are (supposedly) subject-matter experts. This tool is aimed at readers.
What I'd personally like to see is something that could help to create a "work list". "Needs more photos." "Needs better copy." Etc. Ways in which the reader community can contribute easily. We know many readers have suggestions for improvement but honestly feel like they do not have the right to edit the page. So this can help to bridge that gap.
I think we need to make it clear that there is a difference between the article's "Assessment" and it's "Reader Opinion." Though I'm not entirely convinced (at this point) that there is any "correct" way to do this.
I totally encourage you to open new threads with suggestions for Fabrice. I don't know if you've met him; he's a great guy and totally open to good ideas and constructive discussion.
I think you're really onto something with the "work list" as this is (potentially) valuable input that meets both the points of engaging the reader to contribute (a bit, hopefully more) and also providing actionable information for the existing community. You're right that "review" tab isn't necessarily part of the tool and it would be difficult to implement policy-wise because it would introduce a whole new workflow for the editing community. Nevertheless, the implication of having the AFT is that reviews will be made available somewhere and there are disadvantages to having them "dumped" on the talkpage just as there are disadvantages to creating a new "review" space.
I terms of "suggestions to Fabrice"... I've got three things, but I'll put them in a new thread, as you suggest :-)
I think it's worth remembering that the Dashboard is designed to identify skewed examples. What you see on the Dashboard is intentionally not representative of normal ratings.
Thank you all for your thoughtful suggestions about the Article Feedback Tool. It's a real pleasure to meet you, and I am sorry that I couldn't join this conversation sooner. (I just started working for Wikimedia recently as product development consultant on this project, and am still getting oriented.)
Wittylama, I appreciate your concerns about the potential for gaming ratings on Wikipedia. As Eloquence points out, there is always a risk of bias in user ratings like these, but we are exploring a number of remedies to address this issue.
Our current direction is to de-emphasize the ratings in the next version of the AFT, and to invite readers instead to offer specific suggestions for improvement (so their feedback can be more constructive and useful to editors). You can see some examples of that new direction in this new slide show from Oct. 11th, which we updated based on recommendations from the Wikimedia team. To that end, we're looking at services like GetSatisfaction.com for inspiration.
I also really like Jorm's "work list" idea and will aim to integrate that concept in our next round of designs. In coming days and weeks, we will post more wireframes and prototypes -- and seek more feedback from the Wikipedia community, including everyone on this thread. And I will respond next to Wittylama's questions in the newer thread you just posted. Feel free to ping me if you have any more recommendations: we thrive on feedback!
Sorry, don't have time to read the extensive discussion. Just wanted to add another "NO" vote. Not only do I fail to see benefits, I see very real harms in allowing people to rate pages. The obvious application is pseudoscientists using the tool to discredit science articles. There are many unpopular truths in this world.
I have no idea what your current methodology is to quality-control the pages, but with only a few exceptions, it appears to be working. I suggest putting your resources into refining the existing methodology.
Article feedback is meaningless, since the incredibly wide spread of users, all with different abilities and values (not to mention prejudices), means that no star rating can ever be truly validated. In addition, the extraordinary number of articles on Wikipedia ensures that a large number of them will receive very few such ratings, even over a lengthy period. I agree with other comments that if this tool is to be of any value at all, and I still have my doubts, it should appear on the article talk page. Lastly, although I feel I could respond to the rating system, I have no interest in doing so because a) see above and b) I would much rather spend the limited time I have available in editing and otherwise contributing to the encyclopedia. (Anyone who has ever used the eBay Review and Guide system will know that it is frequently confused with the eBay Seller rating system, and that both systems are subject to much misuse and abuse.)
it should appear on the article talk page
Most users of Wikipedia are not even aware that Talk pages exist. Since the goal of the tool is to attract impressions from as wide a range of users as possible, hiding the tool on the talk page is not a viable solution.
How one validates a process depends on what one wants to achieve. Eating five pounds of candy every day is a validated process for gaining weight; it is not a validated process for improving your driving skills.
We will not be able to validate these ratings as providing the One True™ Answer about the article's status. However, that's not actually the goal, and nobody has ever promised you that these ratings will produce the One True Answer for anything.
We can validate this process as being a reliable means of determining what our users (who are mostly non-editor readers) believe about an article. We hope that we will also be able to determine that it is a functional method of increasing the rate at which currently non-editor readers become active editors.
We talked about this tool on eswiki's village pump and we believe it's a great testing tool and also provides valuable insights, so we're wondering abotu the feasibility of running this on eswiki.
I don't know if this is feasible or not, but we really wish to try it (translations are not problem,we can do that easily).
Thank you. Magister Mathematicae 05:32, 30 May 2011 (UTC)
Is it possible to translate and put this in other Wikipedias? For example in the basque Wikipedia? -Theklan 22:20, 30 May 2011 (UTC)
The translation isn't the problem, in fact, the translation is perfect. The issue is that the extension is currently specifically designed for English Wikipedia and everything down to the sizes of the label is made for English.
The tool is currently an experiment to gather data, which, from what I hear and from reading the statistics is going quite well and many people find it to be a useful tool. Over the phases the Article feedback tool has gone over so far, new features have been introduced and things have changed. The original design wasn't fit for that nor was RTL a priority since it had to work on en.wikipedia. If all goes well, it will likely be redesigned at which point any wiki wanting it can get it.