I just saw this pop up on a web page. It's horrible. "Trustworthy?" If it needs an explanation, that is bad. We already have a talk page for each and every article. I was really scratching my head wondering why anyone would want my opinion anyway. And the "I am highly knowledgeable about this topic" part? Do we really expect people to be honest about this? The most I would support might be a small "Rate this article" option. But really I don't see the value in that either. This system has the potential to be gamed/abused. Please, throw this away and move on. Put efforts into making the talk page more accessible or inviting. But not this. Perhaps in addition to the Discussion tab, at the bottom of each article could be a link like "Do you have any feedback about this article?" linking to the discussion page.
Delete AF: "direct reader engagement" is already available on the discussion page, focus your efforts there.
Its the 5th of July. If there are no radical changes the community will overrule the supporters of the AFT on the 7th of July. The supporters of the AFT think they do not need consensus and think they can ignore the community. We, the community, should make clear we make the decisions. Jimbo is able to overrule the community, the supporters of the AFT are not able to do so, even if they believe they are. They do not have the authority.
Wasbeer, there is no "Jimbo overruling the community", neither a "group of supporters of the AFT". There is a team of people working to implement functionality described in the strategic plan and based on the strategic priorities defined by the Wikimedia community. We are looking for constructive feedback to help us understand how to make this tool valuable for the community and how to assess its value (there are several good suggestions in this sense in the above threads). If you simply expect the community to "overrule the AFT supporters" I don't see how we could even start a conversation.
Thanks for taking the time to respond to our questions!
A couple of not-so-important things first: I am not a native speaker, so please forgive me if I make mistakes. I did not say there was a "Jimbo overruling the community", I just said he is able to. There are supporters of the AFT (I did not say anything about a group afaik). Please use quotationmarks only for direct quotes. I know one of the teammembers, and I like him.
I do not think the community needs to overrule the AFT supporters at this moment in space and time because the feedback is no longer being ignored, but mentioning the possibility and using the same deadline was certainly an excellent idea because suddenly you and Howief responded to the feedback.
We have are having a conversation now, and maybe it is a good idea to start with topics that we are more likely to agree on.
What is your opinion about moving the AFT outside of div#content? From my point of view this is a very important step, and I think this goal is easier to achieve because we are less likely to disagree about it.
The tool will not be moved outside of div#content. It is simply not going to happen. It is essential that the tool is associated with the contents of the page itself and be visible to the reader.
Counter examples for your arguments:
- Something doesn't needs to be inside of div#content to be "associated with the contents of the page itself" (see e.g. the sidebar of anypage: there are various links which are related to the content of the page itself).
- The page footer (MediaWiki:Lastmodifiedat) is outside of that div and is still visible to the readers.
Only experienced and expert users naturally associate sidebar contents with the contents of the page itself (within div#content). Testing has shown us that anything outside of div#content is effectively ignored by the standard reader - even the tabs are not typically considered to be associated with the content of the page.
Last modified date is metadata that users honestly don't care about. That's a very poor example, actually - the way it is displayed (small grey font on light grey background) tells the user, simply, "do not pay attention to this at all."
OK, then, just replace the AFT with a link to the discussion page. That is where feedback belongs, visible to the community and interactive to the community. this tool is wrongheaded in so meany ways it makes me think there is a major disconnect in the people who created this tool as far as how Wikipedia works.
To be honest I still do not understand why moving the AFT outside of div#content is not an option. Do you have data to back up your claim that people are unable to understand that stuff outside of div#content may be related to the content of the article? Please explain this again.
The positioning of the tool is critical. Moving it to the toolbar or to the talk pages is simply not a viable option as this would dramatically drop the (already small) volume of ratings we get. If we want to study how to make this tool more valuable and unbias the results it produces we need to increase, not reduce, the number of ratings we collect. We know that positioning the tool at the bottom of an article is not optimal either (we saw that conversions decay very rapidly, following a Power Law, with the length of the article, suggesting that people just don't see it unless they scroll to the bottom of the page) but this is currently the best option we could find. Another option that has been considered is an expansible, fixed position tab on the right of the screen, but that would probably make it too prominent.
I don't know if the content of Article_feedback/Extended_review#Wireframes_3 is still to be considered or is already a discarded option (if so, why?), but you may want to try out the approach currently used on English Wiktionary. Look at the sidebar of a random word. There will be a "Feedback" section with some default options and also a link to wikt:Wiktionary:Feedback. (Unfortunatelly, the toolserver account used to collect the data seems to have expired and the tool doesn't seems to be fully functional at the moment)
There is also something similar on Spanish Wikipedia, right below the "Donate" link in the sidebar, wich allows readers to notify editors about specific problems on articles. To test it, open a random article and click on "Notificar un error". See also the talk about implementing it on English Wikipedia on topic "Report an error" feature of Village pump.
Please don't put it there, I want to use the scrollbar. Lots of people will open it by accident.
Timl2k4 the data we've collected so far indicate that when people submit feedback via the AFT survey they don't even realize there is a talk page or if they do, they are often intimidated by how hard it is to participate in a conversation (because of technical and social barriers). I think the idea of experimenting with a "edit the talk page" Call to Action is a good one, but I expect to see a very low number of conversions. The question we should ask is: do we want to have more people participate in the discussion on individual articles and find lightweight ways to persuade them to become editors or just keep the current high bar and lose these potentially good contributions? Both AFT and WikiLove suggest that there are different entry vectors we can experiment with to successfully engage new users with the contents and members of the community.
- they don't even realize there is a talk page or if they do, they are often intimidated by how hard it is to participate in a conversation (because of technical and social barriers).
Bingo! This is the problem. Lose the high bar to he discussion page. The entry point to editing articles should start there. Having inexperienced editors editing articles without any prior discussion is no good. I've seen this time and time again, they have good intentions but end up with bad results. Look at this another way, is not the discussion tab the built-in starting point for Article Feedback? Let's make it better! The most common mistake is new users have no clue how to sign comments. This is trivial and should not be a problem. Make it automatic for IP users. Currently, if someone hits edit there is no reminder or pointer to the discussion page. So the Wikipeida discussion system is woefully inadequate, instead of fixing that this tool was created instead. Not good.
I think the intention is to have more "inexperienced editors editing articles" (e.g. to fix typos, or correct obvious mistakes, and to become regular editors), since this is one of the priorities determined by the community of Wikimedia wikis.
Increasing both the total number of editors, and their diversity, is a key priority for the Wikimedia movement. (...) We need to improve the editing interface in order to reduce barriers to participation. When people try editing for the first time, we need to support and coach them.
and from strategy:May 2011 Update:
Two months ago, the Wikimedia Foundation published the results of our new Editor Trends Study (you can read the results here), which showed that over the past several years it's been getting increasingly difficult for people to edit the Wikimedia projects.
(emphasis mine) So, I think it is unlikely that "prior discussion" will be ever made into a requirement to edit wiki pages. Instead, the chances are that we will have new easy ways to edit pages (e.g. Extension:InlineEditor, which you can test here) and will get more edits from inexperienced editors
True, the ultimate goal is to get more of our users editing article pages. But we also want to get the right types of users and we want to keep them for as long as we can.
It could be the case that discussions offer the best introduction to editing Wikipedia, and users that first edit a discussion page tend to stay longer (e.g., they have a better understanding of the rules, know how to avoid newbie mistakes, etc.). This would argue for making the invitation to discuss more prominent, simplifying the talk page interface, improving the feedback loop to the newbie once she's entered into a discussion, etc. Or it could be the case that completing a very simple edit like a spelling correction offers the best entry into Wikipedia (e.g., newbies are shy, they often feel like they're not qualified to edit, so accomplishment is a good way to build confidence). This would argue for making this type of editing as simple as possible (e.g., in-line editor). Or maybe the most effective way of drawing new editors in is to have them create an account (e.g., it gives them a sense of belonging and therefore more ownership over their contributions). And we need to measure this against the quality of these editors. The last thing the community needs is a bunch of disruptive editors coming in through one of these entry points.
The reality is probably a mix of these reasons, and many more. Also, an "entry vector" that works for one newbie may not work for another. At present, we don't have quantitative data that enables us to compare the effectiveness of these various "entry vectors". But it's something that we'd like to test. For example, we have indication that AFT is probably a good entry vector, given the 17% click-through rate for the edit call-to-action. But we don't yet know whether these individuals actually make good editors (e.g., how constructive are their edits? How long do they edit for?).
I agree with Howief and Helder. And I think resources should be focused on developing more user friendly tools like LiquidThreads (and perhaps also the InlineEditor) and invite readers to the talk pages instead of the Article Feedback Tool.
I want to interact with other users who actually work and reason, not readers who just may want to express an opinion. I don't understand why the developers choose to prioritize the latter before the former. The underlying problems need to be fixed, not just the symptoms.
I think that's a good way of looking at things. We want features that help us get the "right" kind of editor in the funnel, not just any editor. That's why I'm a little skeptical about the results of experiments like the section edit icon -- this project increased the number of edits by a significant amount, but it may be because the section edit link is nice and shiny and therefore attracted people who really shouldn't have been editors in the first place. We haven't done the follow-up analysis for this project, but my guess is that the cohort of users that come in through the section edit link icon are probably of lower "quality" -- their edits are probably reverted at a higher rate and they probably don't stick around for quite as long. This is the type of analysis that we need to do with AFT. The click-through rates give us some good indication that users respond to an invitation to edit, but the real question is whether these editors will end up becoming constructive Wikipedians.
The invitation to discussion is something we really need to dig into. I agree that we should focus on developing more user friendly tools for discussion (like LQT). One of the underlying problems is that there are disconnects at many levels between readers and the idea of discussion on Wikipedia. Some don't even know discussions take place. Some think they take place but aren't sure where to find them. And those that can't find them are confounded by the interface since it doesn't look like any discussion page they've seen before. Brandon's done a lot of thinking around the next generation of LQT. But this is a much longer-term project -- to get this right will take a significant amount of time to design, develop, iterate, etc.
I think that both the section edit icon and the InlineEditor will encourage readers to become editors. And they do it in a more direct and honest way than the manipulative method AFT employs. Both SEI and IE put the newbies exactly where I prefer them: In the thick of it. But we need better tools to guide them once that first step is taken:
- Positive feedback as soon as possible
- Instant help when needed
In my mind, reducing churn should happen before heavy recruiting starts. Making sure that first edit is a pleasant experience is paramount.
Yes, agreed on positive feedback and instant help. The positive feedback needs to be given to the new editor in a manner in which they understand. We can't assume new editors understand the purpose of their user talk page, nor can we assume they even know that such a page exists. I'm also not sure how likely it is that these new editors either notice or act on the yellow notice across their screen. Email notification might be a better way to make them aware of the feedback, but not everyone provides an email address upon registering.
Instant help would be awesome.
- Positive feedback: Perhaps a tool that shows edit diffs by users who does not yet have an edit on their talk page. Experienced users monitoring that page should then be able to "one-click-welcome" users making useful edits from a selection of welcome templates; "Thank you for copyediting!", "Thank you for adding information!" and so on. I'm thinking along the lines of the (now defunct?) Interwiki-Link-Checker (http://toolserver.org/~flacus/IWLC/start.php) where bilingual wikipedians could compare two wikipages and answer yes or no (or maybe) to the question "create iw links between pages?"
- Instant help: Perhaps a chat on the tool-server where experienced users offer help in a one-on-one chat with newbies.
Well that is slick! Glad to see that in the works. As far as the discussion page, a little entry form for article feedback perhaps? Also make the discussion option more prominent somehow.
Yes, I think so. I have a feeling it would end up with all sort of spam, useless comments. There would need to be an easy way to filter them. Having a captchca would probably dissuade 90% of the population from leaving a comment, I mean the useful ones too, although if someone has a useful comment, they may be more motivated to work through the captchca.
After leaving their feedback they could be presented with what would I hope eventually be hopefully improved ways of article discussion and editing than what currently exists.
I'm going off a bit of a tangent here, but if someone has a fact to add to the article, walking them through the process of adding a proper citation (and helping them find one if they don't and making the edit process very user friendly would help turn readers into constructive editors. If they simply have a correction, hopefully this will become more user-friendly, I mean user-friendly like my grandmother could make a correction without messing up the page. The examples I've seen in this discussion (excluding the AFT) are encouraging.
Pages with highest/lowest ratings
Immediate improvement needed
The absence of any opportunity for the reader to actually make a suggestion for improvement of the articles is a critical absence in this "feature", and is antithetical to the processes that make Wikipedia what it is. We already see articles heavily watched by fans receiving ratings that clearly have no basis in the quality of the article, and I have had four FA-level writers state that they will be stopping participation in the project because their articles, already assessed for quality, are now being "rated" without any rationale for these ratings and completely absent any comments that could explain the ratings. They feel it is one more step toward the facebook-ization of Wikipedia, and it is hard to say they're wrong when there's no reason to believe that rating of articles will result in increased contributions or increased quality.
Please add a free-text box labeled "suggestions for article improvement" or the like, with the written commentary being fed to the article talk page and signed "Article rater" or the like. (Do not use IP addresses, because people responding to this section will not be editing per se but responding to a survey. Their identifying information should not be exposed publicly.) Yes, we'll still get a pile of "fanboy" or "anti-fanboy" comments, but these can be moved off the talk pages as needed. This will serve to introduce the readers to the concept of actively participating, while also giving editors information to improve articles. As it is, this feature is more likely to increase the gulf between readers and editors than it is to bridge the gap, and has little likelihood of turning readers into editors. Risker 04:26, 14 July 2011 (UTC)
Thanks for the thoughtful note Risker.
"Suggestions for improvement" is a great idea. During the office hours we held to discuss this feature, I asked what input from readers would be most helpful for editors. What's missing/are there holes/etc. was a theme that came up. I also think that engaging readers in thoughtful commenting could put them one step closer to becoming contributors.
There are a number of ways to get this type of feedback. A free-text box, as you mention is one. Jorm lists a few others in a different thread on this topic (e.g., article needs more photographs, references, etc.). The trickier part IMHO is where to surface the reader input. The downside of directly entering comments directly on the talk page is that, as you say, editors will then need to move the superfluous comments off the talk page. This strikes me as potentially creating a lot of work for with uncertain payoff for editors. One idea that has been discussed is the idea of a “parking lot” for comments. This “parking lot” would be a separate space for comments where editors can then review and then promote helpful comments as needed. I think suggestions for improvement make a ton of sense – it would be good if we could implement the feature in way that doesn’t create unnecessary work for our editors.
We could certainly use your perspective in further developing this feature -- would you like to be part of the workgroup that helps design the commenting function? Or if you think the community would be okay with having comments directly on the talk page, we should discuss this option in greater detail.
Comments directly on the talk page would be a problem I believe. Some would undoubtedly just be forum style comments which should be removed, as you say giving editors more work. But some would almost certainly be BLP violations and that concerns me a lot more. Sure, readers can edit the talk page now, but I think you'd see a lot more BLP problems if it was easier to get your opinion onto the talk page. 18.104.22.168 12:15, 14 July 2011 (UTC)
Risker, that's a very important suggestion and one that we see frequently mentioned in the survey data. How to make reader feedback specific and actionable and how to further engage with the reader after the rating is submitted are two of the main problems we need to focus on for further development. I added some ideas and suggestions in this sense to the call to action review page and I'd love to hear your thoughts.
It may be a good idea to fix some of the bugs before deploying the AFT.
The words "gaps" and "optional" are in the same spot. Please avoid making the AFT even bigger.
(FF v5.0, default text size)
(Chrome 12.0.742.112, default text size)
The rest of the bugs can be found here: https://bugzilla.wikimedia.org/buglist.cgi?query_format=advanced&list_id=16547&component=ArticleFeedback&resolution=---
Rollout is beginning today (within about three hours, I think), in increments of 5,000 articles. So there won't be time to address bugs before then.
However, we can (and will) address many design concerns in future iterations, post-deployment.
- I think it's irresponsible (and a little arrogant) to deploy a tool with known bugs and with considerate objections from users.
- I am yet to be convinced you have community support for deploying this on the English Wikipedia. Can you please supply me with a link to the source of your strongest argument that you have this support?
Fair enough. Not a showstopper bug. But the links you supplied were not the link I asked for.
I doubt the research will contain evidence of support of the AFT from the community of editors on enwp (which is what I'm asking for), but sure, I'll be patient. :-)
A single edit could change the accuracy of an article completely but the ratings remain unaltered, so what's the motive? If you want to encourage knowledgeable people to contribute then just say so.
Encouraging people to pass casual, unqualified judgement in this way will at best demonstrate the value of an article according to popular opinion, and for a reference work that could be counterproductive.
Ratings expire after (any) 30 edits, so they do not remain "unaltered" forever.
A few editors have said they are offended that mere readers are being permitted to express an opinion on their work, but I can't actually see any of our regular editors screwing up an article just to get a higher rating from "casual, unqualified" readers. Can you?
I agree that any set number of edits is a limited approach. Three edits might result in major changes; thirty edits might be nothing more than 15 pairs of vandalism and reversion. Furthermore, half the ratings might have been made while one of those 15 vandalized versions were up.
I don't agree that the tool is irrelevant to the broad goals of the WMF. I think it may prove to be far more effective at turning readers into editors than anything else we've tried. Passively reading "This page needs to be fixed" has not proven to be very effective. If you start with someone who is already thinking about the article and interacting with the page (by rating it), and then we've already got their attention when we say, "You know, you create an account—you could edit this article—you could try this out", and we've got their focus on the article's strengths and weaknesses.
Might it be irrelevant for experienced editors identifying articles that need work? Quite possibly, but nobody's ever promised that the ratings would be useful for that purpose. In fact, the devs have specifically said that's not a validated use of the tool.
WhatamIdoing, when were expiring rating introduced? Has the tool worked like that from the start? I only become aware of it recently so I was partly misunderstanding how it works.
It's been described in the documentation since at least this spring, but I don't know exactly when the feature was first introduced.
So out of the 3 million+ articles on en.wiki, Justin Bieber, George W. Bush and maybe 100 other high traffic articles could be considered "1 offs" because their survey results will be skewed by fanboys/haters. That is still an infinitesimally small fraction and even if these are high traffic articles, that doesn't diminish the potential usefulness that this tool can offer to the project across the board.
The editors who watch Bieber and GWB will not be making significant changes to those articles based on the AFT tool. However, the editors to articles like the 1984 Summer Olympics, Prince Edward Island, Piston valve, Aging of wine, Royal Scottish Academy, Bamboo, Australopithecus etc might be inclined to look more closely at those articles if they start receiving a pattern of low ratings in comparison to similar articles.
I think this is a key point that critics are missing when they try to bring up "crap" ratings on articles like Bieber and GWB. Articles like that make up a very, very small percentage of Wikipedia and they should not be held as a benchmark for how new tools or ideas are implemented on the project.