Talk:Article feedback

Jump to: navigation, search

About this board

By clicking "Add topic", you agree to our Terms of Use and agree to irrevocably release your text under the CC BY-SA 3.0 License and GFDL

Delete AF: "direct reader engagement" is already available on the discussion page, focus your efforts there.

26
Timl2k4 (talkcontribs)

I just saw this pop up on a web page. It's horrible. "Trustworthy?" If it needs an explanation, that is bad. We already have a talk page for each and every article. I was really scratching my head wondering why anyone would want my opinion anyway. And the "I am highly knowledgeable about this topic" part? Do we really expect people to be honest about this? The most I would support might be a small "Rate this article" option. But really I don't see the value in that either. This system has the potential to be gamed/abused. Please, throw this away and move on. Put efforts into making the talk page more accessible or inviting. But not this. Perhaps in addition to the Discussion tab, at the bottom of each article could be a link like "Do you have any feedback about this article?" linking to the discussion page.

Wasbeer (talkcontribs)

Hi Tim!

Its the 5th of July. If there are no radical changes the community will overrule the supporters of the AFT on the 7th of July. The supporters of the AFT think they do not need consensus and think they can ignore the community. We, the community, should make clear we make the decisions. Jimbo is able to overrule the community, the supporters of the AFT are not able to do so, even if they believe they are. They do not have the authority.

DarTar (talkcontribs)

Wasbeer, there is no "Jimbo overruling the community", neither a "group of supporters of the AFT". There is a team of people working to implement functionality described in the strategic plan and based on the strategic priorities defined by the Wikimedia community. We are looking for constructive feedback to help us understand how to make this tool valuable for the community and how to assess its value (there are several good suggestions in this sense in the above threads). If you simply expect the community to "overrule the AFT supporters" I don't see how we could even start a conversation.

Wasbeer (talkcontribs)

Hello DarTar!

Thanks for taking the time to respond to our questions!

A couple of not-so-important things first: I am not a native speaker, so please forgive me if I make mistakes. I did not say there was a "Jimbo overruling the community", I just said he is able to. There are supporters of the AFT (I did not say anything about a group afaik). Please use quotationmarks only for direct quotes. I know one of the teammembers, and I like him.

I do not think the community needs to overrule the AFT supporters at this moment in space and time because the feedback is no longer being ignored, but mentioning the possibility and using the same deadline was certainly an excellent idea because suddenly you and Howief responded to the feedback.

We have are having a conversation now, and maybe it is a good idea to start with topics that we are more likely to agree on.

What is your opinion about moving the AFT outside of div#content? From my point of view this is a very important step, and I think this goal is easier to achieve because we are less likely to disagree about it.

Jorm (WMF) (talkcontribs)

The tool will not be moved outside of div#content. It is simply not going to happen. It is essential that the tool is associated with the contents of the page itself and be visible to the reader.

He7d3r (talkcontribs)

Counter examples for your arguments:

  • Something doesn't needs to be inside of div#content to be "associated with the contents of the page itself" (see e.g. the sidebar of anypage: there are various links which are related to the content of the page itself).
  • The page footer (MediaWiki:Lastmodifiedat) is outside of that div and is still visible to the readers.
Jorm (WMF) (talkcontribs)

Only experienced and expert users naturally associate sidebar contents with the contents of the page itself (within div#content). Testing has shown us that anything outside of div#content is effectively ignored by the standard reader - even the tabs are not typically considered to be associated with the content of the page.

Last modified date is metadata that users honestly don't care about. That's a very poor example, actually - the way it is displayed (small grey font on light grey background) tells the user, simply, "do not pay attention to this at all."

Timl2k4 (talkcontribs)

OK, then, just replace the AFT with a link to the discussion page. That is where feedback belongs, visible to the community and interactive to the community. this tool is wrongheaded in so meany ways it makes me think there is a major disconnect in the people who created this tool as far as how Wikipedia works.

Wasbeer (talkcontribs)

To be honest I still do not understand why moving the AFT outside of div#content is not an option. Do you have data to back up your claim that people are unable to understand that stuff outside of div#content may be related to the content of the article? Please explain this again.

DarTar (talkcontribs)

The positioning of the tool is critical. Moving it to the toolbar or to the talk pages is simply not a viable option as this would dramatically drop the (already small) volume of ratings we get. If we want to study how to make this tool more valuable and unbias the results it produces we need to increase, not reduce, the number of ratings we collect. We know that positioning the tool at the bottom of an article is not optimal either (we saw that conversions decay very rapidly, following a Power Law, with the length of the article, suggesting that people just don't see it unless they scroll to the bottom of the page) but this is currently the best option we could find. Another option that has been considered is an expansible, fixed position tab on the right of the screen, but that would probably make it too prominent.

He7d3r (talkcontribs)

I don't know if the content of Article_feedback/Extended_review#Wireframes_3 is still to be considered or is already a discarded option (if so, why?), but you may want to try out the approach currently used on English Wiktionary. Look at the sidebar of a random word. There will be a "Feedback" section with some default options and also a link to wikt:Wiktionary:Feedback. (Unfortunatelly, the toolserver account used to collect the data seems to have expired and the tool doesn't seems to be fully functional at the moment)

There is also something similar on Spanish Wikipedia, right below the "Donate" link in the sidebar, wich allows readers to notify editors about specific problems on articles. To test it, open a random article and click on "Notificar un error". See also the talk about implementing it on English Wikipedia on topic "Report an error" feature of Village pump.

Wasbeer (talkcontribs)

Please don't put it there, I want to use the scrollbar. Lots of people will open it by accident.

DarTar (talkcontribs)

Timl2k4 the data we've collected so far indicate that when people submit feedback via the AFT survey they don't even realize there is a talk page or if they do, they are often intimidated by how hard it is to participate in a conversation (because of technical and social barriers). I think the idea of experimenting with a "edit the talk page" Call to Action is a good one, but I expect to see a very low number of conversions. The question we should ask is: do we want to have more people participate in the discussion on individual articles and find lightweight ways to persuade them to become editors or just keep the current high bar and lose these potentially good contributions? Both AFT and WikiLove suggest that there are different entry vectors we can experiment with to successfully engage new users with the contents and members of the community.

Timl2k4 (talkcontribs)
they don't even realize there is a talk page or if they do, they are often intimidated by how hard it is to participate in a conversation (because of technical and social barriers).

Bingo! This is the problem. Lose the high bar to he discussion page. The entry point to editing articles should start there. Having inexperienced editors editing articles without any prior discussion is no good. I've seen this time and time again, they have good intentions but end up with bad results. Look at this another way, is not the discussion tab the built-in starting point for Article Feedback? Let's make it better! The most common mistake is new users have no clue how to sign comments. This is trivial and should not be a problem. Make it automatic for IP users. Currently, if someone hits edit there is no reminder or pointer to the discussion page. So the Wikipeida discussion system is woefully inadequate, instead of fixing that this tool was created instead. Not good.

He7d3r (talkcontribs)

Automatic signatures for IPs seems doable by JavaScript, and could probably be requested on Local Village pump. Besides, the problem will be better solved by Extension:LiquidThreads, wich is enabled on this page.

He7d3r (talkcontribs)

I think the intention is to have more "inexperienced editors editing articles" (e.g. to fix typos, or correct obvious mistakes, and to become regular editors), since this is one of the priorities determined by the community of Wikimedia wikis.

From strategy:Wikimedia Movement Strategic Plan Summary/Increase Participation:

Increasing both the total number of editors, and their diversity, is a key priority for the Wikimedia movement. (...) We need to improve the editing interface in order to reduce barriers to participation. When people try editing for the first time, we need to support and coach them.

and from strategy:May 2011 Update:

Two months ago, the Wikimedia Foundation published the results of our new Editor Trends Study (you can read the results here), which showed that over the past several years it's been getting increasingly difficult for people to edit the Wikimedia projects.

(emphasis mine) So, I think it is unlikely that "prior discussion" will be ever made into a requirement to edit wiki pages. Instead, the chances are that we will have new easy ways to edit pages (e.g. Extension:InlineEditor, which you can test here) and will get more edits from inexperienced editors

Howief (WMF) (talkcontribs)

True, the ultimate goal is to get more of our users editing article pages. But we also want to get the right types of users and we want to keep them for as long as we can.

It could be the case that discussions offer the best introduction to editing Wikipedia, and users that first edit a discussion page tend to stay longer (e.g., they have a better understanding of the rules, know how to avoid newbie mistakes, etc.). This would argue for making the invitation to discuss more prominent, simplifying the talk page interface, improving the feedback loop to the newbie once she's entered into a discussion, etc. Or it could be the case that completing a very simple edit like a spelling correction offers the best entry into Wikipedia (e.g., newbies are shy, they often feel like they're not qualified to edit, so accomplishment is a good way to build confidence). This would argue for making this type of editing as simple as possible (e.g., in-line editor). Or maybe the most effective way of drawing new editors in is to have them create an account (e.g., it gives them a sense of belonging and therefore more ownership over their contributions). And we need to measure this against the quality of these editors. The last thing the community needs is a bunch of disruptive editors coming in through one of these entry points.

The reality is probably a mix of these reasons, and many more. Also, an "entry vector" that works for one newbie may not work for another. At present, we don't have quantitative data that enables us to compare the effectiveness of these various "entry vectors". But it's something that we'd like to test. For example, we have indication that AFT is probably a good entry vector, given the 17% click-through rate for the edit call-to-action. But we don't yet know whether these individuals actually make good editors (e.g., how constructive are their edits? How long do they edit for?).

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

I agree with Howief and Helder. And I think resources should be focused on developing more user friendly tools like LiquidThreads (and perhaps also the InlineEditor) and invite readers to the talk pages instead of the Article Feedback Tool.

I want to interact with other users who actually work and reason, not readers who just may want to express an opinion. I don't understand why the developers choose to prioritize the latter before the former. The underlying problems need to be fixed, not just the symptoms.

Howief (WMF) (talkcontribs)

I think that's a good way of looking at things. We want features that help us get the "right" kind of editor in the funnel, not just any editor. That's why I'm a little skeptical about the results of experiments like the section edit icon -- this project increased the number of edits by a significant amount, but it may be because the section edit link is nice and shiny and therefore attracted people who really shouldn't have been editors in the first place. We haven't done the follow-up analysis for this project, but my guess is that the cohort of users that come in through the section edit link icon are probably of lower "quality" -- their edits are probably reverted at a higher rate and they probably don't stick around for quite as long. This is the type of analysis that we need to do with AFT. The click-through rates give us some good indication that users respond to an invitation to edit, but the real question is whether these editors will end up becoming constructive Wikipedians.

The invitation to discussion is something we really need to dig into. I agree that we should focus on developing more user friendly tools for discussion (like LQT). One of the underlying problems is that there are disconnects at many levels between readers and the idea of discussion on Wikipedia. Some don't even know discussions take place. Some think they take place but aren't sure where to find them. And those that can't find them are confounded by the interface since it doesn't look like any discussion page they've seen before. Brandon's done a lot of thinking around the next generation of LQT. But this is a much longer-term project -- to get this right will take a significant amount of time to design, develop, iterate, etc.

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

I think that both the section edit icon and the InlineEditor will encourage readers to become editors. And they do it in a more direct and honest way than the manipulative method AFT employs. Both SEI and IE put the newbies exactly where I prefer them: In the thick of it. But we need better tools to guide them once that first step is taken:

  • Positive feedback as soon as possible
  • Instant help when needed

In my mind, reducing churn should happen before heavy recruiting starts. Making sure that first edit is a pleasant experience is paramount.

Howief (WMF) (talkcontribs)

Yes, agreed on positive feedback and instant help. The positive feedback needs to be given to the new editor in a manner in which they understand. We can't assume new editors understand the purpose of their user talk page, nor can we assume they even know that such a page exists. I'm also not sure how likely it is that these new editors either notice or act on the yellow notice across their screen. Email notification might be a better way to make them aware of the feedback, but not everyone provides an email address upon registering.

Instant help would be awesome.

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)
  • Positive feedback: Perhaps a tool that shows edit diffs by users who does not yet have an edit on their talk page. Experienced users monitoring that page should then be able to "one-click-welcome" users making useful edits from a selection of welcome templates; "Thank you for copyediting!", "Thank you for adding information!" and so on. I'm thinking along the lines of the (now defunct?) Interwiki-Link-Checker (http://toolserver.org/~flacus/IWLC/start.php) where bilingual wikipedians could compare two wikipages and answer yes or no (or maybe) to the question "create iw links between pages?"
  • Instant help: Perhaps a chat on the tool-server where experienced users offer help in a one-on-one chat with newbies.
He7d3r (talkcontribs)

There is the Extension:WebChat which is currently used on Translatewiki.net (example).

Timl2k4 (talkcontribs)

"we will have new easy ways to edit pages (e.g. Extension:InlineEditor, which you can test here) and will get more edits from inexperienced editors"

Well that is slick! Glad to see that in the works. As far as the discussion page, a little entry form for article feedback perhaps? Also make the discussion option more prominent somehow.

He7d3r (talkcontribs)

See also Visual editor and progress on editor (on Wikitext mailing list)

As for the "a little entry form for article feedback perhaps", do you mean something like the free-text field mentioned on extended review page?

Timl2k4 (talkcontribs)

Yes, I think so. I have a feeling it would end up with all sort of spam, useless comments. There would need to be an easy way to filter them. Having a captchca would probably dissuade 90% of the population from leaving a comment, I mean the useful ones too, although if someone has a useful comment, they may be more motivated to work through the captchca.

After leaving their feedback they could be presented with what would I hope eventually be hopefully improved ways of article discussion and editing than what currently exists.

I'm going off a bit of a tangent here, but if someone has a fact to add to the article, walking them through the process of adding a proper citation (and helping them find one if they don't and making the edit process very user friendly would help turn readers into constructive editors. If they simply have a correction, hopefully this will become more user-friendly, I mean user-friendly like my grandmother could make a correction without messing up the page. The examples I've seen in this discussion (excluding the AFT) are encouraging.

Reply to "Delete AF: "direct reader engagement" is already available on the discussion page, focus your efforts there."
Doug Weller (talkcontribs)

I don't think I could be the only one that thinks this shows this system is failing. Most of the highest rated articles are ones that are susceptible to ratings by fans, and the lowest look like the victims of politics.

This post was posted by Doug Weller, but signed as Dougweller.

He7d3r (talkcontribs)

FYI: There is an open bug (bugzilla:30334) requesting additional options for dashboard.

Reply to "Pages with highest/lowest ratings"
Risker (talkcontribs)

The absence of any opportunity for the reader to actually make a suggestion for improvement of the articles is a critical absence in this "feature", and is antithetical to the processes that make Wikipedia what it is. We already see articles heavily watched by fans receiving ratings that clearly have no basis in the quality of the article, and I have had four FA-level writers state that they will be stopping participation in the project because their articles, already assessed for quality, are now being "rated" without any rationale for these ratings and completely absent any comments that could explain the ratings. They feel it is one more step toward the facebook-ization of Wikipedia, and it is hard to say they're wrong when there's no reason to believe that rating of articles will result in increased contributions or increased quality.

Please add a free-text box labeled "suggestions for article improvement" or the like, with the written commentary being fed to the article talk page and signed "Article rater" or the like. (Do not use IP addresses, because people responding to this section will not be editing per se but responding to a survey. Their identifying information should not be exposed publicly.) Yes, we'll still get a pile of "fanboy" or "anti-fanboy" comments, but these can be moved off the talk pages as needed. This will serve to introduce the readers to the concept of actively participating, while also giving editors information to improve articles. As it is, this feature is more likely to increase the gulf between readers and editors than it is to bridge the gap, and has little likelihood of turning readers into editors. Risker 04:26, 14 July 2011 (UTC)

Howief (WMF) (talkcontribs)

Thanks for the thoughtful note Risker.

"Suggestions for improvement" is a great idea. During the office hours we held to discuss this feature, I asked what input from readers would be most helpful for editors. What's missing/are there holes/etc. was a theme that came up. I also think that engaging readers in thoughtful commenting could put them one step closer to becoming contributors.

There are a number of ways to get this type of feedback. A free-text box, as you mention is one. Jorm lists a few others in a different thread on this topic (e.g., article needs more photographs, references, etc.). The trickier part IMHO is where to surface the reader input. The downside of directly entering comments directly on the talk page is that, as you say, editors will then need to move the superfluous comments off the talk page. This strikes me as potentially creating a lot of work for with uncertain payoff for editors. One idea that has been discussed is the idea of a “parking lot” for comments. This “parking lot” would be a separate space for comments where editors can then review and then promote helpful comments as needed. I think suggestions for improvement make a ton of sense – it would be good if we could implement the feature in way that doesn’t create unnecessary work for our editors.

We could certainly use your perspective in further developing this feature -- would you like to be part of the workgroup that helps design the commenting function? Or if you think the community would be okay with having comments directly on the talk page, we should discuss this option in greater detail.

This post was posted by Howief (WMF), but signed as Howief.

81.109.118.115 (talkcontribs)

Comments directly on the talk page would be a problem I believe. Some would undoubtedly just be forum style comments which should be removed, as you say giving editors more work. But some would almost certainly be BLP violations and that concerns me a lot more. Sure, readers can edit the talk page now, but I think you'd see a lot more BLP problems if it was easier to get your opinion onto the talk page. 81.109.118.115 12:15, 14 July 2011 (UTC)

DarTar (talkcontribs)

Risker, that's a very important suggestion and one that we see frequently mentioned in the survey data. How to make reader feedback specific and actionable and how to further engage with the reader after the rating is submitted are two of the main problems we need to focus on for further development. I added some ideas and suggestions in this sense to the call to action review page and I'd love to hear your thoughts.

Reply to "Immediate improvement needed"
Wasbeer (talkcontribs)

It may be a good idea to fix some of the bugs before deploying the AFT.

http://i.imgur.com/KHa0h.jpg

The words "gaps" and "optional" are in the same spot. Please avoid making the AFT even bigger.

(FF v5.0, default text size)

http://i.imgur.com/vOreG.jpg

(Chrome 12.0.742.112, default text size)

The rest of the bugs can be found here: https://bugzilla.wikimedia.org/buglist.cgi?query_format=advanced&list_id=16547&component=ArticleFeedback&resolution=---

Jorm (WMF) (talkcontribs)

Rollout is beginning today (within about three hours, I think), in increments of 5,000 articles. So there won't be time to address bugs before then.

However, we can (and will) address many design concerns in future iterations, post-deployment.

Howief (WMF) (talkcontribs)

Thanks Wasbeer for pointing out this bug and logging it bugzilla with all of the relevant details on your system environment. This is very helpful.

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

Two things:

  • I think it's irresponsible (and a little arrogant) to deploy a tool with known bugs and with considerate objections from users.
  • I am yet to be convinced you have community support for deploying this on the English Wikipedia. Can you please supply me with a link to the source of your strongest argument that you have this support?
Howief (WMF) (talkcontribs)

This is not a showstopper bug.

I'm working with Dario on putting together some of the most recent research, which will be published shortly. In the meantime, here are some links to quantitative research and qualitative research.

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

Fair enough. Not a showstopper bug. But the links you supplied were not the link I asked for.

Howief (WMF) (talkcontribs)
I'm working with Dario on putting together some of the most recent research, which will be published shortly. In the meantime, here are some links to quantitative research and qualitative research.

Please be patient :)

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

I doubt the research will contain evidence of support of the AFT from the community of editors on enwp (which is what I'm asking for), but sure, I'll be patient. :-)

Reply to "Bug"

Option to close it off the page instead of globally in preferences

7
B0cean (talkcontribs)

I think this is a good idea, but instead of setting it in our preferences, it should have a close [x] button if we don't want to rate a page. As of now, one page I'm looking at, this tool covers up completely the "See Also" section. Rather ironic that I cannot see the "Also See" section. Annoying!

Jorm (WMF) (talkcontribs)

If the tool is covering up a section, that's a bug, and we'd like to know more about it.

As for having a close button per page, that's unlikely to happen.

B0cean (talkcontribs)

Then check out the "Violin" entry. Cannot read at all the "Also See" section.

He7d3r (talkcontribs)

Confirmed at w:Violin#See_also:

  • Bug 30025 - AFT is overiding the article's content
WhatamIdoing (talkcontribs)

It looks fine on my screen (Firefox 3.6.19), so it may be browser dependent.

He7d3r (talkcontribs)

Indeed. Right now I cant reproduce it anymore on Firefox 5 and on Chromium 12.0.742.112 (90304) Ubuntu 11.04.

Is it still happening to someone else?

Howief (WMF) (talkcontribs)

I can't reproduce this on FF 5 or Chrome 12.0.742.122 on OSX. It looks like this bug isn't appearing anymore. If it resurfaces, please document the browser and OS. Thanks.

This post was posted by Howief (WMF), but signed as Howief.

Reply to "Option to close it off the page instead of globally in preferences"
Randomblue (talkcontribs)

I found the following glitch on the "Tour de France 2011" article (see image, under "Trustworthy"). I'm using Google Chrome and the latest version of Mac OS.

Bug with feedback tool
Howief (WMF) (talkcontribs)

Thanks for sending this, Randomblue. I was able to reproduce this both on Chrome 12 and FF 5: Bugzilla 30021. I'll let the developers know.

This post was posted by Howief (WMF), but signed as Howief.

86.213.118.166 (talkcontribs)

The developer is requiring a screenshot on bugzilla! Someone, please give him the screenshot. 86.213.118.166 02:37, 26 July 2011 (UTC)

Catrope (talkcontribs)

This is fixed now, thanks for reporting.

Reply to "Bug"
Owen S Hughes~mediawikiwiki (talkcontribs)

Article feedback is meaningless, since the incredibly wide spread of users, all with different abilities and values (not to mention prejudices), means that no star rating can ever be truly validated. In addition, the extraordinary number of articles on Wikipedia ensures that a large number of them will receive very few such ratings, even over a lengthy period. I agree with other comments that if this tool is to be of any value at all, and I still have my doubts, it should appear on the article talk page. Lastly, although I feel I could respond to the rating system, I have no interest in doing so because a) see above and b) I would much rather spend the limited time I have available in editing and otherwise contributing to the encyclopedia. (Anyone who has ever used the eBay Review and Guide system will know that it is frequently confused with the eBay Seller rating system, and that both systems are subject to much misuse and abuse.)

This post was posted by Owen S Hughes~mediawikiwiki, but signed as Owen S Hughes.

He7d3r (talkcontribs)
it should appear on the article talk page

Quoting this reply by Jorm (WMF):

Most users of Wikipedia are not even aware that Talk pages exist. Since the goal of the tool is to attract impressions from as wide a range of users as possible, hiding the tool on the talk page is not a viable solution.

WhatamIdoing (talkcontribs)

How one validates a process depends on what one wants to achieve. Eating five pounds of candy every day is a validated process for gaining weight; it is not a validated process for improving your driving skills.

We will not be able to validate these ratings as providing the One True™ Answer about the article's status. However, that's not actually the goal, and nobody has ever promised you that these ratings will produce the One True Answer for anything.

We can validate this process as being a reliable means of determining what our users (who are mostly non-editor readers) believe about an article. We hope that we will also be able to determine that it is a functional method of increasing the rate at which currently non-editor readers become active editors.

Von Restorff (talkcontribs)

Well, it is quite clear that these ratings are not a reliable means of determining what our users (who are mostly non-editor readers) believe about an article.

Reply to "Article feedback is meaningless"
76.104.206.106 (talkcontribs)

What if, after the reader completed the short survey, he/she was given a link to provide additional feedback that linked to the article's talk page to elaborate on any possible issues with the article. Or even link directly to the New Section Tab so he/she can immediately begin writing. I believe this is consistent with the "Reader engagement" bullet point at Wikipedia:Article Feedback Tool, and I also believe most general readers aren't even aware each article also has a corresponding discussion page. 76.104.206.106 20:17, 20 July 2011 (UTC)

He7d3r (talkcontribs)

I think it is good idea to have either a free text field to provide additional information about the article or a link to the article's talk page.

WhatamIdoing (talkcontribs)

My preference would be for a link that creates a new section on the article's talk page, labeled something like, "Would you like to leave a detailed message about the article for other users?" It could be run as one of our Calls to action.

Howief (WMF) (talkcontribs)

The idea of allowing readers to submit comments seems like a good one, though I think we need to be careful about how we introduce the feature. Automatically allowing readers to submit comments directly to the talk page is likely going to result in a lot of noise (see related post). We may want a system whereby the comments go into a separate space where they can get "promoted" to the talk page. I do like the idea of running a call to action around comments -- we may be able to get a sense of what kind of comments readers provide prior to rolling the feature out.

This post was posted by Howief (WMF), but signed as Howief.

Foxyshadis (talkcontribs)

A section on the bottom of the talk page (or top?) that by default shows the first dozen or so words of each statement would be excellent. (Or a title if provided, if it's not required.) It would be expandable like any other table, or by clicking on an entry expanded and jumped down to the full thread. That's easy to see, easy to use, and yet visually uncluttered. Plus if someone wants to put together a gadget to notify interested folks of when there are new threads in the survey section, they can do it, since it'll all be integrated into wiki but sectioned off from the main talk.

I don't think they'll really pile up that high, since the percentage of people who leave comments on surveys is much lower than those that just rate. On popular articles, the talk pages pile up and get routinely archived anyway, so it's just another piece to archive. Half the time, talk pages are used for essentially this purpose already, it's just that new users don't know it.

Reply to "Link to discussion page?"
Howief (WMF) (talkcontribs)

Just a note to let everyone know that we will continue the rollout of AFT starting tomorrow, June 13, 2011. Our current plan is to deploy the feature in increments of 5,000 articles while carefully measuring the performance characteristics. If all goes well, we will continue the rollout over the next week or two.

Please let us know if you observe anything unusual during this rollout period. Also, we're still using the Article_feedback/Ideas_log to collect feature enhancements, so I encourage people to put features ideas on that page.

This post was posted by Howief (WMF), but signed as Howief.

Bensin (talkcontribs)

I suggest you halt this project until you can show you are backed up by the community of the wiki where it is to be implemented.

"Please let us know if you observe anything unusual during this rollout period."

Well... I know it's not what you are thinking of, but the most unusual thing I'm observing is the foundation pushing this forward without showing there is consensus to do so, despite pretty words of taking the criticism seriously.
Jorm (WMF) (talkcontribs)

Here's a statistic from the survey we have been running along with the tool:

  • 92.2% of all the users who took the post-rating survey (N=2132) found the tool useful, only 4.8% didn't find it useful and the remaining 3% didn't answer.
Bensin (talkcontribs)

Any chance those liking the tool are using it and those not liking it are abstaining from doing so?

Also, I'm not talking about support from the users of the tool, I'm talking about support from the users in the community. One user, one vote. I'm talking about a poll like in the case with flagged revisions. Here's what scares me: The foundation is taking upon itself to decide what's best without letting the community have a say in it. To forego such a poll, thus eliminating the risk of a no-go, is a mandate I don't think the foundation has. If it does have the mandate, I don't think it should.

Jorm (WMF) (talkcontribs)

The total of the Wikipedia user community is around 4.8 hundred million people. The sum total of what is typically considered the "community" is about 10,000 individuals.

That's a different level of mandate, I think.

Wasbeer (talkcontribs)

*cough* confirmation bias *cough*

Bensin (talkcontribs)

First: w:Special:Statistics says there are 143,047 active registered users on enwp, the wiki where this roll-out is taking place. That's the community I think should have a say in this.

Second: Your post is not an answer to my post, but rather avoiding my two main objections. It is, however, duly noted that you seem willing to dismiss that community of 143,047 individuals (where everyone can vote) and at the same time refer to a survey of 2,132 users of the tool. I'm simply not prepared to put my trust in that survey. I want a poll.

Jorm (WMF) (talkcontribs)

I'm not avoiding your objections; I'm explaining who we are serving.

This is not a tool for the editor community. This is a tool for the millions of users who don't have a voice because they don't know that they can have a voice. Would you like some comments from these people? The tool is rather well liked by pretty much everyone. It has its problems and bugs, but really there are only four or five people who are actively engaged with promoting its demise.

We are going to roll out the tool. We are going to collect data. We are going to make changes and choices based on what we learn. We were told to do this by the community through the strategic plan. There is already consensus for the tool.

I'm sorry you don't like this; I wish I could help you with that. I (and others) have been listening to your concerns and we are going to fold some of them into future iterations of the tool. You have been heard.

Bensin (talkcontribs)

"This is not a tool for the editor community."

I am painfully aware of that. And it's a shame. Tools could, and should, have been developed to suit both editors and potential editors. One tool for editors to evaluate articles and one (or more) tools and features to engage potential editors.

"The tool is rather well liked by pretty much everyone." and "There is already consensus for the tool."

{{citation needed}} please. I don't contest the survey of 2000+ users of the tool, but on what do you base the support from the user community?

"We were told to do this by the community through the strategic plan."

I doubt you were told explicitly by the plan to develop and implement the AFT. If so, can you provide me with a link to that paragraph? (Not just "improve ways to measure quality" but rather something that supports your statement.)

I have also raised concern about the roll-out itself and what I believe is lack of evidence of community support. Is that something you will take into consideration for future roll-outs of other tools?

DarTar (talkcontribs)

We were reluctant to publish the data from the post-rating survey precisely because it cannot be used to make any rigorous generalization. It is very useful, however, to give us a feeling of how the community at large (which as Jorm notes includes our readers) is responding to this experiment. I find it ironic that, as biased as this survey data surely is, we should dismiss it and take the negative feedback from less than 10 self-selected editors participating in this thread as representative of the true feelings of the community.

If we want to discuss the figures, Jorm omitted one small detail. This is data collected over 1 week, i.e. between 2011-07-04 and 2011-07-11. There has been roughly the same amount of users giving us feedback every week since the tool was deployed to 100K articles. There won't be any poll to decide on this or any other new feature that we are implementing as part of the strategic product plan. As we said before, this project is going on until we have collected enough data to understand how to make it a useful tool for our community at large. Constructive feedback on how to make this happen is, as usual, very welcome.

Bensin (talkcontribs)

I have never claimed to be a representative of the community. I represent myself, only. But I also want other editors to have a fair chance of representing themselves. A poll on the wiki in question would be a good way to do that. If you have support for it, then fine, the problem is all mine. But what if you don't? Is it possible there are other solutions to the problem that you and I have not taken into consideration? Solutions that may arise when users comment in the poll? We all know crowdsourcing is a powerful tool when used right.

I don't want to dismiss the survey, but the question of bias is unanswered.

Wasbeer (talkcontribs)

Jorm described the offer to ask for input from the community as a "threat".

Try it yourself. Ask a random person to make an account on Wikipedia. Explain that it is possible to hide the AFT. Then ask them how to hide the AFT.

Count the amount of clicks it takes for them to answer that question.

Then explain that (s)he is able to give feedback on the AFT itself, and ask them to provide feedback on the AFT.

Count the amount of clicks it takes them to give feedback.

Remember: the fact that talkpages are "hard to find and edit" is one of the things that should be solved by the AFT, but this discussion is held on a talkpage on a different wiki. LT makes commenting here even harder and confusing.

Millions of users who may have an opinion on the AFT don't have a voice because they don't know that they can have a voice. It is very hard to find this page, and lots of people do not know they can be a part of the decision making process. The fact that everyone is able to edit almost all articles is very strange to most people, and most people who do know that do not know they can have an influence on the interface too.

But lets assume, arguendo, there are less than ten people in the world who oppose the AFT. Wouldn't it be nice to have this overwhelming support confirmed by a poll on EN wiki?

He7d3r (talkcontribs)
the fact that talkpages are "hard to find and edit" is one of the things that should be solved by the AFT, but this discussion is held on a talkpage on a different wiki. (emphasis mine)

Good point. I think it could be useful to let people to comment about the tool directly on the wiki the tool is being used.

LT makes commenting here even harder and confusing.

Although personally I prefer LQT (because it has being in use on Portuguese Wikibooks for a long time now, and I like it - despite its bugs), I agree that it can represent an additional barrier to people who are accustomed to using the normal talk pages and village pumps.

Jorm (WMF) (talkcontribs)

Remember: the fact that talkpages are "hard to find and edit" is one of the things that should be solved by the AFT

This is not one of the motivations for the Article Feedback Tool and never was.

Jorm described the offer to ask for input from the community as a "threat".

This is being disingenuous. Your language said "I will rally the troops" and indicated that you would canvass people to storm the talk pages in order to "prove" something to me. Flooding a constructive discussion with non-constructive criticism would be pointless, and I called you on it.

Wasbeer (talkcontribs)

Where did you get that quote from? I never said "I will rally the troops", nor has anyone else here afaik. Please use quotation marks only for direct quotes.

Accusing me of being disingenuous is rather silly considering the fact you made that quote up yourself and you have been disingenuous in the past and removed the evidence when I called you on it.

I asked you twice if you wanted me to provide an indication that a direction change is required. The second time I wrote: "OK, do you want me to provide that indication? I can rally the troops if you want me to."

Emphasis added by me.

You said my comments "have ceased to be constructive in any way" on the 19th of June, but a lot of people here probably disagree.

For example, user Helder said I made a good point here, and user Subfader said I made some good points here, and user Tom Morris said I made a good point here.

User Rockfang thanked me for replying here, user Bensin agreed with me here, I asked a couple of questions and both user Dougweller and user Bensin said they would like an answer to these questions too here (under my comment), I made a request and user Helder copied it to the bugtracker "to make sure this is noticed by the developers" (see here).

Howief described the fact that I pointed out a bug here and filed a bugreport on bugzilla with all of the relevant details on my system environment as "very helpful".

These are just a few examples I could find quickly, more of those exist.

I'm afraid you might take the criticism of the AFT a bit too personal. This is not a personal thing. It isn't even very important. Most people do not have the time to debate about silly stuff like this because they have real problems. You even wrote: "Is your primary objection to its being there based around the fact that you are the primary editor of the article and that you feel that low ratings reflect poorly on you?" even though I changed nothing on that article.

Now you seem to be saying I pointlessly flood a constructive discussion with non-constructive criticism, and that you called me on it.

Please comment on content, not on the contributor.

Subfader (talkcontribs)

You made a good point with that one argument. That doesn't mean I agree with all your posts. TBH I think you completely ruined all serious discussion with your aggressive hater posting style. And to leave an objective comment: The happy majority never particaipates on new feature discussions. I think most people find it more useful than a waste of coding time. --Subfader

Wasbeer (talkcontribs)

Of course I know you were just talking about that specific comment, I just used that as an example to prove Jorm wrong.

Thank you for your honesty. As you probably expect, I disagree (I was much nicer to Jorm than he was to me, especially on IRC today), but I decided to move my attention elsewhere because this is obviously useless. Good luck everyone!

Timl2k4 (talkcontribs)

This just shows an incredulous level of cluelessness on your part. Why would someone use the tool if they don't find it useful? Once again, plain evidence that you are only supporting this tool because of the time and effort you have put into it. Not on an objective basis.

He7d3r (talkcontribs)

I have to agree that this "statistic from the survey" do not show "you are backed up by the community of the wiki where it is to be implemented", since it doesn't consider any data about people who doesn't use the tool (e.g. because they do not like it).

DarTar (talkcontribs)

If studying the data we collect from this tool makes AFT "our baby", then you've got a point. I wish you stopped reacting as if there was a secret agenda behind this tool different from its stated goals. There is actually quite a lot of criticism in the comments of the post-rating survey even though the majority of the respondents is supportive and we're trying to be as receptive as possible about future improvements.

213.134.175.225 (talkcontribs)

Article feedback tool developer are completely ignoring feedback from community. Sooo great! What is possible for community: hide class "articleFeedback-panel" as default for all skins

This post was posted by 213.134.175.225, but signed as Bulwersator.

He7d3r (talkcontribs)

Unless there is no community consensus on the proposal of hiding the "articleFeedback-panel". But so far, this was not proposed on en.wp.

WhatamIdoing (talkcontribs)

Bulwersator, there's no question of "completely ignoring feedback from the community". Most of the community supports having some version of this tool. Only a small group of people oppose it. If you don't like it, then you can disable it for your account, but you really ought to quit pretending that the rest of the community agrees with you.

Bensin (talkcontribs)

"Most of the community supports having some version of this tool"

I'm not convinced. Even if it were true, it is unclear what that version is.

"If you don't like it, then you can disable it for your account"

One could also argue "If you like it, then you can enable it on articles in your watchlist and then update the articles according to the input from the tool". There is very little talk about who will actually do the work suggested by the data from the tool.

I'd also be a little surprised if the AFT will identify substantially more articles as "in need of attention" than is already identified by the community itself via templates and flags.

He7d3r (talkcontribs)

One could also argue "If you like it, then you can enable it on articles in your watchlist and then update the articles according to the input from the tool".

But to enable the tool by themselves, people would first need to know that it exists, and this is already a problem with the talk pages, which a lot of people doesn't know about.
Bensin (talkcontribs)

Your're talking about readers. I was talking about editors. That editors could enable the tool and then also take care of the comments and suggestions the tool provides and remedy the article's shortcomings. That way there is less risk of readers pointing out problems that are not taken care of.

It's frustrating when someone asks your opinion but don't listen to your advice.

WhatamIdoing (talkcontribs)

The primary target of the tool is non-editor readers. The English Wikipedia already have two very good systems in place for getting feedback from experienced editors: the talk page and the 1.0 team's assessment program. The whole point of this tool is to get non-editor readers engaged in the Wikipedia page—the very people who don't (and with respect to the second system, shouldn't) use those feedback options.

I don't think that these readers feel like their "advice" is being ignored. The typical reader probably thinks his rating is being given just as much attention as his Facebook "like" votes. Perhaps, though, this refers to your own feelings, resulting from other people rejecting your advice to make the tool unavailable to its intended audience?

Bensin (talkcontribs)

"Perhaps, though, this refers to your own feelings, resulting from other people rejecting your advice to make the tool unavailable to its intended audience?"

No, you got it right. I was referring to the voters, but it sure applies to the roll-out of the AFT too. With the major difference that the community was never asked what it wanted before the roll-out. If you believe it was, please supply a link to that discussion.
174.61.248.93 (talkcontribs)

I certainly don't speak for the foundations or the developers behind this tool but from an outside in perspective, this tool is more for the readers and to get their opinions rather than something designed for the community of editors itself.

Therefore, this "survey taking" of readers opinions doesn't seem like something that needs "community approval" since it is really not about the community at all. Thus, I don't see how the community can even have veto power over this or a reason to cry about lack of consensus. Again, it is not about them. It would be like the WMF deciding to switch to a new public relations firm and the community crying foul because they weren't consulted first for input.

Now....of course, there could be benefits of this tool for community and THAT is what I see the developers asking for input on as a courtesy to the community. With some tweaks and observations on how the readers are using the tool, AFT can be tailored to have some useful by-products that editors can use via dashboard enhancements, etc. But those benefits are only by-products and not the main purpose which appears to be getting reader feedback.

WhatamIdoing (talkcontribs)

It's not possible to supply "a" link to discussions, because there have been multiple discussions over many months, which have generally either been supportive or neutral. I and others have provided links to more than a dozen such discussions and announcements in the past, and I'm sure that there are more that I didn't see at the time. I don't have time to go look up a long list again, but I believe that you will find at least some of them at the end of the FAQ page.

In the meantime, I recommend that you think about what en:WP:Consensus says about the relationship between discussion and consensus: discussion might shape consensus and it might document consensus, but it is not a required step in forming consensus—and it is definitely not necessary to hold a single, organized, long, large, noisy discussion to have a consensus. We have a consensus when most people agree on something, even if nobody says anything out loud. It is not necessary for even normal editors to seek "written permission in advance" for good-faith changes. It is certainly not necessary for the WMF to seek written permission to make good-faith changes.

Fact: The WMF has surveyed thousands of users about this tool, and only about 5% have said that the tool is not useful. That's far less opposition than every single one of en.wiki's major policies received at the time of its adoption, and far more people expressing an opinion. You are doubtless one of the people in that 5%, but 5% out of thousands of users is never going to be accepted as a consensus against the tool.

Subfader (talkcontribs)

"It's not possible to supply "a" link to discussions, because there have been multiple discussions over many months"

Then post 2 or 3 instead.

WhatamIdoing (talkcontribs)

There are more than that linked at the FAQ; have you tried reading it?

Reply to "Continued Rollout of AFT"

Justin Bieber article not complete?

19
Ziko (talkcontribs)

Hello, I just saw that the article en:Justin Bieber is rated only 3.24 with regard to completeness. I did not read the article thoroughly, but I am surprised that such a long (and most certainly very complete) article has no better rating. It seems that some readers' attitude to the article's object have caused that. This shows a not so mature behaviour of some readers.

Wasbeer (talkcontribs)

Yep, that is because people will base their ratings on the subject of the article and not on the article itself. If you give it enough time and data it will become a rather accurate popularity rankingsystem for BLP subjects, not an accurate rankingsystem for articles.

Bensin (talkcontribs)

I'd be very interested to hear a response to this from the proponents of the project.

DarTar (talkcontribs)

Yes, that's an interesting point partly related to my previous comments on trending topics vs. quality. We definitely need to think of how to cope with these effects. I am persuaded the solution should not apply to the rating collection itself but to how we calculate aggregate scores: for example we should filter out suspicious patterns that indicate that an article is being rated as a function of the topic's popularity in the media. Having more data on multiple ratings by the same users or IP addresses should also allow us to study the reliability of ratings. Unfortunately, because we are still using a small sample of articles, we have very little data on people who rated multiple articles.

Wasbeer (talkcontribs)

The easiest solution is to not ask for ratings but instead asking to comment on the article. Those comments can be displayed on the talkpage.

If, for example, someone gives the Bieber article 5 stars in every category, it means "OMG Justin Bieber is so cute I love him!". If that person wrote that in English on the talkpage then we would be able to understand the reason why that person rated that article in that way.

The same principle applies to 1 star ratings in every category, they basically mean "Fuck Bieber".

We should not ask for ratings, we should ask why people rate an article the way they do.

We shouldn't invent a problem (not being able to determine the intentions of the person who gave the feedback), then try adjust algorithms so the effect is less extreme.

Doug Weller (talkcontribs)

Wasbeer, if someone wrote that on the article talk page it should be removed. Talk pages aren't forums and I hope that this doesn't encourage people to use them as forums. Dougweller 07:45, 9 July 2011 (UTC)

This post was posted by Doug Weller, but signed as Dougweller.

Ziko (talkcontribs)

Any solution of the feedback problem that involves the Talk Page is *not* a solution. Usually people don't know about that page, and it is very difficult to edit it. (Yes, it is!) I like the Polish Wikipedia comment function. But this has little to do with this feedback function. Maybe it would be good to explain to people how to rate; that some typos don't mean that the article cannot have five stars.

Timl2k4 (talkcontribs)
Any solution of the feedback problem that involves the Talk Page is *not* a solution. Usually people don't know about that page, and it is very difficult to edit it.

Wrong, wrong, wrong! So wrong in fact I said it three times. The talk page is EXACTLY the place for constructive article feedback. If people don't know about it, that needs to be fixed. It it is too difficult to edit, that should also be fixed.

I bolded constructive, because any other feedback is useless.

Wasbeer (talkcontribs)

If you have a "feedbacktool" that allows people to leave a comment in a free text field on the article you can display the feedback on the talkpage.

Timl2k4 (talkcontribs)

That's pretty close to what I was thinking, the question is how to display the feedback and filter out garbage. Perhaps more seasoned Wikipedians could pick out specific feedback items and move them to the main discussion. Or at the very least take the feedback into account when trying to improve the article, without having to work off of vague ratings like the AFT would produce.

Ziko (talkcontribs)

Yes, that's what the Polish have. And it works; the comments are easy to fill in and are sent to a wiki page where experienced Wikipedians look at it. But our usual talk pages - they don't work. :-)