Topic on Talk:Article feedback

It would be nice to have a somewhat official statement of a (wmf?) supporter of the AFT that assures us that the planned full deployment on all English Wikipedia articles on 31 May 2011 is not going to happen because of the lack of support in the community and the fact that the AFT isn't a positive addition at this point in space and time.

Now we are left guessing what will happen. Jorm asked for a indication that a change of direction was required, the community already has made that very clear, but if there is any doubt about it I am willing to ask the community to provide more input. This sword of Damocles makes me uncomfortable, especially since it has been announced without consulting the community and getting consensus.

May 31, 2011 was just under a month ago, so I can assure that a full deployment will not occur then.

LOL. Oops. Did someone pick another date or is it that plan aborted?

The 31 May date was actually incorrectly reported; it was a "swag" date thrown on the wall several months ago.

We are planning a ramp up but it will be done in stages.

I found these quotes: "We're planning to ramp-up use of the feature starting July 7 through July 21 (2 week period). This schedule may still change." and: "These ideas are described in more detail on the extended review page".

Unfortunately this idea is not described in more detail on the extended review page. Another possibility is that I misread or missed something, if so please link me to the correct location.

The obvious questions are:

• Why are you planning to ramp-up use of the AFT despite the feedback by the community?
• Where is that description of the "ramping-up process" in detail?
• Is there consensus for this ramped-up deployment on the English Wikipedia (please provide a link)?
• Do you have an indication that a change of direction is required ATM? If not, is it OK if I ask the community to provide input?

I feel like I am repeating myself but it would be nice to have a somewhat official statement of a (wmf?) supporter of the AFT that assures us that the planned full deployment on all English Wikipedia articles is not going to happen because of the lack of support in the community and the fact that the AFT isn't a positive addition at this point in space and time. Now we are left guessing what will happen. Jorm asked for a indication that a change of direction was required, the community already has made that very clear, but if there is any doubt about it I am willing to ask the community to provide more input. This sword of Damocles makes me uncomfortable, especially since it has been announced without consulting the community and getting consensus.

Ofcourse it would be nice to get a response before July 7...

I also would like an answer to these questions and a publicised discussion on en.wiki before this happens. Dougweller 05:52, 1 July 2011 (UTC)

This post was posted by Doug Weller, but signed as Dougweller.

... and me too. Ask the community what it actually wants before fully deploying something like this. The WMF attitude in this matter makes me very uneasy and indicates a shift in its view on its position in relation to the community.

Lets wait until the 7th. If nothing happens we can use the village pump to announce that the community decided to stop the experiment with the AFT on EN Wikipedia because of the lack of support in the community and the fact that the AFT isn't a positive addition at this point in space and time.

Then we can file a bugreport asking to remove the AFT from all articles on EN Wikipedia.

The Foundation has not changed its position regarding the importance of the project and the data we can derive from it. The movement of the date does not in any way indicate otherwise.

I'm not certain that the 7th is our launch date; these things are being done at a slower speed than was previously expected due to several factors (vacations, other projects taking precedence, personelle transitions, and conferences). You'll note that the text of the page says, "We're planning to ramp-up use of the feature starting July 7 through July 21 (2 week period). This schedule may still change." (emphasis mine).

I think the shift mentioned is not about this project but about its relationship to en.wikipedia in particular and perhaps others. The issue seems to be is the WMF going to take into account the opinions of en.Wikipedia editors - indeed is it even going to consult them - or is it just going to carry on with this regardless. Hopefully there's some sort of misunderstanding or lack of communication here. Dougweller 05:45, 4 July 2011 (UTC)

The community > The foundation.

The position of the foundation regarding the importance of the AFT is not very important to be honest.

I understand that you, as an WMF'er, think it is or wish it was; but it isn't. Sorry.

We, the community, are Wikipedia. If the WMF completely disappears tomorrow we will start a new foundation if we decide we want to have one. If the community disappears then Wikipedia is dead.

My advice to the supporters of the AFT is simple:

• Never ask for feedback if you are going to ignore it anyway
• Think about whether the community or the WMF should make this decision
• Respond to the feedback, start an honest discussion, make compromises (Move the AFT outside div#content et cetera)

Its the 4th of July. You still have a chance to respond to the feedback before the deadline.

Does this mean that factors such as foundation vacations, other projects taking precedence, personnel transitions and conferences are valid reasons for slowing the project down, but community concern is not? Or do you not recognize that at least part of the community have issues with this?

Also, out of curiosity: What is the WMF estimation of the ratio of editors who "support"/"don't care"/"disapprove" of this project?

The schedule for the AFT rollout has been changed to July 12 (at the earliest) due to a dependency on another feature we're working on.

Jorm, Dario, Erik, others at WMF, and I have been following the feedback on this discussion closely. There are both supporters of the feature as well as people who disapprove of the feature. I don't have a precise ratio at this point.

There are legitimate concerns around number of issues, including potential gaming of the feature, the usefulness of the ratings/ratings categories, the dashboard algorithm, the visual layout of the tool etc. But the current implementation isn't the final implementation. As with any feature, we will analyze the data, get feedback from the community, conduct further user tests in an attempt to improve the feature over time. I would very much encourage everyone to take a look at the data to see how the feature is being used. For example, the raw ratings data can really help us identify gaming patterns so that we can develop features/filters to help manage the amount of gaming that takes place.

One thing that I would like to reiterate from Erik's previous post on the topic -- this is the first time we're engaging readers outside of the edit button. In my opinion, that's really important in light of the overall health trends we're observing. Readership keeps on climbing, yet these readers aren't becoming new contributors to Wikipedia. This feature is an opportunity to engage these readers in hopes that some of them will eventually become Wikipedia editors. Dario just did an updated analysis of the calls-to-action, and 17% of rates that get the edit call to action end up clicking the edit button. Out of those, 15% end up completing the edit. I will ask him to post those results. Those are very encouraging numbers and suggest that a rating feature can really help draw in our readership. That's very, very cool.

This post was posted by Howief (WMF), but signed as Howief.

"this is the first time we're engaging readers outside of the edit button." Wrong, wrong wrong! Have you heard of the discussion tab by any chance? It would be soooo much more fruitful to focus on making the discussion tab more accessible and useable to readers than this effort.

The discussion tab requires an edit to be made, so it doesn't seems to be really a way of engage readers "outside of the edit button". ;-)

Actually, all this is required is to hit "New section". This is still very non-obvious to the newbie, but technically it's outside of the edit button! :-)

"17% of rates that get the edit call to action end up clicking the edit button"

Is there any evidence that the rater did not intend to edit after submitting the vote anyway? Also, how many of those 17% were anonymous users?

The 83% who didn't click the button and the 85% of those who did who did not complete the edit are evidence of how many "did not intend to edit after rating". We only have aggregate data regarding CTA but the vast majority of people who use the AFT (more than 95%) are anonymous users.

So 95% of 15% of 17% = 2.4% of the users of the tool were anonymous users making an edit and assumed to do so with help from the AFT. Is that correct?

yes, that's a fair estimate – even though we only have aggregate data on Edit CTA events and we cannot yet analyze them by category of users.

Do you have any data on how many anonymous users reading an article used the AFT? Do you have any data on how soon after an article was loaded that it was rated? What is the mean size of an article in the AFT experiment?

We don't have data on anonymous readers -- we don't track what our users read. We've considered using some metric like time-to-rate (time between page load and ratings submission), but that was a lower priority than capturing just the basic information.

On the research page, there is data on the article length distribution of the one of the earlier sets (727 articles as of Feb 2011 = PPI + additional articles).

Revision length at the time of rating submission is captured. Please feel free to dig into the data.

This post was posted by Howief (WMF), but signed as Howief.

Yes, time-to-rate is what I'm asking for.

"I don't have a precise ratio at this point."

I didn't ask for a precise ratio. I asked for an estimation. The question of community support is important, don't you agree? If the foundation were to poll hard facts in this matter, as with flagged revisions, and get support for continuing this project, I would drop this in a heatbeat. So:

What is the WMF estimation of the ratio of editors who "support"/"don't care"/"disapprove" of this project?