Thread:Talk:Article feedback/confusing: do you need to fill in all the vote categories?

Hi, I just saw your new Article Rating Software Experiment.

Trying to rate an article that was obviously too short, unfinished and incomplete, I got confused because I didn't really know what to vote for in each of the categories. Obviously I'd vote low points for "Complete" and "Well-written", but what about "Trustworthy" and "Objective"? High medium or low?

I was almost going to give up because I didn't want to spend *that* much time on this new rating experiment either. But then I thought, maybe I can vote just for two of the four categories and leave the others empty?

So I tried it and it *seems* to go through, says it's submitted and all that.

I have no idea if this is expected behaviour, but if you want to allow people to vote for one or more rating categories, you should explicitly say so, because otherwise (some) people (like me) will initially assume you're required to fill out the entire form, which will negatively impact your conversion rates.

It's probably too late in this experiment, but I think that voting stars in four categories is a dumb way to go about it. Who came up with that? There's a whole SCIENCE of research gone into survey-making and taking, people, and it spans DECADES of experience. Yes, this type of questionnaire works usually very well on paper surveys, but we've had the Web for a quarter of a century now, and we know this doesn't work very well online, presented like this. The difference between a paper survey (or the online questionnaire equivalent) and a quick online feedback rating is that with the longer survey, people already committed to filling out the entire thing. That's why Social Research surveys often ask the same question in different wordings. People don't really mind and you get better results. But for quick online feedback forms, people do mind having to fill in more than is absolutely necessary (read any primer on "crowdsourcing"--bullshyte term as it may be).

So three stars is the "neutral" vote, yes? Is this different from not voting for a category? I assume so, because that would be the most "natural" route to go about it, if you don't particularly think it through.

Except that it's completely counter-intuitive from a usability perspective, because "not voting" will look like a "zero star rating", which is NOT what you want. The solution is to make all four ratings start at 3 stars ("neutral") when the user hasn't filled anything in yet. Yes, this will make your numbers mean something different than they meant until now, and yes this will mean that "not voting" equals "neutral vote", which is different from what you've had so far (but not necessarily better or worse). But most importantly, it will mean that the numbers you do get, will have a better chance of coinciding with what the user expects their feedback form submission to mean.

Another idea: You want less buttons. You want one five star "overall quality" rating that is mandatory, and the four categories simply a plus, neutral or a minus. All four start at neutral, so they're optional. Problem here is, you better treat the plus and minus votes as the same thing, because some people are going to confuse voting "double negatives": if you give a low 1-star overall quality, does a minus for "Objective" mean "not very Objective" or "does not apply to Objective"--if you get my point.

Which brings me to a better idea that you can already implement in the current rating system, you know how paper surveys always list the choices as "Un-objective", "Somewhat un-objective", "Neither objective nor un-objective", "Somewhat objective" and "Objective"? (yes I know "un-objective" means "subjective", dunno which one is easier to understand though). This is where web-forms have a distinct advantage over paper, you can actually have this line of text below the stars and change as you change the rating. You really SHOULD implement this, any textbook on Social Research will tell you it will make the user ratings a lot more consistent.