Thread:Talk:Flow Portal/Community consent?/reply (7)

The reason why I started my post with "of course" and "it's even the recommended approach to achieve usability" it's because it can be done, and it really is the recommended and standard approach to achieve usability. There are several professional fields (user-centered design, user experience, information architecture, interaction design, and anything with an UX in its description) which are all based around processes that depend on it being true.

The tl;dr version of the long post below is that users can give informed consent if you don't expect them to achieve anything useful with the broken/unfinished software; the consent should be given to provide feedback about what is actually built at any given time, not what is in the head of the production team as future possibilities. (And again, the "no true ability to judge whether it meets their needs" is a false assumption - the user can always judge whether the unfinished product meets their needs, and that is exactly the question that must be asked). The good thing about asking about the current version, and not your planned fixes, is that you don't just keep talking about the great design that you think will solve all the user's problems - you have to actually test whether it solves them or not. This is the scientific process at its best.

Of course if all the information you provide is "Hey, go to this page to test this really cool new thing" you're not gaining anything. But you originally asked me if the user could give informed consent to "try out" the software, and now you've shifted to ask whether they can "use" it. Those are very different things, and by conflating them you're asking the wrong question and thus making a serious mistake in the way you approach end-user involvement. The message to the user as well as the way to test the software should be "Do you want participate in an experiment to help us build a better version of Thing? The experiment consists in that you try out this Thing, we see how you try to use it, and you tell us what you think of it".

The goal of these early interactions must not be to assess whether the user can use the software, which means that they could successfully complete tasks that meet their goals with it. Of course they can't do that with an early version; it's unfinished, and it has been designed with only a cursory understanding of their needs, based only on early field research (and that if there's luck; most software is created just from developers' gut feelings).

No, the purpose of early testing is to let users try out the software with the only goal to get feedback from people using the software, watching exactly where they have problems with the design, and asking them if they can make sense of the interface.

If your only goal is to get this feedback so that the product can be later improved, of course users can give informed consent for this kind of sessions. See this example of the process required to ask for this consent. When users are guided through a session like this to use the software, they are perfectly capable to assess whether they're willing to proceed and to answer whether it's useful to them; an answer that for these early sessions should always be "no, I can't use it ([which is expected at this time]), because of reasons X, Y, Z ([which is the valuable thing to learn]). A technician performing the sessions that is experienced in the field can make the most of this feedback so that few users need to be tested. The most important part of the test is not only what users say, but what they do - i.e. whether their use of the software corresponds to how developers thought it would be used.

Of course for a site the size of Wikipedia you need more than a few in-place user tests; the kind of unsupervised feedback you talked about is what I described as step 6, for which users can also give informed consent of the exact kind you asked, provided you've built a system that reasonably fulfill their major needs through steps 1-5; and, much earlier, a lot of research must be done even before design begins (that was step 1, creating surveys to compile user needs from the wide user base). But you'll need actual users testing the unfinished product through steps 2 to 4 to be sure that the software makes sense and that there aren't any obvious gaps in your understanding of the problem, which is something much more common than what software developers are willing to admit.

The link to the usability test script above comes from Don't Make Me Think, a short and very useful introduction to the methods of Usability, and its sequel Rocket Surgery Made Easy (I have no relation with the author, I just think they're very good introductory texts). Although they deal primarily with web pages, the test methods they describe are valid for any kind of software development, and they answer your questions much better than I ever could.