Flow/New contributor survey

Liz made the excellent suggestion that we should survey new Wikimedia contributors to find out (in some detail, and with some breadth of feedback) what they need and what the use cases are.

Goal
The goal of this survey is to establish the motivations and needs of new users in engaging in discussions on Wikipedia. In particular, it focuses on:


 * 1) How they understand the purpose of talkpages;
 * 2) How good or bad they find the current system;
 * 3) What elements of the current system work well for them, and should be preserved, and;
 * 4) What elements of the current system work poorly, and should be replaced.

Target audience
The primary purpose of this survey is to gather the opinions of new contributors. Because of the deployment plan for Flow (namely, relying on small, opt-in deployments for a long period of time), there are plenty of opportunities for experienced contributors to give feedback on the designs and features, explain what they need talk pages to do, and correct our mistakes. Most of these interactions, however, will have to happen on, you guessed it, talk pages. That's fine for experienced users, but a hurdle for newcomers, many of whom haven't used talk pages before (or have had difficulties doing so).

New contributors (defined as users who have registered in the last month, made at least 5 edits during that period, and not been blocked) will be surveyed, split into two tranches - one for users who have contributed to a talk: namespace, and one for users who have not. The surveying software used will be the WMF SurveyMonkey account, with analysis performed by Oliver using R.

We have a few options to distribute the survey:
 * 1) We could just drop a link on their talkpage. The problem with this is twofold; first, it increases the chance of "junk" data from people other than the intended recipient filling out the survey. Second, it undermines the neutrality of the data provided by forcing everyone to, one way or another, experience discussion pages in a highly limited way.
 * 2) We can liaise with the Growth team to display a notification or message to users falling within that class for a certain time period. More of that delivery process would be automated (automated is good!) but it requires engineering time, either from Growth or Core.
 * 3) We could retrieve email addresses and mass-mail people through CiviCRM. This has been done before, and does not pose legal issues.

Page 1 - Fixed questions

 * 1) For Tranche 1 users If you wanted to start a discussion with other users on Wikipedia, do you know where you could do that?
 * Yes
 * No
 * 1) For all users Which of these types of discussions do you think are appropriate to have on a Wikipedia discussion page?
 * Discussing the content of articles; suggesting or requesting changes;
 * Discussing the contributions and behaviour of Wikipedia contributors;
 * Discussing the subject of articles, not just the content;
 * Asking for help;
 * Discussing things not related to Wikipedia.
 * 1) For Tranche 1 users, answering "yes" to Q1 Have you tried contributing to Wikipedia's talk pages?
 * 2) For Tranche 1 users, answering "yes" to Q3, or Tranche 2 users Here are a set of statements about the experience of using talk pages. Please indicate whether you agree or disagree with them. If you don't know the answer, feel free to leave the statement alone.
 * I find discussions easy to read.
 * I find discussions easy to contribute to.
 * I find it easy to discover when other people have replied to a conversation I am participating in

Page 2 - freeform questions

 * 1) For Tranche 1 users, answering "yes", or Tranche 2 users Do you think the purpose of Wikipedia's discussion pages differs from the purpose of comment systems on other websites? If so, how?
 * 2) ditto What parts of discussion pages work well
 * 3) ditto What parts of discussion pages work poorly
 * 4) ditto How could discussion pages be improved?

Page 3 - wrapping up
Thank you for completing this survey! If you are comfortable being contacted for follow-up interviews or questions, please provide your email address.

Introduction
The Wikimedia Foundation - the organisation that operates Wikipedia - is trying to evaluate the effectiveness of Wikipedia's discussion system. As part of this, we have prepared a short survey. Please answer as few or as many questions as you are comfortable with.

By answering these questions, you permit us to record your responses for internal use so that we can make Wikipedia better. We will not share your responses publicly, except in an aggregated (non-personal) manner.

Page 1 - short-form questions

 * 1) Which of these types of discussions do you think are appropriate to have on a Wikipedia discussion page?
 * Discussing the content of articles; suggesting or requesting changes;
 * Discussing the contributions and behaviour of Wikipedia contributors;
 * Discussing the subject of articles, not just the content;
 * Asking for help;
 * Discussing things not related to Wikipedia.
 * 1) If you wanted to start a discussion with other users on Wikipedia, do you know where you could do that?
 * Yes
 * No (note: for those answering "no", survey ends here)
 * 1) Have you tried contributing to Wikipedia's discussion pages?
 * Yes
 * No
 * 1) Here are a set of statements about the experience of using discussion pages. Please indicate whether you agree or disagree with them. If you don't know the answer, feel free to leave the statement alone.
 * I find discussions easy to read.
 * I find discussions easy to contribute to.
 * I find it easy to discover when other people have replied to a conversation I am participating in.

Page 2 - long-form questions

 * 1) Do you think the purpose of Wikipedia's discussion pages differs from the purpose of comment systems on other websites? If so, how?
 * 2) What parts of discussion pages work well?
 * 3) What parts of discussion pages work poorly?
 * 4) How could discussion pages be improved?
 * 5) Thanks for filling out this survey! Your responses are very helpful for us. If you would be interested in being contacted for followup interviews or questions, please leave your email address. We will not share this with third parties.