Flow/Community process workflow interviews (June 2015)

Introduction
Flow is an alternative to the current talk page format under development by the Wikimedia Foundation, which aims to provide a cleaner talk page experience/interface for readers and editors on Wikipedia and Wikimedia sites. The Collaboration team is beginning to design a Workflows feature -- a key feature for the more experienced Wikipedia contributors. We need to check in with a group of experienced editors early in the design process, to make sure we’re going in the right direction.

Workflow: A formal, multi-step community process that involves discussion, and ends with a decision.

Goals of this research
The goal of this round of research, to be completed in June 2015, is to hold feedback sessions with prospective users of Flow’s new Workflows feature. The users are very active contributors on English Wikipedia (more than 1,000 edits?), with a strong preference for editors who use workflows as part of their daily wiki work. (Examples of workflows include: Articles for Deletion, Administrator’s noticeboard, Good Article review, Requested edit.)

The purpose of this research is twofold: (a) to learn how these users use workflows, what problems they experience, and what would help them in their work; (b) to find out whether our current ideas for a Workflows feature would help them reach their goals.

Specific goals

 * Understand the mental model of Wikipedia-related workflows (how do they normally participate).
 * Road-test the underlying logic with a knowledgeable user:
 * Is this definition of workflows easy to understand?
 * Is it able to capture the kind of workflows you know or can imagine on Wikipedia?
 * Is the way to execute workflows easy to understand, and would this be a more efficient process than the current system?


 * Identify the gaps and mistakes in our thinking.
 * Get new perspectives and suggestions.

Methods
Conduct interviews with Wiki[m|p]edians who participate regularly in one or more community processes. The goal is to get as much "coverage" as possible with a relatively small set of participants (5-10), so sample for diversity.

Learn about the basic mechanics of the process, its purpose and relative importance, the context in which in occurs, who participates, etc. After the participant has described the process thoroughly, introduce them to some early-stage prototypes and wireframes for Flow workflows: show how Flow might be configured by editors to support a specific workflow (for example, AfD), and how the subsequent discussion could work.

Interview protocol
This is a semi-structured interviewsemi-structured interview protocol. Not all questions will be asked in every interview, and the investigator may ask additional probe questions if interesting things come up.

When possible, the investigator will ask the participant to share links to pages that they talk about or diffs that point to particular actions or events which are referred to over the course of the interview.

Part 1

 * First, I'd like to get a little information about who you are and what you do on Wikipedia
 * What kind of work do you do on Wikipedia?
 * Now I'd like to learn about the community processes you participate in most frequently
 * which community processes have you participated in most during the past year?
 * how do these processes relate to other work you do?


 * Now, let's talk about a process that you've been heavily involved with recently.
 * tell me a little bit about the process and its goals
 * how many people participate?
 * In a normal week, how many of these discussions are in progress?
 * where does discussion take place?
 * how would someone who had never participated in this process learn that it exists? How would they learn how to participate?
 * Tell me a little bit about how it is organized
 * under what circumstances is this process initiated? who initiates it?
 * how long does it take before resolution?
 * do people have specific roles in this process? tell me a little bit about how people in different roles participate in the process.
 * Now I'd like to learn a little more about how you participate in this process. Thinking about the 'work' you do...
 * what role do you generally play in the process?
 * How do you decide whether to participate in a new discussion?
 * How do you keep up to date with what's going on?
 * Now I'd like to hear about what you think about the process in general.
 * Tell me about a recent fun or satisfying experience you had while participating in this process
 * Tell me about a situation where you were frustrated while working participating in this process
 * How successful is this process? How do you measure the success of a particular discussion? Of the process as a whole?
 * How important is this process to Wikipedia? Why?
 * Are there specialized tools, like userscripts, gadgets, or templates, that are used in this process?
 * Is there any tool that you use that you wish worked differently or better? How would it help you?
 * Is there any tool that your project doesn’t have that you wish it did? How would it help the project?
 * Are there improvements that would be especially useful for newcomers? For experienced editors?

Part 2
Walk through the Conversation Pattern prototype and ask participants how it might work for the processes that they participate in.

Basically, from here, we want to have a conversation with the user. We’ll have several concepts and mockups, and possibly a clickable prototype.

We want to show each artifact that we have, and at different points, ask these questions:


 * Does this piece make sense? Can you imagine how you might use it?
 * Is there anything that jumps out at you that looks exciting?
 * Is there anything that just feels wrong, and you don’t think it’ll work?
 * Can you think of a workflow (or a type of workflow) that wouldn’t work with this model?
 * Do you have suggestions?

The Flow team could make (for example) the Articles for deletion workflow in software. There’s a place to nominate the article, and it automatically fills out the user and the article name, prompts for a reason, puts a template on the article and then lists it on the AfD list. That would be relatively easy to build. But -- there are a thousand different workflows, some of them change over time, and there are different versions on different languages. So what we have to build is the set of tools that helps experienced editors to create their own workflows, for other people to use.
 * How to explain the prototypes

Prototypes
(developed before research began)
 * Conceptual prototype (AfD)
 * interactive prototype
 * demo video

(developed during phase 1)
 * Workflow configuration mock-ups
 * Generic workflow builder
 * Generic workflow builder (alternate)
 * Workflow builder - FAC
 * Workflow builder - FAC (alternate)
 * Workflow builder - RfC
 * Workflow builder - AfD
 * Workflow builder - AfD (alternate)

(developed during phase 1)
 * Workflow discussion mock-ups
 * File:Workflow_Pattern_in_action.png
 * File:AfD_in_action.jpg
 * File:RfC_in_action.jpg

Findings (Phase I)



 * workflows are idiosyncratic, but many follow the same general pattern of Proposal -> Discussion -> Resolution
 * editors have developed a panoply of ingenious work-arounds and supporting tools to streamline the process: bots, userscripts, specialized templates, and gadgets
 * initiating a new discussion is often a technically complicated endeavor, although instructions are generally available
 * notification (advertising the discussion to relevant individuals, groups, and on related pages) is important to most processes, but is a highly manual job
 * many community processes are under-staffed and backlogged
 * decision-making (necessary to close/resolve a discussion) is complex and often time-consuming

Proposal phase
Notification is often ad hoc and labor-intensive

Proposing a new instance (starting a discussion) is often complicated and poorly documented

Discussion phase
Discussions often languish

Tracking the progress of a discussion is challenging

Resolution phase
Coming to a decision, and justifying that decision, is time-consuming

supporting material (relevant links to policies, diffs, precedents) is scattered throughout the thread

Records of discussions can be valuable resources, but are hard to find

Closers are often a scarce resource

Theme and variations
Differences between processes, wikis

Use of supporting tools Scale varies widely
 * DRN proposal workflow
 * Commons QuickDelete gadget
 * AfD bot
 * AfC Helper script

Location of the discussion varies widely

Design
meso-structure of discussions

rules and roles

location, location, location

Deployment
Pilot tests - start with smaller, simpl(er), less essential process. Get support of essential process participants. Ask them to experiment with configuring and using the CP in a sandbox wiki, provide clear mechanisms for feedback Arrange a time-limite pilot with clear feedback mechanisms and the ability to restore the legacy workflow if desired, at the pilot's conclusion. Survey/interview all participants (core and perpheral) at the conclusion