Structured Discussions

Flow is a project of the Core features team at the Wikimedia Foundation to build a modern discussion and collaboration system for all Wikimedia projects. Flow will eventually replace the current Wikipedia talk page system and will provide features that are present on most modern websites, but which are not possible to implement in wikitext. For example, Flow will enable automatic signing of posts, automatic threading, and per-thread notifications.

The main goals for the Flow project are:
 * to make the wiki discussion system more accessible for new users
 * to make the wiki discussion system more efficient for experienced users
 * to encourage meaningful conversations that support collaboration

Flow is currently deployed on several talk pages on Mediawiki.org and English Wikipedia. Flow will be released incrementally, as a limited beta. We want this product to change and grow over time, based on the experience and feedback of the people who use it.

Rationale
New users on English Wikipedia have become less and less likely to participate in on-wiki discussions, in spite of a growing and mostly automated body of messages directed at them. We also know that free-form wikitext talk pages present a significant barrier to new users  and even some experienced users.

In addition, experienced users often employ a suite of workarounds to help them manage and keep track of the many ongoing conversations that they're involved in, using tools that aren't necessarily well-suited to the task. For example, watchlists and diffs are page-level tools, making it hard to distinguish between the conversations that users are interested in and the ones that they're not. Experienced users can waste a lot of time checking diffs on an active talk page, especially if the most active conversation isn't the one that they're interested in following.

We believe that user expectations for a modern discussion system are increasingly diverging from the reality of talk pages today, and that all of our users deserve discussion and collaboration software that meets their needs.

Users expect a modern and intuitive discussion interface. Talk pages—as a discussion technology—are antiquated and are not intuitive.

Users are surprised by the cultural norms of the community. Many things about the culture that has grown up around talk pages (such as "talkback" templates or being able to edit other people's comments) are confusing. That is not to say those conventions are wrong, merely not what those users are prepared for.

We believe that a modern user-to-user discussion system will improve the projects. Better methods for collaboration will improve collaboration, which will improve all of the projects.

Roadmap
Please note:

The roadmap for a project like this is extremely fluid, and it will be changed, often. We're actually taking this project step-by-step, not designing and building everything at once. Each release should build off the previous one, adding and changing things as we go. Decisions are based on a lot of factors, including active work with community partners and stakeholders, feedback from community members, making dreadful mistakes and then fixing them, and just straight-up having a better idea today that we didn't think of yesterday.

This is not a hard and fast plan. Anybody who tells you that there's a hard and fast plan for a project like this is probably trying to sell you something. Building a feature like this is creative work, done collaboratively. Like any creative work, it runs on inspiration, mistakes and hope, more or less in equal measure. It's more of an art than a science, and it's not even that much of an art.

The roadmap items that appear below are guesses. They're listed here to give people an idea of the direction that we think we're going, and show the current thinking on relative priorities. This roadmap will be outdated sometimes. We're a very small team, with a very big list of things to build, and it's not possible to update the documentation every time we have a good idea or make a dumb mistake. We have good ideas and make dumb mistakes all the time, often simultaneously.

The general expectation for a big feature's roadmap like this is that predictions for the next month or two are usually pretty strong. The following quarter is usually in the right direction, but priorities will change based on what happens this quarter. Predictions for six months from now are almost certainly wrong in at least a couple of important respects. Everything after that is entirely fictional. The roadmap is not a promise, and it is not a plan. It's a living document that probably won't get updated often enough. This is a process of discovery, learning and growth. That's how interesting software gets built. -- DannyH (WMF), aka: Danny Horn, WMF Product Manager.

Q1 (July-Sept 2014)
Q1 theme – Never miss a message

User-facing features:
 * Subscriptions:
 * ✅ Subscribe to individual topics
 * ✅ Auto-subscription for topic user has created or participated in
 * ✅ Subscribing to Flow board = notifications for new topics on that board


 * Notifications:
 * ✅ Roll-up notifications for activity, one notification item per topic
 * ✅ Clicking on a notification takes you to topic page, with unread messages marked with color
 * ✅ Read/unread state for Flow notifications
 * ✅ Mark all as read


 * ✅ Log item improvements for Watchlist and Contributions (initial step done, but bugs in Contributions need to get fixed)
 * Hide feature redesign -- actually removing hidden threads from the page, still accessible through history and contributions
 * Mobile UI and design - board and threads are done, we'll do update to mobile Echo later

MediaWiki changes:
 * ✅ Create a namespace for topic pages
 * ✅ Notifications: dynamic read/unread states

Community partnerships:


 * Work with the Co-Op team on changes that we'd need for Flow to be useful to the Co-op.
 * Work with the French WP Forum des nouveaux, ditto

Q2 (Oct-Dec 2014)
User-facing features:
 * Table of contents
 * Search on a Flow board
 * Structural UI changes to enable editing other people's comments
 * LiquidThreads: Transition to wikitext or Flow (based on Andrew's script)
 * A more sophisticated version of subscribe to a board -- giving users more options on how notifications appear in Echo and in watchlists. ("Passive" vs "active" watching? Lots more work to be done here.)
 * Review of fixed width and indentation levels. (Just a review. Not a promise.) If we keep fixed width, we need a solution for tables and galleries.
 * Special:EnableFlow to have a more scaleable solution for creating new Flow boards
 * Turn summarize into a scratchpad area of full wikitext, connected to the conversation. Actual design TBD.
 * "Resolved" function -- mark a topic as resolved. Then we build a sort for resolved/unresolved.
 * Visual Editor integration with Flow input boxes

Community partnerships:
 * Continue to work with Co-op and Forum des nouveaux
 * Establish more partnerships with communities in other languages, TBA

Q3 (Jan-March 2015) and beyond
Note: Here's where things start to get very fuzzy -- updating this in Sept 2014, and there are a lot of ideas in play. Everything from this point on is still in concept work and will change drastically. Things will be added, things will be re-prioritized. See Danny's note above.

User-facing features:
 * More sophisticated options for different types of "close", add more workflows.
 * Option to write an edit summary
 * Move (rename) Flow boards
 * Tagging -- several possible options:
 * Surfacing threads on multiple boards
 * Splitting up board activity among several tags
 * Subscribing to a tag, rather than a board
 * Splitting threads -- turning tangents into separate pieces, with their own notifications
 * Media Uploader modal inegration with Flow input boxes
 * Support of MVP bots used on user talk pages
 * Newsletter support
 * Template & Category-driven workflows like block and notify
 * RfC-style structured conversations
 * Connect with Mobile Apps
 * Moving threads between user talk and article talk

Components of the discussion system
Here's a brief list of the main feature areas for a discussion system, a description of what the plan is for each one, notes about the status, and links to any ongoing discussions. It'll be regularly updated to factor in new discussions, and to note the permanency of the feature.

Is this replacing talk pages?
Yes, but not for a long time. Talk pages are used for many different reasons -- some simple, and some very complex. The team is working on the basic use cases first. Features are being added in an iterative way; we plan on rolling this feature out gradually as it meets the requirements for a particular use case.

At some point in the future, once the software is stable and a significant portion of active Wikimedians have had a chance to trial it and offer their feedback, we may mandate Flow on all discussion pages, but we have a lot of work ahead of us before that happens.

What pages will be Flow-enabled initially?
As of August 2014, Flow is enabled on Mediawiki.org, on several Beta Features talk pages, including Talk:Flow, Talk:Winter, Talk:Beta Features/Hovercards and Talk:Content translation. On EN.Wikipedia, it's live on WikiProject Breakfast and WikiProject Hampshire.

In late 2014, we will increase the number of places where Flow is deployed; these deployments will occur with consensus from community members who use those discussion spaces.

Can I opt out?
In terms of viewing, you won't be able to opt out of seeing Flow. If other users have converted their talk pages to Flow pages, or reached consensus that a Wikipedia or article talk page should be converted, there's not going to be a way to avoid using Flow if you want to contribute to that page.

We don't have a detailed plan yet for how the Flow rollout will happen; a lot depends on the next couple test deployments in fall 2014. It's likely that we'll offer Flow as an opt-in feature on user talk pages, which may graduate to the default experience, enabling Flow on new talk pages, begin converting existing talk pages to Flow, etc. If you're concerned about how Flow will look and behave, your best bet is to trial the software early, so you can give us feedback on what needs changing well before we begin discussion on whether or not it should become the default on all discussion pages.

What will happen to current talk page discussions?
When Flow is enabled on an active wiki talk page, the existing dicussions will be moved to an archive page. There will be a clear link from the Flow board to the archive page. Old discussions will not be "thrown away"; we know how important discussions are to understanding the collaborative history of a wiki page or project.

We won't be able to "convert" wiki talk pages into the Flow format, because there isn't a clear one-to-one correspondence between a wiki talk page edit to a Flow post. It's more respectful to the discussions if we preserve the original format, on an archive page.

What about IP/anonymous editors?
Anonymous users will be able to use Flow-enabled talk pages as any other user. However, subscriptions to individual topics, and notifications based on those subscriptions, will only be available for logged-in users.

Why don't we use a pre-built system, like PHPBB or Discourse?
The simple answer is that "they won't cut it". The complex answer is that pre-built systems are designed around singular use cases, and Wikimedia projects do not have "singular" use cases. On the English Wikipedia alone, the following are discussions that take totally different forms: The list goes on and on. A singular package will not and cannot cover these cases. So we have to build our own.
 * User talk
 * Article talk
 * Requests for Comment
 * Request for Adminship
 * Deletion discussions
 * Merge discussions
 * Village pump discussions
 * AN/I
 * Arbcom cases

Why not use LiquidThreads?
LiquidThreads (LQT) was an early attempt at creating a structured discussion system. It has both strengths and weaknesses. After exploration and discussion, LQT was abandoned as a solution going forward for several reasons: poor performance, due to the way individual comments or posts are stored, parsed, rendered, cached, and assembled; no support for globally unique identifiers; and lack of flexibility with regards to workflows and collaboration techniques beyond simple discussion.

We'll be working on a conversion script that turns LQT discussions into Flow topics, in late 2014.

How will Flow handle spam and vandalism?
In the current release (Aug 2014), each topic and individual post come with a set of moderation features. These are:
 * 1) Hide (equivalent to reversion/rollback);
 * 2) Delete (equivalent to revision-deletion), and
 * 3) Suppress (equivalent to oversight or suppression)

Handling spam and vandalism in Flow should actually be easier than dealing with it on existing talkpages. Because Flow topics and comments exist as discrete elements, a specific post can be removed with a single set of actions: there's no need to go through the page history removing intervening comments, for example. In terms of spam, Flow is integrated with the AbuseFilter and the global and local Spam Blacklist.

Will Flow support wikitext or VisualEditor?
Both. Flow is linked into Parsoid, which is the software that VisualEditor uses to read and write wikimarkup. Hooked into that is a wikitext editor and, in later releases, VisualEditor. Users will be able to pick which they want to use.

Will Flow support all the templates and other markup we need in discussions?
Yes. Whatever markup Parsoid supports, Flow can support. There may be restrictions on certain magic words or complex templates that affect performance, but, generally speaking, anything users commonly need to use (mathematical symbols, references, templates) can be added to a Flow post. You can see and test templates and other markup in the prototype, eg this thread has some examples.

What happens to my custom signature?
Flow will not directly support custom signatures, for a couple of reasons.

The first is that they're disruptive from a UI point of view. Custom signatures are great for letting people know who said what; they allow for a specific user to be easily, visually distinguishable in discussions. The problem is that the way this comes about is by allowing refactoring of where links live, how they're displayed, and so on. As a result, there's a complete lack of consistency, which hurts the ability of users to navigate easily. One user might have their talkpage link in one place – another in a different place, and with a signature that doesn't actually reveal their username. The second is technical; allowing people to add raw HTML formatting into Flow boards could cause serious issues and errors in how the page is displayed.

Having said that, we appreciate the advantages of custom signatures; it allows for some distinguishing elements, it allows for forms of identification that extend beyond username. We're going to be working a "preferred name" field into the interface to allow for the latter; while it won't be as malleable as the status quo, it will allow for some originality. Your own posts will be visually highlighted to be more easily findable.

Will we be able to edit other people's posts?
In the current release, no. Only administrators can modify the posts of others (anyone can edit topic titles, however). As with all design decisions at the early stage of this project, this is open to being revisited if it negatively impacts users.

Users being allowed to edit the posts of others is atypical and unexpected: nearly every discussion medium on the modern Internet does not allow this, because it breaks the expectation that the comment you are replying to is genuine, and that your comment will not itself be tweaked or corrupted by others. The need to meet this expectation should be balanced with the fact that the community has developed various workflows in discussions and in the talk namespaces that are dependent on that kind of editing and interleaving.

How will you support all of the processes Wikipedia uses?
It's a complex problem, which is why we're starting with the most basic process – unstructured user-to-user discussion – first. Before we proceed to enable Flow in different namespaces, we will identify the critical workflows that need to be supported with software and build lightweight support for those processes: for example, a simple block/unblock workflow on new user talk pages. In the first 1-2 years of this project, we do not expect to cover 100% of the workflows across all our projects. Because each Wikipedia (and other Wikimedia sites) has developed its own workflows, based on the local social conventions, templates, categories, and bots, there is no easy way to discover all of these processes or build something based on feedback from one Wikimedia community and abstract it to other projects.

Longer term, once we've completed the discussion side of Flow, we may build a workflow engine – a structured language that editors can use to describe a workflow and incorporate it into Flow, for a specific action or a specific page.

Why is the design using a fixed-width? What about my widescreen monitor?
This gets into what is called a measure. The human eye is able to best read large volumes of text when the lines are capped at a certain length. This reduces eye fatigue. Fixed-width is useful for the following reasons:
 * 1) It makes it easy for the eye to jump to the next line.
 * 2) It provides resting spots for eye away from big blocks of text.
 * 3) Usability tests demonstrate that this reduces stress level.
 * 4) Best practice is 13 words per line, 50% of total screen width.

We'll keep iterating to make sure Flow falls into the optimal readability range for different size screens.

Why is there so much whitespace/padding?
Whitespace is used for your eyes' resting spots. It helps you focus on content and increases content comprehension by 20%, which is important in discussing complicated issues. It decreases user unsatisfaction, which could result in an unhealthy discussion and harassment. We know what kind of harm to productive conversations misunderstandings and misinterpretation can do.

In addition to the above, white space helps structure a wall-sized amount of text, communicates the relationship between content, and improves scannability and readability. "The eye cannot focus on excessively close lines and … the reader expends energy in the wrong place and tires more easily. The same also holds true for lines that are too widely spaced ... Where reading is smooth and easy, the meaning content of the words is grasped more clearly[.]" The optimal text leading is x1.5 of the text size. Text size we will be using is 16pt at 25pt leading

See also:


 * Whitespace myth
 * Best practices analysis

That being said -- there is a wide range in users' preference for information density. Core MediaWiki is very information-dense, with a completely fluid width and tight line spacing -- and, possibly not coincidentally, experienced MediaWiki users tend to prefer greater information density. (This may be the equivalent of natural selection.) One of the goals for Flow is to make the discussion system more accessible for new users, and part of that may include a less info-dense design. We're still figuring out the right balance, and this may include individual preferences for display density, with Gmail's recent redesign as an example.

Why is there limited indenting/threading of comments?
Mobile readership is growing dramatically (currently, representing almost 1/4th of total pageviews to Wikipedia), so we need to build for a future where we have users who are purely mobile-only. Multiple indentations will have problems displaying on a small screen.

There are also many general usability arguments out there for why discussion systems that use indentation (also called nesting, branching, or threading) to indicate the relationship between comments are problematic. As Joel Spolsky writes: "Branching is very logical to a programmer's mind but it doesn't correspond to the way conversations take place in the real world. Branched discussions are disjointed to follow and distracting. (...) Branching makes discussions get off track, and reading a thread that is branched is discombobulating and unnatural. Better to force people to start a new topic if they want to get off topic."

We want to structure conversations in a way that makes sense and fosters a healthy discussion. The framework we're proposing places more emphasis on discussing the topic and less on tangents. (Users can still engage in tangents with one another, but the primary action that is emphasized is responding to the topic.) To complement this system, we are planning to add the ability to easily split tangents off into new topics, since it's likely that long tangents are, in fact, different discussions altogether. What we want to de-emphasize (without entirely disabling) are long back-and-forth tangents between two users that have no relation to the main topic. This is usually the hallmark of an unhealthy conversation or just one that needs to be moved off to a new discussion.

Why are we using big colored buttons?
The buttons in Flow are part of Agora, a new standardized UI library for Wikimedia projects. We use colored buttons to emphasize what to click next in order to complete a particular workflow. The blue-colored button signifies a progressive action that requires more steps ahead, while a green-colored button signifies the final progressive action. Links are used in areas where the action is not necessarily encouraged or discouraged (e.g., replying to any specific post). See Wikimedia Foundation Design/Agora Control Library for more information.

Are you working on a design without JavaScript, for users with accessibility or browser concerns?
Yes, we aim to make most of the core Flow functionality work without JavaScript, on older browsers, etc. The software will work, but it will not work optimally. Also, given the nature of the way Flow is designed to behave, disabling JavaScript "for performance reasons" may actually create poorer performance than with JavaScript. For example, Flow will make use of a technology called lazy loading that will reduce immediate request overhead for browsers that have JavaScript enabled. This will have the effect of causing the pages to appear to load faster than the non-JavaScript version, which will require all elements to load before achieving full functionality. Replying to a post with JavaScript enabled will be done in-line and quickly, in the background, while replying without JavaScript will require loading a new page, submitting the reply, and then reloading the discussion.

Where did the Table of Contents go?
Since Flow Boards are, for lack of a better word, "infinite", we're exploring different ways for users to navigate the discussions on a board. There is an icon at the top of a Flow board that collapses and shrinks the topics currently loaded down to "topic bars", and you can click a topic bar to collapse/expand it.

We're also planning to build a Table of Contents (probably fall 2014) that will help users navigate quickly between discussions.

Why is there gray text?
Pure black text is harsher for the eye to read on a white background, due to strong color contrast. Text will start to flicker, users will experience eye fatigue and strain, and they will be tempted to stop reading. If my eyes gives me a limited amount of time to read and try to understand a discussion thread, especially a complicated one, I'm more likely to get frustrated and give up. That's not a good user experience. The benchmark of text color on white backgrounds to avoid these issues is #333.

Why does it look like Facebook/Twitter/StackExchange, etc.?
The Wikipedia community has very different values from many of the other popular discussion websites. But the important thing to remember is that those sites have done lots of elaborate user research and A/B testing to find what works best for facilitating discussions, reducing flame-wars, and creating inviting and easy-to-use interactions. We don't want to reinvent the wheel when it comes to the basics of online discussion, and we don't want to create something that breaks most Internet users' expectations of what a discussion system looks and behaves like. Flow should be intuitive enough for anyone to use without having to read a manual, and that means following the best practices of discussion interface design.

However, we are aware that the discussion system on Wikipedia is more than just a tool for communication, but also a medium for collaboration and consensus-based decision-making. We are going to keep iterating and responding to your feedback on how to better serve those usages.

In addition to the sites you may be familiar with, here are some other sites we look to for inspiration/guidance that have structured discussion systems:


 * YouTube
 * Flickr
 * PhpBB
 * Simple Machines Forum
 * 1) Gawker
 * 2) Discourse
 * 3) Gizmodo
 * 4) StackOverflow
 * 5) Medium
 * 6) Quora
 * Tumblr
 * 1) The Verge
 * 2) Reddit
 * Livejournal

If there are patterns from these sites that you think might be interesting to explore on Wikipedia – or if there are patterns you feel do not belong in a Wikipedia discussion! – please let us know.

Why are we using this Topic and Post order?
The default order in the current release is: new topics appear at the top of the page, and new posts within those topics appear at the bottom, below older posts. There is also a sorting feature that allows users to see the most recently active topics at the top.

Infinite scroll, and the elimination of the need to archive - this aspect necessitates top-posting.
 * Topic order:

More intuitive for newcomers and more efficient for seeing new/updated content. New/updated topics appearing at the top follows standard set by many forums, social media, and email.

Replies to topics appearing oldest to newest (top to bottom) make sense as a way of encouraging users to read through the existing conversation before jumping in to comment.
 * Post order:

Development principles
With any complex software project, it's almost impossible for the designers, developers and product people to know everything. This is a particular problem in an environment like that of the Wikimedia projects - due to the sandbox-like nature of the place, there are tens of thousands of different workflows, different user needs, and different problems. Even if we could accurately identify all of them, we can't necessarily tell if our solution is the right one until we put it in front of users.

Accordingly, we're keeping two things in mind while we're developing the Flow software. First: we are partners with the community on this. Editors are welcome and encouraged to participate in the development process, pointing out things we've missed, identifying and describing new workflows, and helping keep us honest - when something hits this level of complexity, it's impossible to make things work without as many people helping as possible. Before and after we build things, we'll open a conversation about the feature. Second: a lot of the work we're going to do, at least initially, is experimental: we don't know if it's the right implementation of a feature. If it's not, we'll be happy to roll it back.

Who is working on Flow?
The Flow team focuses on delivering a delightful experience for user-to-user interactions.


 * Erik Bernhardson (ebernhardson)- tech lead
 * Matthias Mullie (mlitn) - engineer
 * Shahyar Ghobadpour (shahyar) - engineer
 * S Page (spagewmf) - scrum master
 * Andrew Garrett (werdna) - engineer
 * Quiddity (quiddity)- Community Liaison
 * Danny Horn (dhorn)- Product Manager
 * May Galloway (violetto) - Designer
 * Brandon Harris (jorm) - UX designer
 * Matthew Flaschen (superm401) - engineer

Known issues
On WMF wikis, Flow stores the parsed HTML of the header area and posts. When you view Flow content you see this stored HTML; when you edit the header area or a post you see the original wikitext and it is parsed again on save. This means that changes to a template will not appear in the Flow headers and posts using it until they are edited. (Flow posts are snapshots of a moment in time, so allowing templates within posts to change days, weeks, or years later, without any outward indication of the change, is not something we want to support.) -- NOTE: I don't know if this is actually true, or smart to say, or anything. Depending on user needs, we may change the way the header area works, so it can accept transclusion or can automatically update for commonly used project and article banners.

Contact and links

 * Flow development team roles and responsibilities
 * IRC channel:
 * Test instances and locations: ee-flow, flow-tests, beta.wmflabs
 * Trello (product management): current work, planned work (backlog)
 * Bugzilla: open bugs, all bugs, submit new bug