Talk pages project/New topic

This page talks about the editing team's work to improve contributors' workflows for starting new discussion threads on talk pages, across Wikipedia's 16 talk namespaces.

This new workflow for starting new conversations is intended to make it more intuitive for Junior Contributors to initiate conversations in ways other contributors can easily engage and help Senior Contributors do the same with less effort.

This initiative sits within the Talk pages project, our team's larger effort to help contributors, across experience levels, communicate more easily on Wikipedia using talk pages. To accomplish this, we are building upon the Talk pages consultation 2019, and existing community conventions, to evolve existing wikitext talk pages.

To participate in and follow this project's development, we recommend adding this page to your watchlist. We will use this page to:


 * Share and invite feedback on designs
 * Announce deployment plans
 * Share data about how the feature is being used

Status updates
This section contains updates about the project's development.

10 June 2022
Scaling

On 7 June, the New Topic Tool became available by default on desktop to everyone at Wiktionary, Wikinews, Wikibooks, Wikiquote, Wikisource, Wikispecies, Wikiversity, and Wikivoyage. You can see the deployment status of all Talk Pages Project features by visiting the Deployment Status page.

New Setting

This week, we introduced a new setting within Special:Preferences that enables you to decide for yourself what you see when you click on a red-linked talk page (or otherwise arrive at a talk page that has not yet been created). More details in T297990.

11 May 2022
Scaling

Today, the New Topic Tool became available by default on desktop to everyone at the 20 Wikipedias that participated in the recent A/B test. You can see the deployment status of all Talk Pages Project features by visiting the Deployment Status page.

6 May 2022
Additional A/B Test Analysis

We noted in the A/B test results we shared on 21 April that, "logged-in Senior Contributors (defined as people who have made ≥100 edits) are slightly more successful publishing topics using the existing section=new workflow than they are using the New Topic Tool."

Curious about why this might be so, we did an additional analysis to investigate the impact of experience level on Senior Contributors' New Topic Tool usage. In this additional analysis, we learned that everyone who has made ≤ 2,500 edits are more successful using the New Topic Tool.

This is new data is leading the Editing Team to have more confidence in the conclusion that most people are more successful publishing new topic with the New Topic Tool and any regressions in the data are signals of Senior Contributors preferring the existing workflow rather than of deficiencies in the New Topic Tool.

Opt-Out Deployment

As a result of the findings above, we plan for the New Topic Tool to become available by default on desktop to everyone at the 20 Wikipedias that participated in the A/B test.

21 April 2022
The Editing Team recently analyzed data from an A/B test of the New Topic Tool that ran at 20 Wikipedias from 27 January 2022 through 25 March 2022.

What follows are the conclusions we are drawing from this analysis and the next steps the team is taking as a result. Conclusions The full A/B test reports can be found here: Logged-In New Topic AB Test Report + Logged-Out New Topic AB Test Report.
 * The New Topic Tool increases the likelihood that logged-in Junior Contributors will successfully publish a new topic.
 * 44.2% of Junior Contributors that opened the New Topic Tool were able to successfully publish at least one new topic during the A/B test compared to 37.2% of Junior Contributors using the existing section=new workflow.
 * Junior Contributors using the New Topic Tool are 1.3 times more likely to publish a new topic they start than Junior Contributorsusing the existing section=new workflow.
 * The New Topic Tool did not lead to any statistically significant changes in revert rates for logged-in Junior Contributors, logged-in Senior Contributors, or Logged-out users.
 * Logged-in Senior Contributors are slightly more successful in publishing topics using the existing section=new workflow than they are using the New Topic Tool.
 * 66% of Senior Contributors that opened the New Topic Tool were able to successfully publish at least one new topic during the A/B test compared to 71% of Senior Contributors using the existing section=new workflow.
 * The New Topic Tool led to a slight increase in the new topic completion rate for logged-out users, however, there is not sufficient evidence to confirm this observed change was due to the New Topic Tool.

Next steps

The Editing will propose that the New Topic Tool be enabled by default for all volunteers (logged in and out) across experience levels at the 20 Wikipedias that participated in the A/B test.

In parallel, the Editing Team will do a follow-up investigation to help determine what could be contributing to Senior Contributors being slightly more successful in publishing topics using the existing section=new workflow. This investigation will happen in T306481 and T306579. We will post the findings here once they are finalized.

15 April 2022
Naming

Going forward, we will be referring to the "New Discussion Tool" as the "New Topic Tool." We have made this decision in order to:


 * 1) Reduce the likelihood that people confuse the DiscussionTools extension with this tool
 * 2) Better relate this tool with the other Talk Pages Project features: Topic Subscriptions and Topic Containers

A/B Test

On 25 March 2022, the team finished running the A/B test of the New Topic Tool. In early May (or slightly before), you can expect the results from this A/B test to be published on this page.

27 January 2022
A/B Test

Today, 27 January 2022, an A/B test of the New Discussion Tool began at 20 Wikipedias: Amharic,  Bengali,  Chinese,  Dutch,  Egyptian,  French,  Hebrew,  Hindi,  Indonesia,  Italian,  Japanese,  Korean,  Oromo,  Persian,  Polish,  Portuguese,  Spanish,  Thai,  Ukrainian, and  Vietnamese.

20 December 2021
Prototype: New Discussion Tool Hint

A working prototype of the dismissible notice that will appear within the New Discussion Tool is ready for you all to try.

We would value you trying the prototype and sharing what you think about it on the talk page.

24 November 2021
Design Concept: New Discussion Tool Hint The team is working on the design for a dismissible notice that will appear within the New Discussion Tool.

This notice will offer people the ability to switch back to the legacy  experience.

The in-progress design is pictured here. If there are ways you think this design could be improved to make it more clear to people that they have the ability to switch to the legacy  experience, we would value you sharing these ideas on the talk page.

15 October 2021
New Functionality: Preserving text formatting

Starting next week, you will gain the ability to preserve formatting when pasting text into the New Discussion Tool's  mode.

24 September 2021
A/B Test

On 14 September 2021, the New Discussion Tool became available as an opt-out feature at the first projects: ar.wiki and cs.wiki.

Before offering the New Discussion Tool as an opt-out feature at more projects, we will be running an A/B test. The A/B test will help us evaluate the impact the New Discussion Tool has on the likelihood that Junior Contributors successfully publish the new discussions they start writing.

We are now planning how the A/B test will be implemented. In the coming weeks, we will consult with wikis about participating in the test. You follow this work in Phabricator.

14 September 2021
First opt-out deployment

Today, 14 September 2021, the New Discussion Tool was made available as an on-by-default feature to all volunteers, logged in and out, at the Arabic and Czech Wikipedias. Today's deployment marks the first time the New Discussion Tool is being made available to all volunteers at a project, by default.

25 August 2021
Metrics: Initial adoption

On 17 August, the team completed an initial analysis of how people have been using the New Discussion Tool while it has been available as a beta feature.

The objectives for this analysis was to answer the following questions;


 * 1) Are people finding the tool to be disruptive?
 * 2) Are people finding the tool behaves in the ways they expect?
 * 3) Are people depending on the New Discussion Tool start new conversations?

In summary, this analysis is leading us to conclude:


 * 1) People are NOT finding the New Discussion Tool disruptive
 * 2) People are finding the New Discussion Tool behaves in the ways they expect
 * 3) A segment of people are using the New Discussion Tool to start the majority of new discussions they initiate

With all of the above in mind, the Editing Team is going to move forward with offering the New Discussion Tool as an on-by-default feature at an initial groups of Wikipedias (Arabic and Czech) in the next few weeks.

Review the findings from this analysis or view the full report.

6 August 2021
New toolbar in  mode

Yesterday, 5 August, the  mode toolbar that was introduced as an opt-in setting in May, became available to all people who have the New Discussion Tool enabled. Note: you can still turn this toolbar off if you would like within Special:Preferences.

Empty talk pages

Next week, the first iteration of the empty state experience we shared on 2 August will be visible to everyone who has the New Discussion Tool enabled as a beta feature.

2 August 2021
New functionality New Discussion Tool as an opt-out setting
 * Empty talk pages: on many wikis, the majority of talk pages have not yet been created.[1] To make the experience of creating these "empty" talk pages more intuitive for Junior Contributors, we are working on some improvements (pictured here). Note: this change only impacts people who have the New Discussion Tool enabled.
 * Edit notices: support for edit notices was added to the New Discussion Tool. For now, we have not made any changes to how said edit notices will appear (read: they will continue to appear atop the New Discussion Tool as they do atop the existing experience and be styled in the same way).

Later this month (August), we plan to offer the New Discussion Tool as an opt-out setting at the first two projects: Arabic Wikipedia and Czech Wikipedia.

Before moving forward with this deployment, we will first complete an analysis of how people have been using the tool while it's been available as a beta feature. The purpose of this analysis is to:


 * 1) Learn whether the tool is disrupting volunteers (e.g. causing more reverts) and
 * 2) Learn whether people are finding the tool behaves in the ways they expect.

The newly-created Extension:DiscussionTools/NewTopicMethods page is a great resource for seeing how the New Discussion Tool will interact with the variety of methods there are for starting new talk page sections.

13 May 2021
New toolbar in  mode

As of Tuesday, 11 May, you can enable a toolbar within the New Discussion Tool's   mode. This toolbar introduces easier ways for pinging other people and inserting links into the comments you are drafting.

To start, this new mode is being offered as an opt-in preference. You can enable it by taking the following steps:


 * 1) Ensure you have the New Discussion Tool enabled
 * 2) Visit
 * 3) Locate the   section
 * 4) "Check" the checkbox next to the setting that reads:
 * 5) Click
 * 6) ✅ You are done. You should now see the new toolbar whenever you open the New Discussion Tool's   mode

23 April 2021
New tools in  mode

In the coming weeks, you will be able to use a new   mode in the New Discussion Tool. This "new   mode" introduces easier ways for pinging other people and inserting links into the topics you are drafting.

To start, this new mode will be offered as an opt-in preference. You can experiment with the prototype version of this new functionality on a test wiki here and review the deployment plan in Phabricator here: T276607.

16 March 2021
Today, 16-March, the New Discussion Tool became available as an opt-in beta feature at the English and Russian Wikipedias and all Wikimedia Sister Projects.

Note: you can review how and where all Talk pages project features are available here: Talk pages project.

11 March 2021
Offering the New Discussion Tool as an opt-in beta feature at more Wikipedias.

Yesterday, 10-March, the New Discussion Tool became available as a beta feature to logged in users at all Wikipedias except the English, German, and Russian Wikipedias. We made the decision to offer the tool at more projects after verifying with people who have been using the tool at the Arabic, Czech, and Hungarian Wikipedias that it is working as expected.

26 February 2021
1 week after initial beta feature deployment.

The New Discussion Tool has been available at the Arabic, Czech and Hungarian Wikipedias for one week. So far, people have had positive things to say about the tool. Also, most people who have tried the tool at these wikis have used it more than once.

Over the next couple of weeks, the team will continue monitoring how people are using the tool, the experiences they are having with it and determine when and how the tool can be offered as a Beta Feature at more projects. In the meantime, if you are curious to see how people are using the tool, you can view this query:


 * New Discussion Tool usage at the Arabic, Czech and Hungarian Wikipedias: https://quarry.wmflabs.org/query/52630

18 February 2021
Availability at Arabic, Czech and Hungarian Wikipedias.

Today, 18 February, the New Discussion Tool became available as a beta feature at the Arabic, Czech and Hungarian Wikipedias. Once we are confident the tool is working as expected, we will begin the process of offering the New Discussion Tool as a beta feature at more projects.

In the meantime, volunteers at all projects can try the New Discussion Tool on any talk page by appending  to the page's URL. For example: .

22 January 2021
Trying the New Discussion Tool in production

As of this week, you can try the New Discussion Tool on any talk page URL by appending ?dtenable=1 to it. For example: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(WMF)?dtenable=1.

Usability testing + first deployment

Junior and Senior Contributors recently completed testing the New Discussion Tool prototypes. Both groups of people found the New Discussion Tool to be intuitive and to work in ways they would expect. These test findings have encouraged the team to move forward with plans to offer the tool as a beta feature at the Arabic, Czech and Hungarian Wikipedias in the coming weeks.

You can read more about these usability tests in the Usability testing section below.

17 December 2020
Version 1.0 prototype

The prototype of version 1.0 of the New Discussion Tool is ready and we would value you trying it out and sharing what you think could be improved about how it functions. You can try the prototype by clicking the link below on a desktop computer: https://patchdemo.wmflabs.org/wikis/3e14959a196db0f7b0c32a35c99dc0fc/w/index.php/Project:Teahouse. (You do not need an account to maintain privacy. The IP address that is posted for all comments (172.16.0.164) belongs to the server.)

You can read the instructions for sharing feedback by visiting the talk page: Topic:Vzswfe2hn2zuuwtc.

19 November 2020
Version 1.0 design feedback

Over the past two weeks, people shared ideas for how version 1.0 of the New Discussion Tool designs could be improved to ensure Junior Contributors can confidently and correctly start new conversations.

We understood the feedback people had to share about version 1.0 of the New Discussion Tool design as falling into two broad categories:


 * 1) The discoverability of the affordance for adding a new topic
 * 2) The functionality offered within the "drafting" part of tool

You can view a full summary of the feedback people shared and the plans we have for incorporating this feedback on the talk page here: Topic:Vy2rcl8fax0gyuxo.

28 October 2020
Initial designs

The team has a design proposal ready for how the first iteration of the New Discussion Tool could look and function. We would value you hearing what you think about them.

You can review the designs in the #Design section below and share the thoughts and questions they bring to mind on the talk page here: Topic:Vwpwr84naer42ovi.

Sharing feedback by Wednesday, 11-November-2020 will help us ensure what you have to say is considered as part of the prototype we will be building in the coming weeks.

2 October 2020
Design is underway

The team is working on a few different design directions the New Discussion Tool could take to meet the requirements listed below (see: New discussion#Approach). These design directions are intended to help us make the following implementation decisions:


 * Initial touchpoint: Where and what do people click to start the process of adding a new discussion topic to a talk page?
 * Composition: Where do people draft the title and body of the topic they are planning to publish?
 * Posting/Feedback: What is the best way to communicate to people that the topic they intended others to be able to see and interact with has been posted successfully?

In the next few weeks, you can expect mockups of these "different design directions" to be posted to this page along with a message on the talk page (Talk:Talk pages project/New discussion) inviting to share what you think about them. In the meantime, you can follow the work to produce these mockups by visiting this Phabricator task: T243248.

22 May 2020
Design approach


 * Initially, we will focus on improving the following aspects of the new discussion workflow:
 * Adjusting the language throughout the workflow to make it more discussion-specific.
 * Enhancing the editing tool so people can write what they want to talk about without needing to learn or know about wikicode.
 * Introducing subtle automations that guide people towards writing and posting topics that make it easy for others to understand and engage with.
 * You can read more details about this approach and examples of interventions we are exploring here: Design approach.

1 May 2020
Usability testing of existing experience


 * We have completed four rounds of usability tests to better understand what people who are new to editing Wikipedia talk pages experience when attempting to start a new discussion.
 * You can review these findings and what are planning in response below: Usability testing.

27 March 2020
Usability testing of existing experience


 * To ensure the designs we create meet the needs of newer contributors seeking to talk with other people on Wikipedia, we are running a series of usability tests of the existing "start a new discussion thread" workflow.
 * You can expect to see the research findings posted by mid-April, in the Usability testing section below.

Open questions
This section contains questions the team would value your input on.


 * 1) How do you typically go about starting a new conversation/section on a talk page? Does your workflow vary depending on the namespace?
 * 2) Are there aspects about your current workflow(s) for starting a new conversation/section you find to be inefficient or limiting?
 * 3) Have you noticed others having difficulty starting new discussion threads?

The team is curious to hear what you have to say about these questions on the talk page here: Topic:Vjl9e4d6kwjbtbxf.

Objectives
The improvements to the workflow for starting a new discussion thread on Wikipedia talk pages is intended to make it more intuitive for Junior Contributors to initiate conversations in ways other contributors can easily reply to and to help Senior Contributors do the same, with less effort.

We think helping contributors start conversations more easily and with less effort will increase the likelihood these contributors receive the input and guidance they are seeking from others.

It is important to note that making it easier and more accessible for people with less experience contributing to Wikipedia to start new conversations on talk pages could cause an increase in "unproductive" behavior. Accordingly, as this new tool is deployed, we will monitor the edits people publish using it to ensure they are not disruptive to others.

Evaluating impact
This section will contain the methods and data we will use to evaluate the impact of this feature.

Analysis 2: Impact
In the A/B test of the New Topic Tool, we sought to learn two things: To answer the questions above, we ran an A/B test of the New Topic Tool from 27 January 2022 through 25 March 2022 on 20 Wikipedias. The results from this test can be found in the "Findings" section below.
 * 1) Does the New Topic Tool cause a greater percentage of Junior Contributors to publish the new topics they start without a significant increase in disruption?
 * 2) Does the New Topic Tool cause Junior Contributors to be more likely to start (activation) and continue (retention) participating on Wikipedia talk pages?

Timing

This analysis was completed on 13 April 2022. The A/B test ran from 27 January 2022 through 25 March 2022.

Findings
 * The New Topic Tool causes logged-in Junior Contributors to have more success publishing topics:
 * Junior and Senior Contributor New Topic Tool completion rates.pngr Contributors who open the New Topic Tool are ~1.3 times more likely to successfully publish a new topic than Junior Contributors who open the existing add new section workflow.
 * 44.2% of Junior Contributors that opened the New Topic Tool were able to successfully publish at least one new topic during the A/B test compared to 37.2% of Junior Contributors using the existing add new section workflow (+ 7 percentage points; 19% increase).
 * The New Topic Tool causes logged-in Junior Contributors to be less disruptive:New topic reverts Junior Contributors.png
 * There was a 45% decrease (-1.8 percentage points: 4.05% → 2.22%) in the percent of Junior Contributors blocked after posting a new topic using the New Topic Tool.
 * There was an 11.6% decrease (-1.3 percentage points: 11.2% → 9.9%) in the percent of new topics reverted 48 hours after they were posted using the New Topic Tool; however, there is not sufficient evidence to confirm this observed change was due to the New Topic Tool.

Reports
 * Senior Contributors are more successful publishing topics using the existing add section workflow than they are using the New Topic Tool.
 * 66% of Senior Contributors that opened the New Topic Tool were able to successfully publish at least one new topic during the A/B test compared to 71% of Senior Contributors using the existing add new section workflow (- 5 percentage points; 7% decrease).
 * The New Topic Tool led to a slight increase in the new topic completion rate for logged-out contributors; however, there is not sufficient evidence to confirm this observed change was due to the New Topic Tool.
 * 9.6% of logged-out contributors that opened the New Topic Tool were able to successfully publish at least one new topic during the A/B test compared to 8.1% of Junior Contributors using the existing add new section workflow (+ 1.5 percentage points; 18.5% increase).
 * The New Topic Tool led to slight increases in the revert rate for new topic tool edits made by logged-out users and logged-in Senior Contributors; however, there is not sufficient evidence to confirm either of these observed changes were due to the New Topic Tool.
 * There was a 0.6 percentage point increase (1.5% → 2.1%; 40% increase) in the revert rate for new topic tool edits made by Senior Contributors compared to edits they made using the previous add section link.
 * There was a 3 percentage point increase (20.7% → 23.7%; 14.5% increase) in the revert rate for new topic tool edits made by logged-out users compared to edits they made using the previous add section link.

The full A/B test reports can be found here: Logged-In New Topic AB Test Report + Logged-Out New Topic AB Test Report.

Analysis 1: Adoption
In this first analysis, we sought to learn whether people who have the New Discussion Tool enabled as a beta feature are finding the tool disruptive and behaving in the ways they expect. We were also curious to begin to understand how heavily people were depending on the tool to start talk page discussions.

To determine the extent to which people were finding the tool "disruptive" and "behaving in ways they expect," we looked at the following metrics:


 * The percent of distinct contributors that explicitly disabled the New Discussion Tool after publishing at least one new topic with the tool.
 * The percent of all published edits made with the New Discussion Tool that are reverted within 48 hours of being published compared to the percent of all talk page edits made with full-page editing that are reverted within 48 hours of being published.

To begin to understand how "heavily people were depending on the tool to start talk page discussions", we looked at the following metrics:


 * The percent of distinct contributors who published at least one new topic with the tool.
 * For contributors that have published at least 1 new topic with the New Discussion Tool, the percent of distinct contributors that used the New Discussion Tool to create the following percentage of all new topics: 0-25%, 26-50%, 51-75%, or 76-100% of new topics.

Timing

This analysis was completed on 17 August 2021.

Conclusions and Findings


 * Conclusion: The New Discussion Tool is not disruptive to the majority of people who have used it as well as to the people who maintain the pages on which the tool is being used.
 * Finding: 91.68% of contributors that published at least one new topic with the New Discussion Tool kept the tool enabled during the reviewed time period.
 * Finding: Of all the edits Junior Contributors made to talk pages during the time period that was analyzed, the proportion of reverted edits Junior Contributors made with the New Discussion Tool was 21.3% less than the proportion of reverted edits they made with the existing, full-page editing interface.
 * Conclusion: Among people who started multiple discussions during the analysis period, nearly all of them came to depend on the New Discussion Tool to start new discussions.
 * Finding: Of the contributors that published more than one new topic on a talk page including at least one with the New Discussion Tool, 95.3% of these contributors published between 75 to 100 percent of all their new topics using the New Discussion Tool.

Report

You can review the full analysis results in this Jupyter notebook.

Background
The Editing Team is committed to improving how contributors communicate about their work on Wikipedia, by making existing wikitext talk pages easier and more efficient to use.

A key part of the process of communicating with other contributors is starting a new discussion thread. Trouble is, as previous research and the Talk Page Consultation 2019 uncovered, contributors, across experience levels, find the workflow for starting a new discussion on talk pages challenging. Specifically, our research has found:


 * Junior Contributors do not recognize talk pages as places to talk with others. To many newer contributors, talk pages look like Wikipedia articles. There is no discussion-specific interface or elements that make it clear to people what these pages are used for: to talk with other editors to improve the encyclopedia. This lack of context makes it difficult for people to recognize the conversations happening on talk pages as well as the affordances (e.g. buttons and links) that would enable them to start new ones.
 * Junior Contributors do not sign their edits. When newer contributors do not sign the new discussions they start, they create more work for people wanting to respond. For in order for them to know who to address their response to, they need to navigate to a separate page (the talk page's history page). Additionally, when people do not sign their comments, others will miss out on the opportunity to quickly reply using tools like the new Replying feature.
 * Junior Contributors find the workflow difficult to discover. Many talk pages contain large yellow infoboxes. While these infoboxes are helpful for communicating the quality of an article, editing instructions and links to archived conversations, they, "...are so prominent they distract people from most important actions on a talk page (start a new topic, reply, edit, etc)." [ 1] Other research has shown newer contributors can miss the "New section" link altogether, clicking "Edit source" instead, causing them further confusion. [ 2]

Design
This section contains information about design strategy.

Approach
For people to be confident starting a new conversation on a talk page, we have identified four broad conditions that need to be met: Initially, we are going to focus on improving "conditions" 2., 3., 4., and 5. This means, we will start by experimenting with designs that:
 * 1) People recognize talk pages as places to communicate with others.
 * 2) People know what to click/press to initiate the process for talking about something new.
 * 3) People have the tools they need to represent what they want to talk about in ways others can understand.
 * 4) People know what to click/press to make others aware of what they want to talk about.
 * 5) People are confident what they are wanting to talk about is visible to others.
 * Adjust the language throughout the workflow to make it more discussion-specific. An example: adjusting the "Publish changes" button to read "Add topic."
 * Enhance the editing tool so people can write what they want to talk about without needing to learn or know about wikicode. An example: introducing a way for people to draft discussion topics using a rich text editor.
 * Introduce subtle automations that guide people towards writing and posting topics that make it easy for others to understand and engage with. An example: automatically signing new discussion posts and requiring people to include a "Subject" in the section/discussion they are drafting.

Version 1.0
The "designs" in this section show the approach we are considering implementing for the first iteration of the New Discussion Tool.

These designs are intended to arrive at an opinion about the following aspects of the tool's user experience: Designs
 * Composition: Where and how do people draft the new topic they are planning to add to a talk page?
 * Posting/Feedback: What is the best way to communicate to people the topic they have written has been posted successfully.

Deployment
This section will contain information about how and where the Starting a new discussion tool will be deployed.

Usability testing
This section contains information about usability testing the team has conducted to better understand contributors' experiences starting new discussions.

V1.0 Prototype
Test goals

The primary goals of this test were to better understand the following:

Testing method (Junior Contributors)
 * Do Junior Contributors find the new workflow for adding new topic intuitive?
 * Is the New Discussion Tool compatible with Senior Contributors' existing workflows?

Two rounds of usability tests were run on usertesting.com with 5 participants in each test. Participants were screened to ensure they were technically advanced web users who have used Wikipedia in some capacity before.

They were asked to start new discussion topics on article and user talk pages on a prototype server via a web browser, while narrating their experience.

Findings (Junior Contributors)

''Below is a summary of the test findings. More details can be found in this ticket on Phabricator: T243249.''


 * ✅ 10/10 participants were able to add new topics on article and user talk pages.
 * ✅ 10/10 participants reporting feeling confident about having taken the correct steps to start a new discussion topic.
 * ✅ 10/10 participants were able to successfully locate the new topics they published to the page.
 * ⚠️ Several participants expected there to be quick and familiar ways for deleting or editing the topic they had published. Work on this will happen in T245225.

Testing method (Senior Contributors)

Invitations for Senior Contributors to try the New Discussion Tool prototype were posted on mediawiki.org as well as at the Arabic, Czech and Hungarian Wikipedias.

Participants were asked to share answers to the following questions:


 * Compare the prototype to the current  experience: are there particular workflows you use the existing   /   workflow for and that the prototype does not support?
 * What do you wish was different about the prototype?
 * What do you appreciate about the prototype?

Findings (Senior Contributors)

Below is a summary of the test findings.


 * ✅ Most of the Senior Contributor test participants appreciated the consistency between how the New Discussion Tool and Reply Tool look and function and were able to use it to publish new topics in ways they expected.
 * ⚠️ Several people noted how it could be valuable to make it so the affordance (think: link, button, etc.) for starting a new topic is easier for people who are new to notice. Work on this will happen in T267444.
 * ⚠️ Several people noted how the tool could be made to appear more visually distinct from the rest of the talk page. Work on this is happening in T269157.

Control test: current start new discussion experience
Test goals

The primary goals of this test were to learn:

Testing method
 * What challenges do Junior Contributors face in the process of starting a new discussion on Wikipedia article's talk page?
 * What challenges do Junior Contributors face in the process of starting a new discussion with another Wikipedia contributor on their user talk page?
 * What steps do Junior Contributors expect to be involved with starting a new discussion on Wikipedia article and user talk pages?

Four rounds of usability tests were run on usertesting.com with 5 participants per test. Each participant was screened to ensure they were technically advanced web users who have used Wikipedia in some capacity before.

They were asked to start new discussion topics on article and user talk pages on a prototype server via a web browser, while narrating their experience.

These test pages can be viewed here: test article talk page / test user talk page.

Findings

Below is a summary of the test findings. More details can be found in this ticket on Phabricator: T239175#5723843.

Positive


 * The majority of test participants were able to start new section/discussions.
 * With the above said, the majority of people who participated did not do so successfully. Where "successfully" means they signed the discussion they started and had confidence they were taking the right steps to begin a conversation about an article or with another editor.

Challenges


 * None the test participants signed the discussions they started.
 * Test participants did not understood how talk pages worked and what effect starting a new discussion would have:
 * “I'm beginning to see this page is just a transcript for previous things. Maybe Alice is helping people so she is putting it on there for people to see.”
 * "If I known my subject line would be displayed so openly here, I would have written something better. I thought this was a personal message."
 * Test participants expressed uncertainty about whether the actions they were taking would lead them to accomplish the task at-hand. Here are a few examples that demonstrate this lack of clarity and uncertainty:
 * "It was not intuitive - I thought I needed to go to the bottom of the page."
 * "...everything was painful, I got lost a couple of times, the layout is very difficult to digest with lots of text and too much colours that don´t mean much unless you are familiar with the site."
 * Multiple test participants found the calls to action confusing. This quote exemplifies the challenge people experienced:
 * "It was challenging to understand that add topic meant starting a discussion.. these are not the same words and it could be mistaken as a fully new topic for example, a subsection of cats that would create an entirely new page, not just a discussion within the current page."
 * The majority of test participants had difficulty locating the "Add topic" button tab on the talk page (article and user).
 * The templates that appear at the top of many article talk pages (Talk page templates) seemed to distract test participants from locating the button to start a new discussion.
 * Test participants expected to be automatically notified when someone responded to the discussion they started:
 * "I would hope that somehow my email account is linked to this so then I can get an email saying something like “Alice has responded to your question” and thenI can go and click on that link from my email that will open the Wikipedia link and then I can reply to Alice there."

Improvements

The findings above are leading us to pursue the following improvements to help people start conversations, in ways other contributors can easily engage, with less effort:


 * Make it easier for people to find/discover the affordance(s) for starting a new discussion.
 * Make it more clear to people what the effects will be of the actions they take.
 * E.g. People will be confident about knowing where the content they post is published and who it will be able to see it.
 * Make it easier for people to start conversations in ways that makes it easy for others to reply to. Where "easy for others to reply to" means things like:
 * Potential responders can quickly see who started the conversation and when they started it.
 * Potential responders can easily understand what the person starting the conversation is wanting to talk about.
 * Make it easier for people to know when someone has responded to something they have said.

Comments

This test highlighted an important tension many Junior Contributors seem to face: technically they finish the task they set out to complete, but do so without being confident they did it correctly. And if they do realize they have made a mistake, they are not equipped to fix it because the proper ways of doing so are not intuitive enough for them to understand. This tension seems to map to a larger issue that we intend to incrementally address: it is not clear to people how talk pages work. Said another way: the current implementation of talk pages lead people to hold various and often inaccurate mental models for how talk pages work.

Next steps

In response to the findings above, we are designing interventions to address the challenges the test surfaced. We will then share a cohesive design/mockup that we will ask for feedback on.

In parallel, we are inviting feedback from Senior Contributors about their experiences starting new discussions on talk pages. You can review what people are saying and/or add your thoughts on the talk page: Topic:Vjl9e4d6kwjbtbxf.

History
Many projects have, and are, working to improve contributors' experiences with talk pages. This project is better off for their existence. Some of the projects the team continues to learn from are listed on the main project page and below. If there is a project you think we should be aware of, please boldly add it here.


 * Convenient Discussions
 * reply-link
 * Flow satisfaction survey
 * Moderated Testing, November 2014: talk pages and Flow
 * Talk pages consultation

Glossary
The Talk pages project glossary is intended to help us all communicate about talk pages more effectively by making sure we have a shared understanding about the words we use in our discussions and documentation throughout the project.