Product Safety and Integrity/Incident Reporting System
|
Incident Reporting System
Helping to get help in harmful incidents
|
As Product Safety and Integrity, we want to provide safer and healthier environments for communities. We are building the Incident Reporting System (IRS) to make it easy for users to find the right path to get help when experiencing a harmful incident.
Goals
[edit]- Make it easier for people who experience harmful incidents to get help.
- Eliminate situations in which people stay silent about incidents because they don't know what to do.
- Ensure people who experience harmful incidents reach the right entities that can help them per community processes.
- Ensure responders are not overwhelmed with invalid reports or issues that belong elsewhere.
Background
[edit]Reporting and processing of harmful incidents has been a topic of interest for Wikimedia communities for many years. With the Universal Code of Conduct, it became crucial to have a discussion about user reporting systems.
The way misconduct and policy violations are dealt with across Wikimedia spaces, wikis, and platforms has developed organically. Each community and group has their way of reporting and processing incidents. It happens via wiki talk pages, noticeboards, email, or private discussions on off-wiki communication channels (Discord, Telegram, and more).
For many users, it's been unclear what to do when an incident happens: where to go, who to talk to, how to report, what information to include in the report, how the report is processed, what happens afterwards, etc. Users must know how to flag an issue and where to go to get help. Some do not feel safe reporting incidents because of the complexity of the reporting process or due to privacy concerns. There is also very little information on what will happen once a report is made and what expectations the user should have.
Updates
[edit]: Trial on English Wikipedia
[edit]We are planning to launch a trial on English Wikipedia. It will be focused on helping editors report potentially bad behavior, without overloading the system. It will take two months, and start small (visible to 5% of users). We will coordinate with English Wikipedia functionaries along the way to agree on the right points to expand visibility.
For the last few weeks, we have been working with functionaries to optimize this trial for English Wikipedia, given English Wikipedia's longstanding policies and processes for dispute resolution. Based on those conversations, we have added a few categories of potentially unwanted behavior. These will only be available on English Wikipedia as they are created specifically for the purposes of the trial. We have also made it possible to set external links as the destinations for reporting. This allows the use of URL query string parameters such as preload and may make the reporting even simpler and more efficient. It is available on all wikis where IRS has been deployed.
During the trial, we will focus on monitoring the volume of new reports, checking that reports are routed correctly, and identifying any immediate issues. We will be coordinating closely with all community members to fix bugs if they arise, and to otherwise streamline the process. For example, we are exploring some ways to help people more directly submit their reports.
: Share your opinion on the latest mockups!
[edit]
Earlier this year, we talked to a few groups of community members and addressing their comments, we made changes to the previous design:
- Added a clear distinction between incidents involving immediate threats of physical harm (reported to the Wikimedia Foundation's Trust and Safety team) and any other incidents (the ones for the community to act on): the user needs to choose between these two in the first step
- Consolidated incident categories to make them clearer ("Bullying, intimidation, threats or repeated insults" or "Trolling or hounding (stalking)" to name just two) and added content vandalism-related categories (like "Spam or promotional content")
- Each community-processed incident type now points to a customized "Support information" step
- Added more customization to the "Support information" steps, to allow communities adjust the tools to their needs and existing processes and documentation:
- Mandatory and optional sections with default text provided
- Editable links in pre-defined paragraphs
- Paragraph text that can be enabled or disabled. Unfortunately, we can't make the text editable. Technical limitations prevent us from this.
We would like you to review the new mockups and let us know:
- Does the flow that helps users report incidents processed by the community look good to you? We are particularly interested in hearing your thoughts on:
- The types of community-processed incidents
- The "Support information" step (in the design we use "doxing" as an example)
- Is there anything we should improve before rolling it out on more wikis?
Please share comments on the talk page. Thanks!
Product & design specifications
[edit]Use cases
[edit]The needs of a user looking to get help with a harmful incident are as follows:
- I want to be able to indicate the type of the incident I'm experiencing so that I can get help with resolving the issue.
- I want to easily get information about the help resources available that are relevant to the issue I'm experiencing.
- I want to be clearly informed when the help is provided by the community and not the Foundation, so that I can adjust my expectations accordingly.
The needs of an administrator/functionary are as follows:
- I want to be able to customize the "support information" step that is presented to reporters, according to the incident they're experiencing and the policies of my wiki.
- If my wiki is not large, I want to have a standard/default template that I can use in case my wiki doesn't have established policies or processes.
Immediate threats of physical harm (processed by T&S)
[edit]
The user can submit a report when they become aware of a situation involving serious risk to their (or someone else's) physical safety. Once submitted, these reports are sent to the Trust and Safety team at the Wikimedia Foundation who are responsible for reviewing and responding to such issues.
Any other incidents (processed by community members)
[edit]Users can select one of the types of unwanted behavior as defined in the Universal Code of Conduct. They are guided to a "support information" step that provides clear information on how to get support with an incident of the selected type. The goal is to ensure users receive the right support in a way that reflects the needs and practices of the local community.
For each incident type, local communities can customize a limited number of content sections with wiki-specific terminology and links, according to their policies and practices.
Administrators can now set up external links as the destinations for reporting. This allows the use of URL query string parameters such as preload and may make the reporting even simpler and more efficient.
Incident types
[edit]| 1st-tier category | 2nd-tier category | UCoC provision | UCoC definition, or description |
|---|---|---|---|
| Immediate threat of physical harm (processed by T&S) | Threats of physical harm | — | Intending to commit violence against an individual |
| Threats of self harm | — | Intending to commit an act of harming oneself | |
| Threats of public harm | — | Intending to commit an act of mass violence | |
| Unacceptable user behavior (processed by community members) | Bullying, intimidation, threats or repeated insults | 3.1 – Harassment | "This includes name calling, using slurs or stereotypes, and any attacks based on personal characteristics. Insults may refer to perceived characteristics like intelligence, appearance, ethnicity, race, religion (or lack thereof), culture, caste, sexual orientation, gender, sex, disability, age, nationality, political affiliation, or other characteristics. In some cases, repeated mockery, sarcasm, or aggression constitute insults collectively, even if individual statements would not."
"Explicitly or implicitly suggesting the possibility of physical violence, unfair embarrassment, unfair and unjustified reputational harm, or intimidation by suggesting gratuitous legal action to win an argument or force someone to behave the way you want." "This includes encouraging someone else to commit self-harm or suicide as well as encouraging someone to conduct violent attacks on a third party." |
| Sexual harassment | "Sexual attention or advances of any kind towards others where the person knows or reasonably should know that the attention is unwelcome or in situations where consent cannot be communicated." | ||
| Exposing private personal information (Doxing) | "Sharing other contributors' private information, such as name, place of employment, physical or email address without their explicit consent either on the Wikimedia projects or elsewhere, or sharing information concerning their Wikimedia activity outside the projects." | ||
| Trolling or Hounding (Stalking) | "Deliberately disrupting conversations or posting in bad-faith to intentionally provoke."
"Following a person across the project(s) and repeatedly critiquing their work mainly with the intent to upset or discourage them. If problems are continuing after efforts to communicate and educate, communities may need to address them through established community processes." | ||
| Hateful or discriminatory content | 3.3 – Content vandalism and abuse of the projects | Content vandalism containing "hate speech in any form, or discriminatory language aimed at vilifying, humiliating, inciting hatred against individuals or groups on the basis of who they are or their personal beliefs"
"The use of symbols, images, categories, tags or other kinds of content that are intimidating or harmful to others outside of the context of encyclopedic, informational use. This includes imposing schemes on content intended to marginalize or ostracize" (Hate speech towards an editor falls under "Bullying, intimidation, threats or repeated insults") | |
| Spam or promotional content | Other content vandalism | ||
| Something else | Including 3.2 – Abuse of power, privilege, or influence |
Community configuration
[edit]Community Configuration is a tool allowing communities to set up and control the configuration of different features. For the Incident Reporting System we are using Community Configuration to allow communities to customize the "support information" step.
Example of how customizing the "support information" step for "Exposing private personal information (Doxing)" might look:
-
The "support information" step will have a default view (fields that are left blank will not display) -
Communities will then be able to customize the default view via Community Configuration
.png)
.png)
Deployment timeline
[edit]- First and limited version on Portuguese Wikipedia –
- Next pilot wikis –
- English Wikipedia trial – –
Contact
[edit]
Frequently asked questions
[edit]Is the Foundation trying to change existing community processes?
Our plan for the IRS is not to change any community process. The goal is to connect to existing processes.
Is there data available about how many incidents are reported per year?
We have usage data collected on the wikis where IRS has been deployed. The largest of them is Portuguese Wikipedia. However, the volume of data is too low to make predictions based on it. We want to address this by launching a trial on English Wikipedia, with the audience similar to the entire audience on Portuguese Wikipedia.
Beyond the IRS usage data, there is not a lot of clear data we can use. There are a couple of reasons for this. First, issues are reported in various ways and those differ from community to community. Capturing that data completely and cleanly is highly complicated and would be very time consuming. Second, the interpretation of issues also differs. Some things that are interpreted as harassment are just wiki business (e.g. deleting a promotional article). Review of user conduct/behaviour issues may also need cultural or community context. We cannot automate and visualize data or count it objectively. The incident reporting system is an opportunity to solve some of these data needs.
How are different types of harmful incidents – such as harassment, threats, or misconduct – defined in the Incident Reporting System?
We are using these terms as defined in the Universal Code of Conduct.
What questions are you trying to answer with the first version of the product?
Here are the questions we need to answer:
- What kind of user conduct/behavior issues do people need help with?
- How many people report the respective kinds of issues?
- How big is this problem?
- Can we get a clearer picture of the magnitude of user conduct issues? Can we get some data around the number of reports? Is user misconduct underreported or overreported?
- Are people currently not reporting various types of user conduct issues because they don't happen or because they don't know how?
- Will this be a lot to handle with our current setup, or not?
- How many are valid complaints compared to people who don't understand the wiki process? Can we distinguish/filter valid complaints, and filter invalid complaints to save volunteer or staff time?
- Will we receive lots of reports filed by people who are upset that their edits were reverted or their page was deleted? What will we do with them?
How does the Wikimedia movement compare to how other big platforms like Facebook/Reddit handle harassment?
While we do not have any identical online affinity groups, the Wikimedia movement is most often connected with Facebook and Reddit in regard to how we handle harassment. What is important to consider is nobody has resolved harassment. Other platforms struggle with content moderation, and often they have paid staff who try to deal with it. Two huge differences between us and Reddit and Facebook are the globally collaborative nature of our projects and how communities work to resolve harassment at the community-level.
Pre-project research
[edit]The following document is a completed review of research from 2015–2022 the Wikimedia Foundation has done on online harassment on Wikimedia projects. In this review we've identified major themes, insights, and areas of concern and provided direct links to the literature.
Our team has been studying previous research and community consultations to inform our work. We revisited the Community health initiative User reporting system proposal and the User reporting system consultation of 2019. We have also been trying to map out some of the conflict resolution flows across wikis to understand how communities are currently managing conflicts. Below is a map of the Italian Wikipedia conflict resolution flow. It has notes on opportunities for automation.
-
A synthesis of research from 2015–2022 that identifies major themes in the problem space as well as user needs, challenges, considerations, and previous work. -
On Italian Wikipedia, there's a 3-step policy in place for conflict resolution. This map visualizes this process and tries to identify opportunities for automation for both editors and admins.
