Product Safety and Integrity/Incident Reporting System
|
Incident Reporting System
Helping to get help in harmful incidents
|
As Product Safety and Integrity, we want to provide safer and healthier environments for communities. We are building the Incident Reporting System (IRS) to make it easy for users to find the right path to get help when experiencing a harmful incident.
Goals
[edit]- Make it easier for people who experience harmful incidents to get help.
- Eliminate situations in which people stay silent about incidents because they don't know what to do.
- Ensure people who experience harmful incidents reach the right entities that can help them per community processes.
- Ensure responders are not overwhelmed with invalid reports or issues that belong elsewhere.
Background
[edit]Reporting and processing of harmful incidents has been a topic of interest for Wikimedia communities for many years. With the Universal Code of Conduct, it became crucial to have a discussion about user reporting systems.
The way misconduct and policy violations are dealt with across Wikimedia spaces, wikis, and platforms has developed organically. Each community and group has their way of reporting and processing incidents. It happens via wiki talk pages, noticeboards, email, or private discussions on off-wiki communication channels (Discord, Telegram, and more).
For many users, it's been unclear what to do when an incident happens: where to go, who to talk to, how to report, what information to include in the report, how the report is processed, what happens afterwards, etc. Users must know how to flag an issue and where to go to get help. Some do not feel safe reporting incidents because of the complexity of the reporting process or due to privacy concerns. There is also very little information on what will happen once a report is made and what expectations the user should have.
Updates
[edit]: Share your opinion on the latest mockups!
[edit]
Earlier this year, we talked to a few groups of community members and addressing their comments, we made changes to the previous design:
- Added a clear distinction between incidents involving immediate threats of physical harm (reported to the Wikimedia Foundation's Trust and Safety team) and any other incidents (the ones for the community to act on): the user needs to choose between these two in the first step
- Consolidated incident categories to make them clearer ("Bullying, intimidation, threats or repeated insults" or "Trolling or hounding (stalking)" to name just two) and added content vandalism-related categories (like "Spam or promotional content")
- Each community-processed incident type now points to a customized "Support information" step
- Added more customization to the "Support information" steps, to allow communities adjust the tools to their needs and existing processes and documentation:
- Mandatory and optional sections with default text provided
- Editable links in pre-defined paragraphs
- Paragraph text that can be enabled or disabled. Unfortunately, we can't make the text editable. Technical limitations prevent us from this.
We would like you to review the new mockups and let us know:
- Does the flow that helps users report incidents processed by the community look good to you? We are particularly interested in hearing your thoughts on:
- The types of community-processed incidents
- The "Support information" step (in the design we use "doxing" as an example)
- Is there anything we should improve before rolling it out on more wikis?
Please share comments on the talk page. Thanks!
: Conclusions from the conversations and next steps
[edit]As we previously posted, we planned conversations with a few groups (Stewards, Arbitration Committees, and U4C members) to hear their comments on the development of the Incident Reporting System. Based on what we heard, we are designing a flow for reporting incidents that are not emergencies.
The new flow needs to be legally and community-acceptable, and designed to work across all wikis. It will reflect and complement, rather than duplicate or disrupt, existing practices of different communities. In consequence, we have decided to make it configurable via Community Configuration.
We are also exploring an improved list of violation types, and customizable "Support information" pages based on the type of incident. In parallel, we are researching different wiki communities to better understand existing practices. This is to make the default setting reflect the typical needs well, and to make it address compliance requirements.
Lastly, we wanted to note here that as the previous Trust and Safety Product team, we have been merged with the former Product Security and formed Product Safety and Integrity. People previously involved in the IRS project have not changed their roles, including the owner (Madalina). However, due to this merger, Madalina now has support from Eric (the owner of the broader Safety and Security area) and Szymon (the Movement Communications person for the Safety and Security area). We encourage you to contact any of us if you'd like to talk about IRS. 🖖
Product & design specifications
[edit]Use cases
[edit]As a user looking to get help with a harmful incident
- I want to be able to indicate the type of the incident I'm experiencing so that I can get help with resolving the issue.
- I want to easily get information about the help resources available that are relevant to the issue I'm experiencing.
- I want to be clearly informed when the help is provided by the community and not the Foundation, so that I can adjust my expectations accordingly.
As an administrator/functionary,
- I want to be able to customize the "support information" step that is presented to reporters, according to the incident they're experiencing and the policies of my wiki.
- of a not large wiki, I want to have a standard/default template that I can use in case my wiki doesn't have established policies or processes.
Immediate threats of physical harm (processed by T&S)
[edit]
The user can submit a report when they become aware of a situation involving serious risk to their (or someone else's) physical safety. Once submitted, these reports are sent to the Trust and Safety team at the Wikimedia Foundation who are responsible for reviewing and responding to such issues.
Any other incidents (processed by community members)
[edit]Users can select one of the types of unwanted behavior as defined in the Universal Code of Conduct. They are guided to a "support information" step that provides clear information on how to get support with an incident of the selected type. The goal is to ensure users receive the right support in a way that reflects the needs and practices of the local community.
For each incident type, local communities can customize a limited number of content sections with wiki-specific terminology and links, according to their policies and practices.
Incident types
[edit]| 1st-tier category | 2nd-tier category | UCoC provision | UCoC definition, or description |
|---|---|---|---|
| Immediate threat of physical harm (processed by T&S) | Threats of physical harm | — | Intending to commit violence against an individual |
| Threats of self harm | — | Intending to commit an act of harming oneself | |
| Threats of public harm | — | Intending to commit an act of mass violence | |
| Unacceptable user behavior (processed by community members) | Bullying, intimidation, threats or repeated insults | 3.1 – Harassment | "This includes name calling, using slurs or stereotypes, and any attacks based on personal characteristics. Insults may refer to perceived characteristics like intelligence, appearance, ethnicity, race, religion (or lack thereof), culture, caste, sexual orientation, gender, sex, disability, age, nationality, political affiliation, or other characteristics. In some cases, repeated mockery, sarcasm, or aggression constitute insults collectively, even if individual statements would not."
"Explicitly or implicitly suggesting the possibility of physical violence, unfair embarrassment, unfair and unjustified reputational harm, or intimidation by suggesting gratuitous legal action to win an argument or force someone to behave the way you want." "This includes encouraging someone else to commit self-harm or suicide as well as encouraging someone to conduct violent attacks on a third party." |
| Sexual harassment | "Sexual attention or advances of any kind towards others where the person knows or reasonably should know that the attention is unwelcome or in situations where consent cannot be communicated." | ||
| Exposing private personal information (Doxing) | "Sharing other contributors' private information, such as name, place of employment, physical or email address without their explicit consent either on the Wikimedia projects or elsewhere, or sharing information concerning their Wikimedia activity outside the projects." | ||
| Trolling or Hounding (Stalking) | "Deliberately disrupting conversations or posting in bad-faith to intentionally provoke."
"Following a person across the project(s) and repeatedly critiquing their work mainly with the intent to upset or discourage them. If problems are continuing after efforts to communicate and educate, communities may need to address them through established community processes." | ||
| Hateful or discriminatory content | 3.3 – Content vandalism and abuse of the projects | Content vandalism containing "hate speech in any form, or discriminatory language aimed at vilifying, humiliating, inciting hatred against individuals or groups on the basis of who they are or their personal beliefs"
"The use of symbols, images, categories, tags or other kinds of content that are intimidating or harmful to others outside of the context of encyclopedic, informational use. This includes imposing schemes on content intended to marginalize or ostracize" (Hate speech towards an editor falls under "Bullying, intimidation, threats or repeated insults") | |
| Spam or promotional content | Other content vandalism | ||
| Something else | Including 3.2 – Abuse of power, privilege, or influence |
Community configuration
[edit]Community Configuration is a tool allowing communities to set up and control the configuration of different features. For the Incident Reporting System we are using Community Configuration to allow communities to customize the "support information" step.
Example of how customizing the "support information" step for "Exposing private personal information (Doxing)" might look:
-
The "support information" step will have a default view (fields that are left blank will not display) -
Communities will then be able to customize the default view via Community Configuration
.png)
.png)
Deployment timeline
[edit]- First and limited version on Portuguese Wikipedia –
- Next pilot wikis –
- Final deployment (estimated deadline) – TBD
Contact
[edit]
Frequently asked questions
[edit]Is the Foundation trying to change existing community processes?
Our plan for the IRS is not to change any community process. The goal is to connect to existing processes.
Is there data available about how many incidents are reported per year?
Right now there is not a lot of clear data we can use. There are a couple of reasons for this. First, issues are reported in various ways and those differ from community to community. Capturing that data completely and cleanly is highly complicated and would be very time consuming. Second, the interpretation of issues also differs. Some things that are interpreted as harassment are just wiki business (e.g. deleting a promotional article). Review of user conduct/behaviour issues may also need cultural or community context. We cannot automate and visualize data or count it objectively. The incident reporting system is an opportunity to solve some of these data needs.
How are different types of harmful incidents – such as harassment, threats, or misconduct – defined in the Incident Reporting System?
We are using these terms as defined in the Universal Code of Conduct.
What questions are you trying to answer with the first version of the product?
Here are the questions we need to answer:
- What kind of user conduct/behavior issues do people need help with?
- How many people report the respective kinds of issues?
- How big is this problem?
- Can we get a clearer picture of the magnitude of user conduct issues? Can we get some data around the number of reports? Is user misconduct underreported or overreported?
- Are people currently not reporting various types of user conduct issues because they don't happen or because they don't know how?
- Will this be a lot to handle with our current setup, or not?
- How many are valid complaints compared to people who don't understand the wiki process? Can we distinguish/filter valid complaints, and filter invalid complaints to save volunteer or staff time?
- Will we receive lots of reports filed by people who are upset that their edits were reverted or their page was deleted? What will we do with them?
How does the Wikimedia movement compare to how other big platforms like Facebook/Reddit handle harassment?
While we do not have any identical online affinity groups, the Wikimedia movement is most often connected with Facebook and Reddit in regard to how we handle harassment. What is important to consider is nobody has resolved harassment. Other platforms struggle with content moderation, and often they have paid staff who try to deal with it. Two huge differences between us and Reddit and Facebook are the globally collaborative nature of our projects and how communities work to resolve harassment at the community-level.
Pre-project research
[edit]The following document is a completed review of research from 2015–2022 the Wikimedia Foundation has done on online harassment on Wikimedia projects. In this review we've identified major themes, insights, and areas of concern and provided direct links to the literature.
Our team has been studying previous research and community consultations to inform our work. We revisited the Community health initiative User reporting system proposal and the User reporting system consultation of 2019. We have also been trying to map out some of the conflict resolution flows across wikis to understand how communities are currently managing conflicts. Below is a map of the Italian Wikipedia conflict resolution flow. It has notes on opportunities for automation.
-
A synthesis of research from 2015–2022 that identifies major themes in the problem space as well as user needs, challenges, considerations, and previous work. -
On Italian Wikipedia, there's a 3-step policy in place for conflict resolution. This map visualizes this process and tries to identify opportunities for automation for both editors and admins.
