Incident Reporting System
|
Incident Reporting System
Helping to get help in harmful incidents
|
As Product Safety and Integrity, we want to provide safer and healthier environments for communities. We are building the Incident Reporting System (IRS) to make it easy for users to find the right path to get help when experiencing a harmful incident.
Goals
- Make it easier for people who experience harmful incidents to get help.
- Eliminate situations in which people stay silent about incidents because they don't know what to do.
- Ensure people who experience harmful incidents reach the right entities that can help them per community processes.
- Ensure responders are not overwhelmed with invalid reports or issues that belong elsewhere.
Background
Reporting and processing of harmful incidents has been a topic of interest for Wikimedia communities for many years. With the Universal Code of Conduct, it became crucial to have a discussion about user reporting systems.
The way misconduct and policy violations are dealt with across Wikimedia spaces, wikis, and platforms has developed organically. Each community and group has their way of reporting and processing incidents. It happens via wiki talk pages, noticeboards, email, or private discussions on off-wiki communication channels (Discord, Telegram, and more).
For many users, it's been unclear what to do when an incident happens: where to go, who to talk to, how to report, what information to include in the report, how the report is processed, what happens afterwards, etc. Users must know how to flag an issue and where to go to get help. Some do not feel safe reporting incidents because of the complexity of the reporting process or due to privacy concerns. There is also very little information on what will happen once a report is made and what expectations the user should have.
Updates
: End of the trial on English Wikipedia
Between May and early July, Incident Reporting System was tested on English Wikipedia, starting from 5% to 100% of the eligible users. Overall, the tool was not generating a high volume of reports, and we did not see signs or hear any volunteer concerns that the tool was creating any disruption. We did see users interacting with it, and submitting both emergency and non-emergency reports.
Early in the trial, per the English Wikipedia functionaries feedback, we added a feature (we call it direct reporting) where, if the community selected an email-based destination for a particular category of unwanted behavior, the form would show the user a text area to type in the report, and it would then send the report directly to that community-chosen email address.
The tool does not support direct reporting for most categories of unwanted behavior – instead, the user is provided a link and description (chosen by the community) of where to go and what to do there.
The link may include the preload parameter, making it easier to add well-structured reports.
The two charts show that:
- At the end of the trial, the Report button was being shown to around 12,000 users per day.
- The categories most commonly initially selected by users were: Bullying / intimidation, Disruptive editing, and Vandalism – these three add up to over a half of all reports.
We know there are still more improvements to make, including the potential for streamlining the flow through deeper integration with on-wiki processes. But we think the trial has at least demonstrated that this feature can be integrated in a way that doesn't disrupt abuse handling, and can be iterated on from there. We appreciate the community members' thoughts as we consider what to do next. And a special thank you to the functionaries and other volunteers who helped support this trial and help other users.
Product & design specifications
Use cases
The needs of a user looking to get help with a harmful incident are as follows:
- I want to be able to indicate the type of the incident I'm experiencing so that I can get help with resolving the issue.
- I want to easily get information about the help resources available that are relevant to the issue I'm experiencing.
- I want to be clearly informed when the help is provided by the community and not the Foundation, so that I can adjust my expectations accordingly.
The needs of an administrator/functionary are as follows:
- I want to be able to customize the "support information" step that is presented to reporters, according to the incident they're experiencing and the policies of my wiki.
- If my wiki is not large, I want to have a standard/default template that I can use in case my wiki doesn't have established policies or processes.
Immediate threats of physical harm (processed by T&S)
The user can submit a report when they become aware of a situation involving serious risk to their (or someone else's) physical safety. Once submitted, these reports are sent to the Trust and Safety team at the Wikimedia Foundation who are responsible for reviewing and responding to such issues.
Any other incidents (processed by community members)
Users can select one of the types of unwanted behavior as defined in the Universal Code of Conduct. They are guided to a "support information" step that provides clear information on how to get support with an incident of the selected type. The goal is to ensure users receive the right support in a way that reflects the needs and practices of the local community.
For each incident type, local communities can customize a limited number of content sections with wiki-specific terminology and links, according to their policies and practices.
Administrators can now set up external links as the destinations for reporting. This allows the use of URL query string parameters such as preload and may make the reporting even simpler and more efficient.
Incident types
| 1st-tier category | 2nd-tier category | UCoC provision | UCoC definition, or description |
|---|---|---|---|
| Immediate threat of physical harm (processed by T&S) | Threats of physical harm | — | Intending to commit violence against an individual |
| Threats of self harm | — | Intending to commit an act of harming oneself | |
| Threats of public harm | — | Intending to commit an act of mass violence | |
| Unacceptable user behavior (processed by community members) | Bullying, intimidation, threats or repeated insults | 3.1 – Harassment | "This includes name calling, using slurs or stereotypes, and any attacks based on personal characteristics. Insults may refer to perceived characteristics like intelligence, appearance, ethnicity, race, religion (or lack thereof), culture, caste, sexual orientation, gender, sex, disability, age, nationality, political affiliation, or other characteristics. In some cases, repeated mockery, sarcasm, or aggression constitute insults collectively, even if individual statements would not."
"Explicitly or implicitly suggesting the possibility of physical violence, unfair embarrassment, unfair and unjustified reputational harm, or intimidation by suggesting gratuitous legal action to win an argument or force someone to behave the way you want." "This includes encouraging someone else to commit self-harm or suicide as well as encouraging someone to conduct violent attacks on a third party." |
| Sexual harassment | "Sexual attention or advances of any kind towards others where the person knows or reasonably should know that the attention is unwelcome or in situations where consent cannot be communicated." | ||
| Exposing private personal information (Doxing) | "Sharing other contributors' private information, such as name, place of employment, physical or email address without their explicit consent either on the Wikimedia projects or elsewhere, or sharing information concerning their Wikimedia activity outside the projects." | ||
| Trolling or Hounding (Stalking) | "Deliberately disrupting conversations or posting in bad-faith to intentionally provoke."
"Following a person across the project(s) and repeatedly critiquing their work mainly with the intent to upset or discourage them. If problems are continuing after efforts to communicate and educate, communities may need to address them through established community processes." | ||
| Hateful or discriminatory content | 3.3 – Content vandalism and abuse of the projects | Content vandalism containing "hate speech in any form, or discriminatory language aimed at vilifying, humiliating, inciting hatred against individuals or groups on the basis of who they are or their personal beliefs"
"The use of symbols, images, categories, tags or other kinds of content that are intimidating or harmful to others outside of the context of encyclopedic, informational use. This includes imposing schemes on content intended to marginalize or ostracize" (Hate speech towards an editor falls under "Bullying, intimidation, threats or repeated insults") | |
| Spam or promotional content | Other content vandalism | ||
| Something else | Including 3.2 – Abuse of power, privilege, or influence |
Community configuration
Community Configuration is a tool allowing communities to set up and control the configuration of different features. For the Incident Reporting System we are using Community Configuration to allow communities to customize the "support information" step.
Example of how customizing the "support information" step for "Exposing private personal information (Doxing)" might look:
-
The "support information" step will have a default view (fields that are left blank will not display) -
Communities will then be able to customize the default view via Community Configuration
.png)
.png)
Deployment timeline
- First and limited version on Portuguese Wikipedia –
- Next pilot wikis –
- English Wikipedia trial – –
Contact
Frequently asked questions
Is the Foundation trying to change existing community processes?
Our plan for the IRS is not to change any community process. The goal is to connect to existing processes.
Is there data available about how many incidents are reported per year?
We have usage data collected on the wikis where IRS has been deployed. The largest of them is Portuguese Wikipedia. However, the volume of data is too low to make predictions based on it. We want to address this by launching a trial on English Wikipedia, with the audience similar to the entire audience on Portuguese Wikipedia.
Beyond the IRS usage data, there is not a lot of clear data we can use. There are a couple of reasons for this. First, issues are reported in various ways and those differ from community to community. Capturing that data completely and cleanly is highly complicated and would be very time consuming. Second, the interpretation of issues also differs. Some things that are interpreted as harassment are just wiki business (e.g. deleting a promotional article). Review of user conduct/behaviour issues may also need cultural or community context. We cannot automate and visualize data or count it objectively. The incident reporting system is an opportunity to solve some of these data needs.
How are different types of harmful incidents – such as harassment, threats, or misconduct – defined in the Incident Reporting System?
We are using these terms as defined in the Universal Code of Conduct.
What questions are you trying to answer with the first version of the product?
Nke a bụ ajụjụ anyị kwesịrị ịza:
- What kind of user conduct/behavior issues do people need help with?
- How many people report the respective kinds of issues?
- How big is this problem?
- Can we get a clearer picture of the magnitude of user conduct issues? Can we get some data around the number of reports? Is user misconduct underreported or overreported?
- Are people currently not reporting various types of user conduct issues because they don't happen or because they don't know how?
- Will this be a lot to handle with our current setup, or not?
- How many are valid complaints compared to people who don't understand the wiki process? Can we distinguish/filter valid complaints, and filter invalid complaints to save volunteer or staff time?
- Will we receive lots of reports filed by people who are upset that their edits were reverted or their page was deleted? What will we do with them?
How does the Wikimedia movement compare to how other big platforms like Facebook/Reddit handle harassment?
While we do not have any identical online affinity groups, the Wikimedia movement is most often connected with Facebook and Reddit in regard to how we handle harassment. What is important to consider is nobody has resolved harassment. Other platforms struggle with content moderation, and often they have paid staff who try to deal with it. Two huge differences between us and Reddit and Facebook are the globally collaborative nature of our projects and how communities work to resolve harassment at the community-level.
Pre-project research
The following document is a completed review of research from 2015–2022 the Wikimedia Foundation has done on online harassment on Wikimedia projects. In this review we've identified major themes, insights, and areas of concern and provided direct links to the literature.
Our team has been studying previous research and community consultations to inform our work. We revisited the Community health initiative User reporting system proposal and the User reporting system consultation of 2019. We have also been trying to map out some of the conflict resolution flows across wikis to understand how communities are currently managing conflicts. Below is a map of the Italian Wikipedia conflict resolution flow. It has notes on opportunities for automation.
-
A synthesis of research from 2015–2022 that identifies major themes in the problem space as well as user needs, challenges, considerations, and previous work. -
On Italian Wikipedia, there's a 3-step policy in place for conflict resolution. This map visualizes this process and tries to identify opportunities for automation for both editors and admins.
