Wikimedia Apps/Team/Android/Anti Vandalism/ja

背景
Android チームではさまざまな言語版のコミュニティから、アプリ版を使った編集の品質を改善してほしいと要請を受けています. 言語版やウィキの枠を越えてコミュニティ参加者の皆さんから公平で偏重のない仲裁用のツールがほしいという声が寄せられています. 当チームの目標は仲裁の公平性を適度に確保しながら既存のワークフロー（作業手順）を強化し、例は2022年コミュニティ要望調査のこちらになります. The Android team has received requests from various language communities to improve the quality of edits made through the app. Community members have asked for moderation tools that are fair and unbiased across language wikis. The team aims to enhance existing workflows while ensuring equity in moderation, an example can be found in the 2022 wishlist.

アプリによってネーティブなウォッチリスト、投稿履歴、編集履歴などの最近の更新、また差分画面に元に戻すボタンとロールバック・ボタンの追加ばかりでなく、さらにアプリ内で仲裁の解決策を作成する機会も得られます. 解決策は既存のモバイル用マイクロ貢献ワークフローに基づいて構築されます.

当チームでは、さまざまな言語版で経験豊富なモデレーターのお役に立つ破壊行為対策ツール（V1）を開発する予定です. さらに、経験豊富でも、これまでに巡回を行ったことがない編集者には、巡回の方法を習得する研修タスクを導入する予定です. これはコミュニティと緊密に協力して実施します. またプラットフォーム横断型の利用者にシームレスな体験を確保することを目指して、ORES などのツール改善は研究チームと協力、さらに、この問題についてモバイル・ウェブ版のツールに取り組むモデレーター・ツール・チームと協調していきます.



製品の要件：

 * Editors can review diffs for a variety of edits
 * Editors have the option to 'revert' or 'skip' suggested changes
 * Recent edits made in various languages can be displayed
 * Editors can create and save warning messages to use when reverting changes
 * Experienced editors will be introduced to patrolling on the app in a later version (V2)

Technical Limitation
Currently, it's not possible to have multiple languages in a single feed, but users will have the ability to switch between languages.



Hypothesis & Assumptions
We believe that simplifying the patrolling tasks and structuring them as advanced-level tasks would be beneficial. Experienced editors without sysops or rollback rights could be assigned easier patrolling tasks or a training task, and once they reach a threshold of initial patrolling activities, they can be promoted to higher levels. To achieve this, we plan to start with a proof of concept for general patrolling of edits based on users with rollback rights.

We understand that patrolling recent edits is a priority and more suitable for mobile than tasks like NPP (New Page Patrol), AFD (Articles for Deletion), and AFC (Articles for Creation). Additionally, we expect that there will be community consensus on implementing a training task across different languages.

Validation
Things that would be helpful to know
 * Key Indicator 1: 65% of target mature audiences using the tool find it helpful for maintaining the quality of wikis and would recommend it to other patrollers
 * Key Indicator 2: Increase in edits made by mature audiences by 5%
 * Key Indicator 3: 10% of target mature audiences engage with filter preferences
 * Key Indicator 4: 65% of target mature audiences engage with the tool at least three times in a thirty-day window
 * Guardrail: Experienced users without rollback rights, both those who have and have not used alternative patrolling tools, should equally understand the workflow
 * Guardrail: There should be no reports of the tool being used to negatively target underrepresented content or contributors based on in-app reporting mechanisms


 * 私たちのこのツールの仕様と他の巡回ツールを比べた場合、MediaWiki のタグ類はどのように対照するか（SWViewer、Huggle、Twinkle）
 * 導入後、取り消し／巻き戻し／感謝の発生は増えたか
 * その他のおすすめ編集タスクと比べ、この対象の監修の間でこのタスクはどのくらい知名度があるか
 * この機能で最も多用されるのはどの捜査か

How to Follow Along
We have created T322083 as our Phabricator Epic to track this work. We invite you to collaborate with us there or on our Talk Page.

このページでは、進捗のたび情報更新をしていく予定です. 改訂版設計フィードバックのセッション参加に興味がある人から、当チームのトークページにてご連絡を受け次第、こちらから手順などをお返事します.

We have the following roll out strategy:

2023年8月：プロジェクトの情報更新と技術面の考慮事項

 * このプロジェクトの更新情報をウィキマニアで発表しました. 録画はYoutube、開始点は 1:24:30 あたりです. 質問があり、警告メッセージを一括して扱う場所を作って、アプリ横断で使えるようにしないかとのことでした. Growth チームと協働するプランがあり、一括警告メッセージの蓋然性を検討する予定です.
 * ORES を非推奨にして、差し戻しのリスクモデルをホストする Liftwing への移行を認めました. 当チームでは「最近の編集」フィードの表示に MediaWiki API を用いており、少なくとも V1 では引き続き善意モデルと有害モデルを表示します（Goodfaith、Damaging）. ReverRisk モデルを使用するとAPI 2件の呼び出しが必要なために機能が遅くなる可能性があり、これもフィルタリングには対応していません. この技術面の決定にフィードバックがある場合は、お気軽に議論のページにコメントしてください.
 * 機能 T343242 のオンボーディングの設計ができました（Onboarding）.

August 2023: Project Updates and Technical Considerations

 * We shared updates about this project at Wikimania. The recording can be found on Youtube starting at 1:24:30. We received a question about having a central place for Warning messages that can be used across apps. We are planning to work with the Growth team to explore the possibility of centralized warning messages.
 * We became aware of the deprecation of ORES and migration to Liftwing which will host the revertrisk model. Due to our team using the MediaWiki API to show the recent edits feed, we will continue to show the Goodfaith and Damaging Models at least in V1. Using the Revert Risk model  would require two API calls, which could slow down the feature, this also doesn’t address filtering. If you have feedback about this technical decision, feel free to comment on our discussion page.
 * We created designs for Onboarding to the feature T343242.



使用感のテスト
リモートの使用感テスト改良版は Android 端末を使う経験豊かな投稿者で編集活動の巡回に時間を使っている6名を対象に実施、それぞれスペイン語版、フランス語版、英語版、中国語版のウィキ類で活動しています. 対象者をこれらのウィキから選んだ基準は、それぞれ管理者あるいはまた巻き戻し者権限を預かる編集者が多いからで、使用感のフィードバックが効率的に集まりました.

Usability Testing
A moderated remote usability test was conducted involving six experienced contributors who use Android devices and volunteer their time for monitoring activities on Spanish, French, English, and Chinese language wikis. We selected participants from these wikis due to their high number of editors with sysop/rollback rights, which allowed us to gather usability feedback effectively.

The patrollers provided valuable insights on early designs and the usability of the 'Edit patrol' tool.

During the test, the editors were tasked with completing short assignments using a limited prototype of the tool. They were also asked to provide real-time narration of their experience as they worked through these tasks.

使い心地テストの詳細は、以下のリンクをご参照ください.

Results
The feedback and valuable suggestions from participants have contributed to enhancing the initial designs. The following improvements were implemented:


 * Participants expressed the desire to have tags displayed for each edit on the recent edits page, and the ability to discern if an editor made multiple edits to a single page.


 * Participants provided further feedback regarding patrolling, which we aim to convey within the tool via onboarding, tooltips, or reminders. The points include:The significance of communicating with empathy to fellow contributors, and the emphasis on prioritising the quality of patrolled edits over their quantity.

The 'Edit patrol' tool is currently in the development phase, and you can track its progress through Phabricator T343224.

February 2023: Initial Design Concepts
The team developed initial design concepts by incorporating insights from existing tools and feedback we gathered on their limitations. We shared these concepts with the Moderator Tools Team and Research Team for their input and feedback. We will use the feedback received to improve the design concepts and establish a protocol for both moderated and asynchronous feedback from app users. Currently, we are initiating direct outreach to schedule moderated feedback sessions and ensure translation mechanisms are in place for those sessions. Once the protocol is available on MediaWiki, we will send out a broader communication to start gathering asynchronous feedback on our talk page.

The design concepts provided are meant to offer examples of how we aim to address the issues faced by other tools. However, please note that we are in the early stages of this process, and the designs will evolve and expand based on the feedback we receive in the coming weeks.

January 2023: Comparative Review and Existing User Research
The team conducted a comparative review to analyze the existing patrolling tools. During the review, we assessed the similarities and differences between the tools and identified which ones were effective on mobile devices.

The complete list of tools we reviewed include:


 * Huggle
 * Twinkle


 * TwinkleMobile
 * STiki
 * SWViewer
 * WikiLoop Double Check
 * CheckWiki
 * ReWarn
 * CheckWiki

Through our evaluation, we found that the majority of tools were optimised for Desktop use, with only three exceptions. Additionally, only two of the above solutions performed well across various language wikis.

Common elements found in these tools were:
 * Icons to represent the main action (e.g. revert, speedy deletion, welcome user, etc.)
 * Access to templates
 * Filters
 * List of contributions
 * Diffs
 * Information about the editor

In addition to the comparative review of existing tools, we also collected recurring requests from patrollers regarding those tools. The common requests include:

By taking these requests into account, we aim to develop a comprehensive patrolling solution that addresses the needs and preferences of patrollers across various languages and user experience levels.
 * Filter contributions by topics of interest and specific time period
 * An easy way to exclude contributions that have already been patrolled
 * No requirement for special privileges like rollback rights to access the tools
 * An option between "skip" and "revert" for more nuanced actions
 * Tools that promote collaboration among patrollers

December 2022: Proof of Concept Prototype
The team developed a proof of concept prototype to explore the capabilities and possibilities using existing APIs. The prototype allowed us to test and experiment with different functionalities and assess the feasibility of our ideas. This early exploration helps us understand the potential of the tools and refine our approach before moving forward with further development.

During our development process, we successfully exposed the ORES score, which provides users with an indication of the likelihood that something may be vandalism. This feature allows users to get insights into the quality of edits and helps them make informed decisions during the patrolling process. We would ideally send feedback to the research team's API for improvements, which will be valuable in improving the accuracy and effectiveness of the ORES score.

2022年11月：当初の慣習を定義する
当チームは調査を実施して管理者や巻き戻し権限を預かる人のウィキペディア・アプリ利用者数の概数を調べました（T322065）. 分析の結果、Android アプリ利用者372名、iOS 利用者176名が権限を預かっていて、このデータに基づき、ロールアウトの優先順位は Android アプリを先にして、iOS 版の開発チームにはウォッチリスト機能の構築に回ってもらいました.

Additionally, we sought to understand the distribution of users with sysops and rollback rights across different languages. This information played a crucial role in determining the initial target language communities for our direct outreach during the initial research phase. By focusing on these specific language communities, we aimed to gain valuable insights and feedback from users who have experience in moderation and patrolling, which will be instrumental in shaping the development of the feature for wider use:
 * enwiki
 * frwiki
 * zhwiki
 * eswiki

当チームでは特定の言語版ウィキならびに OS 利用者に注目して直接、働きかけていますが、 フィードバックとアイデアの投稿はアプリの種別にとらわれず、巡回の経験者ならどなたからも歓迎している点を強調させてください. ウェブ利用者の皆さんの投稿も大切であり、フィードバック募集の優先順位は、やはりアプリ版利用者とさせていただきます.

It is essential to create a unified experience across platforms, and to achieve this, we are in close communication with the Moderator Tools team, who will be building experiences for Web users. By learning from their findings and collaborating closely, we aim to ensure continuity and consistency across both the app and Web platforms. Your feedback and ideas will play a crucial role in shaping the development and improvement of the patrolling feature for all users.