Edit Review Improvements/New filters for edit review/調査

編集の査読改善 (ERI: Edit Review Improvements)
編集の査読改善 用語集
機能
編集査読用のフィルター クイックツアー フィードバック 調査研究 ハグルの改善 （提案）
説明文書
絞り込み 詳細フィルター 強調表示 品質と編集意図 フィルターの組み合わせを保存 ライブ・アップデート よくある質問
技術的事項
絞り込みの実行 Objective Revision Evaluation Service (ORES) ReviewStream ガジェット

このページは編集の査読の新しい拡張機能について行った、繰り返し法による利用体験調査の概要です。段階ごとにまとめ、調査報告書へリンクさせました。目標はユーザーに利用体験調査のやり方や、結果を製品の方向性や決定の場にどう伝えるか詳しく知ってもらい、現段階で検討した製品の可能性を紹介することです。

概要

The New filters for edit review became available on Recent Changes as a beta feature on May 2017, and as default in October 2017. The new filters are intended to help editors to find their edits of interest to review. Some filters provided were not available before, and some are based on new machine learning capabilities.

直感的でより強力な経験をユーザーに提供し、以前はうまくサポートされなかった使用例への対処を改善したい。そう考えた今回の取り組みではこの分野に固有の課題が示され、ユーザーを理解するとともに私たちの解決案がどれほどお役に立つのか、主に調査を介して学ぶことができたのです。

よろしければ以下の調査の概要、さらにリンク先にご用意した詳細な報告書に目を通してください。

主な調査項目と結果

• 項目 1: 調査の参加者は「最近の変更」で、新フィルタのユーティリティと編集の査読の機能に気づいたか？
  • 成果: 気づいた上におおむね新フィルタと機能面に非常に肯定的だった。
  • プロジェクトの意思決定に反映した方法: 肯定的なフィードバックを受け、フィルタと機能を増やしてテスト。既存の編集査読の要素を繰り返し修正した。
• 項目 2: 参加者が「最近の変更」で新フィルタや機能に気づいた頻度と理解度、実際に使用したかどうか。
  • 成果: 調査の実施回ごとに結果はばらついたものの、発見しやすさ・理解度を高める改善点がわかった。さらにそれと相関して、日々のウィキのワークフローで参加者がどの程度ベータ版を試用しようとしたか把握できた。
  • How it informed the project decisions: At each stage in the research process, we were constantly adding, removing, and editing both new and existing elements of the Recent Changes page based on our evaluation of the participant sessions and also their direct feedback/recommendations.
• 項目 3: フィルタと機能面の充実はウォッチリストにどう反映したか？
  • 成果: これらの機能がウォッチリストにあるべきかどうか、参加者集団の関心は低かった。この概念でユーティリティに気づかなかったことがうかがえる。
  • How it informed the project decisions: For the moment, this is not moving forward, but further investigation is warranted as the participant cohort were primarily users of Recent Changes, not Watchlist. The new filters on the Watchlist will not be exposed by default, remaining as a beta feature as more feedback is collected.

課題

• マクロレベルでは、全ての人にとってウィキとそのツールをより直感的かつ便利にすることが目標だが、これらの調査では参加者を編集査読ページの（比較的常連の）ユーザーに制限した。 結局のところ、調査の参加者の《ほとんど》が英語版ウィキペディアを中心に活動していたが、他の言語版のウィキペディアやウィキバーシティからも参加を得た。
• 実益や趣旨、成果にかかわらず、変更は困難な場合がある。必ず喜ばれる変更というものはないにせよ、原則はフィードバックと最大数のユーザーに役立つかどうかに基づき、製品に関する意思決定をできるだけ多く行うことを目指していく。
• 比較的広い範囲で捉えると、特に生成的な研究要素が加わった場合、ユーザビリティは大きな取り組みである。私たちは製品の意思決定を下す際に、細部と全体的な外観、使い心地と有用性の両立である。
• もっと多くの一般の参加者が必要！ 貢献者空間に興味があって今後の調査に参加したい場合は、関心のある分野をDaisy Chenまでご連絡ください。一緒にやりましょう！

今後の見通し

• この調査は期待のできる成果が出ましたが、さまざまな用途をサポートするこのウィキで、あなたの活動に新しいフィルタがどのように役立つか聞かせてください。プロジェクトのトーク ページで経験を共有してみませんか。
• 貢献者空間にご興味は？ 今後の研究に参加してフィードバックを提供しませんか？ もしよろしければ関心のある分野をDaisy Chenに連絡してください。一緒にやりましょう！

Recent Changes Filters Usability Round 1 2016.09-11

調査項目

• 構想と趣旨
  • Do users understand what the filter categories represent, specifically the Quality, Intent, and Experience filters?
  • Do subjects understand what the different ORES (Objective Revision Evaluation Service) levels mean?
  • Do users understand the difference between filtering and highlighting, and how they might use these singly or in combination?

• Usability
  • Do users notice and/or understand the default filters?
  • Are users able to find the correct filters and use them in combination effectively?
  • Do users find and learn to effectively use the highlighting to isolate targets?
  • Can they manage and interpret the highlight colors?
  • Do they find the “ignore highlights” button and understand its use?

• Effectiveness
  • How do users feel the new tools will affect their work and workflows?
  • What are users’ reactions regarding the new interface and its functionalities?
  • Do users feel any differently about the new interface as compared to the current Recent changes page?
    • Do they see that the new interface as being functionally neutral, inferior or superior?

Curious about the more nitty-gritty details on the research set-up? Take a look at the 2016 Recent Changes Filters research.

結果

Overall, the new Recent Changes filters features were well-received. Users generally find that the prototype Recent Changes are improvements over the current and a majority think they would incorporate these features into their work.

Though features can still use some clarification, refinement, and ongoing iteration/improvement, using them is pretty intuitive overall and even when there are user hesitations, is a learnable process.

Over the course of this first round of usability testing, two prototypes were used with the second prototype building on findings from the first prototype.

次のステップ

Discuss existing issues and corresponding recommendations for improvement. Prototype 2 is recommended to go to beta for further user feedback and iteration.

研究報告書

調査結果の説明。

Recent Changes Filters Usability Round 2 2017.05-06

調査項目

After introducing the new filters for edit review to positive feedback, the Collaboration team’s next iteration aimed to include support for advanced functions.

In these sessions, we utilized a prototype testing environment and lead users through a series of tasks. Through completing said tasks, we are able to evaluate if research participants were aware of the functionalities, found them intuitive to use, and whether they found utility in them.

• What elements of the extended filters are most and least useful for users?
• Do users notice the new capabilities?
• If they find them, are they easy to understand and use?

New page elements/functionalities

• Namespaces, tag, users, and categories filters
• Filter set bookmarking
• Display options (size and pagination of results)
• Navigating and updating results

Read more details on the research planning and protocol at the 2017 Recent Changes Extended Filters research.

結果

Generally, participants had positive responses to the new filters and functionalities. To different degrees/varying use cases, most of these new features have audiences ready to wield these tools.

However, there are some areas in the prototype that can benefit from additional clarifications, UI tweaks, and workflow considerations. The details on user recommendations can be found in the research report.

次のステップ

Discuss existing issues and corresponding recommendations for improvement.

If possible, integrate changes into beta feature to allow time for users to experience them. After period of use, users of the beta should be prompted for information on how much the feature was utilized day-to-day, what specific elements/functionalities they used, and their feedback and recommendations for how the beta has worked for them and where areas for improvement remain.

研究報告書

調査結果の説明。

「最近の変更」用フィルタ：使いやすさ 第3期及びベータ版拡張機能満足度調査 2017年7月-8月

調査項目

The new filters for edit review have been out as a beta feature on various wikis. Our goal is to learn from users directly how the experience of using the new filters over a period of time has been for them.

Since the initial rounds of generative and usability testing, the team made some tweaks from past usability testing and feature requests that were then applied to a new prototype, named ‘integrated’ filters, along with a set of filters made specifically for the functionality on watchlist. In this round of testing, we evaluated the usability of the latest iteration, the applicability of its watchlist mirror, and the live updates feature in conjunction with the beta satisfaction testing.

ベータ版「最近の更新」の利用者満足度評価：

• What do users think of the filters? Have their opinions changed over time?
• Have they put the filters to use? If so, how? What are some of these uses?
• Are the filters supporting their existing activities and workflows?
• Are they creating new ‘existing activities/workflows’ as a result of having the additional capabilities?
• How do they feel about the beta Recent Changes vs current Recent Changes?
• More specifically, how do the new set of filters compare with the current set?
• What does the page need to contain for it to reach peak effectiveness for the users?

フィルタの統合/「最終更新」との使いやすさ：

• How are discoverability of the filters and general navigation for users? Overall, has there been improvement?
• What reactions do users have to the live updates button and pagination options?
• Do users know about / recall / use the resources listed in the related links section?

ウォッチリストの動作と使いやすさ：

• What actions do users typically take on their watchlists?
• What filters would be useful? How do they feel about the watchlist filter set on the prototype? What filters are still missing?

• Is the purpose of the watchlist-related filters provided in the Watchlist page understood, discoverable, and useful for watchlist users.
• Do users create the same filter sets in the Watchlist they would for Recent Changes?
• Are users able to identify which pages require their attention (are not visited) at a glance and why the distinction is made? Is it clear for users how to mark some/all of them as visited?

この調査プロジェクトの詳細はこちら。

結果

新フィルタのベータ版と利用者満足度：

Overall, the users feel that the beta feature provides a cleaner interface and a better experience than the current RC page. Though not without caveats; there are user suggestions for improvement, and adoption and use of the Content Quality and User Intent filter categories does not appear widespread. Improvements should be made to clarify the filters’ function, means of usage, and reliability, and other user concerns and suggestions should be evaluated and addressed.

The primary applications of the filters are related to vandalism patrol, and users are generally able to achieve their intended goals using the beta page.

フィルタの統合/「最終更新」と使いやすさ：

Discoverability, general navigation, pagination

• Testing revealed no issues involving the discoverability of the dropdown menu
• Though participants successfully complete tasks involving ‘advanced’ filters (users, tags, namespaces, categories), their initial discoverability on the page is still lacking
• Pagination does not appear to cause much confusion for users even when testing an atypical use case (sorting oldest first), although most do not process the need for that functionality on the page.

最終更新

• All 5 participants had overall positive reactions to the introduction of this feature, with some minor concerns and suggestions for improvement.

参考リンク

• All 5 participants use the options provided in the related links to some degree.
• Mixed reaction as to whether the section should be hidden/collapsible.

ウォッチリストの動作と使いやすさ：

Among this cohort, the number of articles in their watchlists varied (along with the reasons for adding them) but their use of watchlist is rather consistent; they don’t use the page much if at all and if they do, it is not rigorous and more of a casual scanning for new activity. The recruiting of this cohort, however, was based on their use of Recent Changes page, so they may not be representative of the opinions of heavy watchlist users.

A majority of participants indicate that they do not and would not use filters on their watchlist page, consistent with responses when presented with the hypothetical posed during beta satisfaction testing.

All participants understood the bold/filled bullet represented a page with new activity/not yet visited, and it's an expectation that clicking on any part of the entry (diff, hist, or article link) should mark the entry as ‘read’.

次のステップ

Discuss existing issues and corresponding recommendations for improvement (this will always be a recurring element of our work).

Discuss the utility of further examining the idea of bringing the edit review filters to pages like watchlist, but with more seasoned/heavier users of watchlist pages specifically.

研究報告書

調査結果の説明。