Article feedback/Version 5/Technical Design

From MediaWiki.org
Jump to: navigation, search

This page describes technical designs for the Article Feedback Tool Version 5 (AFT V5).

See also: feature requirements page, project overview page, useful links, as well as data and metrics plan.

Contents

Overview [edit]

The front end of the AFTv5 extension follows the ArticleFeedback extension: on article pages, a set of javascript modules attach a new div to the bottom of the page and fill it with a feedback form. On submission, the form is replaced by a CTA (call to action). Most of this code has been rewritten to allow for the selection of one of a set of possible forms, and one of a set of possible CTAs. Right now, form options 1, 2, and 3 are available, and the edit CTA is the primary choice (falling back to the learn more CTA if the current user cannot edit the article in question).

The back end is entirely new: rather than displaying aggregate data, the Special page shows a feed of the responses to the article passed in to the url. Right now the back end is not available in production; however, it is the first priority for the next phase. (For now, aggregate data from production is available via Dario's tools.)

Database schema [edit]

AFTv5 Schema

More detailed information is available on the schema documentation page

There are three types of tables that compose AFTv5: data type tables (describing the available fields and options by form), feedback tables (one for posts, and one for the answers that compose it), and aggregate tables (keeping easily-accessible counts for the special page).

When a feedback form is submitted, a record is created in aft_article_feedback and also, depending on the feedback form, a number of rows may be added to aft_article_answer -- one row per answer.

Each answer is associated with a row in aft_article_field, which dictates the data type of the answer. Fields of type text have their answers stored in aa_response_text, for example. If the comment text is longer than 255 characters, the first 255 are stored in this field and the full text in aft_article_answer_text.

In the same transaction, each row in the appropriate aggregate tables is updated -- by rating or boolean value for aft_article_feedback_ratings_rollup, or by select option for aft_article_feedback_select_rollup. For each filter the new feedback post matches (both for its article and for the central feedback page), the row in aft_article_filter_count is updated.

Later, when editors and monitors perform various actions on the feedback, the associated filter counts are updated -- for example, hiding a comment would increment the hidden filter and decrement the visible filter.

Upgrading and Maintenance [edit]

As of June 5, 2012, AFT can be upgraded using maintenance/update.php. If you're currently using an earlier version of the code, and you don't mind losing data, you can manually delete the aft_* tables from your database. If you do mind losing data, you'll need to compare your schema to sql/ArticleFeedbackv5.sql. The sql/wmfupdate-*.sql files may be useful, depending on what version you have.

The upgrade hook is located in ArticleFeedbackv5.hooks.php, in the loadExtensionSchemaUpdates method. If you are bringing in new schema changes, you need to:

  1. Create a SQL upgrade file in the sql directory
  2. Add a $updater->addExtensionUpdate call to loadExtensionSchemaUpdates indicating what to check for and what file to run if it's missing.

There are two maintenance scripts available:

  • maintenance/refreshFilterCounts.php resets the filter counts on the special page
  • maintenance/refreshLastActivity.php pulls the activity on all feedback from the log and sets the af_last_*, af_activity_count, and af_suppress_count columns

There are also a few plain-sql scripts:

  • sql/filter_count.sql is identical to the filter counts maintenance script, except that it doesn't wait for slaves
  • sql/fix_logs.sql moves any log entries that should have been suppressed from log type articlefeedbackv5 to log type suppress and updates the activity counts
  • sql/relevance_score.sql sets the relevance scores for all feedback (beware: untouched feedback gets a score of -50 rather than zero, as this script is intended as an initial setting, not an update)

Startup [edit]

AFT consists of three parts: a front-end form (displayed on pages in the main namespace), a link to the feedback page (displayed on talk pages), and the feedback page itself. Each of these can only appear when the main article falls within the configurable parameters:

  • Not disabled via user preferences (AFT appears if other checks pass)
  • Category blacklist (AFT never appears)
  • Category whitelist (AFT always appears)
  • Lottery (AFT appears if it's not in the category whitelist or blacklist, and the page ID falls within range allowed)

We also have to make sure that the user's browser is supported.

Ideally, we'd be performing all of these checks in the javascript, which has a reasonably short cache timeout compared to the back end, which only clears cache when the page is edited -- lower-traffic pages could take up to 30 days to clear. However, the talk and special namespaces don't have a way of retrieving the category information about their associated article from the javascript, so we must perform that part of the check in the back end.

The back end startup check is located in ArticleFeedbackv5.hooks.php, in the beforePageDisplay method. It finds the associated main article (talk/special), makes sure that it's a regular view page (not edit, printable, or old revision), and loads the appropriate javascript:

  • Articles:
    • Sets aftv5Whitelist to true if the category was whitelisted
    • Loads module ext.articleFeedbackv5.startup
  • Talk pages:
    • Sets aftv5Whitelist to true if the category was whitelisted
    • Sets aftv5PageId to the associated article's page ID
    • Loads module ext.articleFeedbackv5.talk
  • Special page:
    • Sets aftv5Whitelist to true if the category was whitelisted
    • Sets aftv5PageId to the associated article's page ID (or zero for the central feedback page
    • Loads module ext.articleFeedbackv5.dashboard

NB: The front end startup check has been refactored as of https://gerrit.wikimedia.org/r/13549 into modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.verify.js; before that, it was repeated in each file.

  1. Check to see if aftv5Whitelist is present.
  2. If not, and it's an article page, run the whole check on the js side
  3. If it's present and false, run the lottery check
  4. If all this has resolved to AFT being enabled, check the user agent

On the article and talk pages, AFT being not enabled will just stop the relevant javascript from loading and the page will appear as normal. On the special page, the entire contents are removed and replaced with an error message saying that AFT is not enabled for the requested page -- or, if it was just the user agent check that failed, the error message will say that.

Front End [edit]

If startup is successful, the primary module (ext.articleFeedbackv5.js) is loaded. The primary module attaches a new div to the bottom of the page and invokes the jQuery plugin (jquery.articleFeedbackv5.js) on it -- if the user scrolls to the bottom of the page, they'll see the form there; if they click the toolbox link or one of the optional prominent links, they'll see it in a overlay window. The particular form the user sees, and the presence/location of the optional link, are chosen via bucketing.

If the user hits the submit button, it runs an error check and, if there are no issues, invokes an AJAX request. Upon successful return, the form is replaced with a CTA (call to action).

Flow [edit]

  1. Startup phase (modules/ext.articleFeedbackv5/ext.articleFeedbackv5.startup.js)
    Determines whether to display the tool:
    • the user agent is supported
    • the page is an article
    • the category is whitelisted/blacklisted
    If all is well, it loads the main module.
  2. Main phase (modules/ext.articleFeedbackv5/ext.articleFeedbackv5.js)
    Creates a new div, inserts it at the bottom of the article, and invokes the jQuery plugin on it
    Selects a trigger link option and adds it to the page (if any)
    * NB: The click event calls the plugin's open-as-modal event
  3. Init phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.init)
    Sets some state variables, chooses a display option, and binds the "appear" event to the holder div
  4. Load phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.load)
    Sets up the bottom of the page and overlay containers, builds the selected form, and puts it in the appropriate container
  5. Submit phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm)
    Runs validation, locks the form, and sends off an ajax request
  6. Submit response phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm)
    On success, selects a CTA and loads it into the appropriate container, and removes the feedback link(s)
    On error, sets an error state and unlocks the form
  7. CTA close (when displayed in modal)
    If the user closes the CTA, it is entirely removed from the page and cannot be retrieved.

Query string options [edit]

You can choose a form and/or a trigger link option and avoid bucketing by passing the following in the url:

URL variable Possible values Values active on enwiki What it does
aftv5_link A, B, C, D, E, F, G, H, or X X Show a particular trigger link
aftv5_form 0, 1, 4, or 6 6 Show a particular form
aftv5_cta 0, 1, 2, 3 , 4, 5 or 6 4, 5, or 6 Show a particular cta after the form is submitted

Form Bucketing Process [edit]

Bucketing mechanism flowchart

The decision of what form to display works like this:

  1. If the plugin is in the debug state, and a form has been requested in the url, and that form is known, select it.
  2. If there's a form-bucket cookie set, select it.
  3. Otherwise, call mw.user.bucket.

The decision of what link to display works like this:

  1. If the display bucket is 0 or 5, select - (nothing).
  2. If the bucketing object is missing from the config, select -.
  3. If the plugin is in the debug state, and a link has been requested in the url, and that link is known, select it.
  4. Otherwise, call mw.user.bucket.

Note that the form selection doesn't check for a missing bucketing config object -- it should - TODO.

Submit AJAX call [edit]

This call is made when the user submits any of the feedback forms.

Call can be found at:
modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm

Response can be found at:
api/ApiArticleFeedbackv5.php

Parameters:

Name Type Description
pageid Integer Page ID to submit feedback for
revid Integer Revision ID to submit feedback for
anontoken String Token for anonymous users
bucket Integer Which form was shown to the user
experiment String The name of the current experiment
link String Which link, if any, the user clicked on to get to the widget (-, A through H, and TBX)
found Boolean Whether the user found what they were looking for (Form option 1 and 6)
comment String The text the user entered (Form options 1, 2, 3 and 6)
tag Integer The type of comment entered, matching an id from aft_article_field_option (Form option 2)
rating Integer The helpfulness rating, on a scale of 1 to 5 (Form option 3)
trustworthy Integer The trustworthiness rating, on a scale of 1 to 5 (Form option 5)
objective Integer The objectivity rating, on a scale of 1 to 5 (Form option 5)
complete Integer The completeness rating, on a scale of 1 to 5 (Form option 5)
wellwritten Integer The well-written rating, on a scale of 1 to 5 (Form option 5)
expertise-general Boolean Whether the user self-identified as an expert (Form option 5)
expertise-studies Boolean Whether the user said their expertise came from academic work (Form option 5)
expertise-profession Boolean Whether the user said their expertise came from professional work (Form option 5)
expertise-hobby Boolean Whether the user said their expertise came from a hobby (Form option 5)
expertise-other Boolean Whether the user said their expertise came from something else (Form option 5)
email String The user's email address, if they want to be contacted (Form option 5)

Success response:
JSON object with one key, articlefeedbackv5, containing this object:

Key Type Description
result String Always "Success"
feedback_id Integer The id of the new row in aft_article_feedback
cta_id Integer The index of the selected CTA
aft_url String The link to /Special:ArticleFeedbackv5 - for use in CTA5
permalink String The parmalink URL to this feedback entry

Error response:
JSON object with one key, error, containing the token for the appropriate error message

Abuse Filtering [edit]

AFTv5 abuse filtering

AFT makes use of $wgSpamRegex, SpamBlacklist, and AbuseFilter. You can turn abuse filtering on by setting $wgArticleFeedbackv5AbuseFiltering to true. If any of these three are available they will be used to filter out abusive feedback. See flowchart for details.

Actions [edit]

AFT makes use of AbuseFilter's "disallow" and "warn" actions, and it provides custom actions to automatically flag incoming feedback:

Action Source Description Message Key Implemented
disallow AbuseFilter Prevents the posting of feedback articlefeedbackv5-error-abuse Yes
warn AbuseFilter Shows a message on your first attempt, but proceeds with other actions on your second attempt Any key you set on the rule, but abusefilter-warning-feedback is available Yes
aftv5flagabuse AFTv5 Flags the incoming feedback as abuse none Yes, but commented out for 4/4 release
aftv5hide AFTv5 Hides the incoming feedback none Yes, but commented out for 4/4 release
aftv5requestoversight AFTv5 Requests oversight on the incoming feedback none Yes, but commented out for 4/4 release

Creating Filters [edit]

AFT makes use of AbuseFilter's new "group" feature -- any filters set to "feedback" will be processed; other ones will be ignored. (If you're using an older version of AbuseFilter without the group feature, all filters will be processed.)

When you set up filters for AFT, you should:

  1. Prefix the filter name with "Feedback: "
  2. Set "feedback" as the group
  3. Write conditions approprite for feedback
  4. Use one of the action options above

Writing filters is just as risky for feedback as it is for edits. You should read through the AbuseFilter conditions documentation carefully before proceeding.

You will have access to the following variables when writing conditions:

Variable Description
summary constant, "Article Feedback 5"
action constant, "feedback"
context constant, "filter"
timestamp time the feedback was posted
new_wikitext comment text
new_size comment length
user_editcount posting user's edit count
user_name posting user's name
user_emailconfirm posting user's email confirmation timestamp
user_age posting user's account age
user_groups posting user's groups
article_articleid id of the article posted to
article_namespace namespace of the article posted to
article_text title text of the article posted to
article_prefixedtext prefixed title text of the article posted to
article_restrictions_create, _edit, _move, _upload any restrictions on the article posted to (names of groups allowed)
article_recent_contributors last ten editors of the article posted to

If you're adapting an edit filter to work with feedback, you should change conditions checking added_lines to use new_wikitext. You can remove any conditions related to the previous state of the article (e.g., removed_lines).

You can test these filters just as you would the edit filters, by watching the AbuseFilter log.

Special Page [edit]

Feedback list [edit]

The feedback list is loaded via an AJAX call, using the $.articleFeedbackv5special.loadFeedback function. This provides for paginated (incremental) display of feedback without the need to reload the entire page.

The $.articleFeedbackv5special.pullHighlight function pulls the highlighted post(s) separately.

Feedback post actions [edit]

Implementation of actions for the special page revolves around the $.articleFeedbackv5special.actions array of objects. Each array element represents an action, keyed by the action name, and contains an object with the following structure:

  • hasTipsy - true if the action needs a flyover panel*tipsyHtml - html for the corresponding flyover panel
  • click - click action
  • apiFlagType - flag type for api call
  • apiFlagDir - flag direction for api call (+/-1)
  • onSuccess - callback to execute after action success. Callback parameters:
  • id - respective post id
  • data - any data returned by the AJAX call

The following actions are implemented:

JS object key Flagging class name Description Increment filters Decrement filters Relevance change
helpful helpful, increase flag post as helpful visible-helpful visible-unhelpful +1
reversehelpful helpful, decrease undo flag post as helpful visible-helpful visible-unhelpful -1
unhelpful unhelpful, increase flag post as unhelpful visible-helpful visible-unhelpful -1
reverseunhelpful unhelpful, decrease undo flag post as unhelpful visible-helpful visible-unhelpful +1
abuse abuse, increase flag post as abuse visible-abusive, hidden filters if auto-hidden visible filters if auto-hidden -5
unabuse abuse, decrease undo flag post as abuse visible filters if auto-unhidden visible-abusive, hidden filters if auto-unhidden +5
feature feature, increase feature post visible-featured visible-unfeatured if previously unfeatured +50
unfeature feature, decrease undo feature post visible-unfeatured visible-featured -50
resolve resolve, increase mark post as resolved visible-resolved visible-unresolved if previously unresolved -45
unresolve resolve, decrease undo mark post as resolved visible-unresolved visible-unresolved +45
hide hide, increase hide post hidden filters visible filters n/a
show hide, decrease undo hide post visible hidden filters n/a
requestoversight oversight, increase request oversighting of post all-requested, notdeleted-requested, hidden filters all-unrequested and notdeleted-unrequested if previously unrequested, visible filters n/a
unrequestoversight oversight, decrease undo request oversighting of post all-unrequested, notdeleted-unrequested all-requested, notdeleted-requested n/a
oversight delete, increase oversight post all-oversighted, hidden filters notdeleted, and as applicable: notdeleted-hidden, notdeleted-unhidden, notdeleted-requested, notdeleted-unrequested, notdeleted-declined, all-unoversighted, visible filters n/a
unoversight delete, decrease undo post oversight all-unoversighted, and as applicable: notdeleted-hidden, notdeleted-unhidden, notdeleted-requested, notdeleted-unrequested, notdeleted-declined, visible filters all-oversighted n/a
declineoversight resetoversight decline oversight request all-declined, notdeleted-declined all-requested, notdeleted-requested n/a
activity n/a view activity log n/a n/a n/a
activity2 n/a view activity log in permalink info section n/a n/a n/a
  • hidden filters are: all-hidden (increase), all-unhidden (decrease), notdeleted-hidden (increase), and notdeleted-unhidden (decrease)
  • visible filters are: visible (increase), visible-abusive (increase if applicable), visible-comment (increase if applicable), visible-featured (increase if applicable), visible-helpful (increase if applicable), visible-relevant (increase if applicable), visible-resolved (increase if applicable), visible-unfeatured (increase if applicable), visible-unhelpful (increase if applicable), and visible-unresolved (increase if applicable)

The actual AJAX call for flagging actions is performed by the $.articleFeedbackv5special.flagFeedback function. The function locks the flagging actions for its duration to avoid multiple simultaneous requests.

User activity tracking [edit]

The user activity on the special page is stored client-side, via cookies. This functionality provides some limitation on actions repetition by the user.

TODO

  1. Moderation tools - hide, promote (to talk page), or flag various pieces of feedback.
  2. Sorting and filtering options - good comments first (how that's defined is unclear), newest first, etc. Basically, tools are needed to manage or at least hide the inevitable flood of spam "feedback".
  3. A special page, on the level of the talk pages - add a link to this in the not-quite-top-level navigation on said talk page.
  4. Need to work out which bits of this page are must-have-now vs must-have-eventually.
  5. Phase 1 will have a limited version with:
    1. Filters: All, Visible only
    2. Moderation Tools: Hide this comment
    3. Sorting: By date only
    4. Pagination: Fixed at 50

Filters [edit]

The feedback page allows the user to see feedback grouped according to a set of filters. These are available according to permissions (e.g., to see hidden or deleted feedback), and each action performed may adjust the counts in aft_filter_counts:

Title Visible filter Not deleted filter All filter SQL
Most Relevant visible-relevant N/A N/A (af_has_comment IS TRUE OR af_net_helpfulness > 0 OR af_is_featured IS TRUE) AND af_relevance_score > $cutoff
All Comments visible-comment N/A N/A af_has_comment IS TRUE
Featured visible-featured N/A N/A af_is_featured IS TRUE
Helpful visible-helpful N/A N/A af_net_helpfulness > 0
All Visible visible N/A N/A N/A
Unhelpful visible-unhelpful N/A N/A af_net_helpfulness < 0
Flagged as Abusive visible-abusive N/A N/A af_abuse_count > 0
Un-featured visible-unfeatured N/A N/A af_is_unfeatured IS TRUE
Resolved visible-resolved N/A N/A af_is_resolved IS TRUE
Un-resolved visible-resolved N/A N/A af_is_unresolved IS TRUE
Hidden N/A notdeleted-hidden all-hidden af_is_hidden IS TRUE
All N/A notdeleted N/A N/A
Un-hidden N/A notdeleted-unhidden all-unhidden af_is_hidden IS TRUE
Oversight requested N/A notdeleted-requested all-requested af_oversight_count > 0
Oversight un-requested N/A notdeleted-unrequested all-unrequested af_is_unrequested IS TRUE
Oversight declined N/A notdeleted-declined all-declined af_is_declined IS TRUE
Oversighted N/A N/A all-oversighted af_is_deleted IS TRUE
Un-oversighted N/A N/A all-unoversighted af_is_undeleted IS TRUE
All (oversight) N/A N/A all N/A
  • Filters in the visible column also have af_is_deleted IS FALSE AND af_is_hidden IS FALSE appended the the where clause.
  • Filters in the not deleted column also have af_is_deleted IS FALSE appended the the where clause.
  • Filters in the all column have nothing appended the the where clause.

Relevance [edit]

Relevance scores are adjusted according to actions taken on the feedback. The weight of each action is configurable ($wgArticleFeedbackv5RelevanceScoring). The default values are:

Action Points change
vote helpful +1
vote unhelpful -1
feature +50
unfeature -50
mark resolved -45
unmark resolved +45
flag as abuse -5
unflag as abuse +5

Logging [edit]

Central activity log (https://en.wikipedia.org/w/index.php?title=Special%3ALog&type=articlefeedbackv5) calls upon class ArticleFeedbackv5LogFormatter to format the AFTv5 entries. This class extends from the default LogFormatter class and adds in some additional details to the entry (feedback text, page title & link, feedback n° & link)

The texts for all actions being logged are defined in ArticleFeedbackv5.php (as is the hook to the class that'll be formatting the log: ArticleFeedbackv5LogFormatter)

Note: We don't want some of the activity to show up, which is why certain actions are prefixed with suppress/ (see *core*/includes/logging/LogFormatter.php; "case 'suppress'") rather than articlefeedbackv5/. This way, we still have logs of all these actions, though not immediately visible.

The logging of all activity happens through a function call to ApiArticleFeedbackv5Utils::logActivity (ApiArticleFeedbackv5Utils.php) which will both insert the log entry, and increment the activity count on an entry (joining with the logging table is complex and not desired on such huge dataset).

The "view activity" link in the feedback page toolbox (both on central feedback page & permalink) also fetches its data from this logging table.

Database [edit]

ApiArticleFeedbackv5Utils::logActivity is the wrapper method to call when inserting log entries. This method performs some additional checks and will build the final data to be inserted.

logging table

Column Description
log_id auto-increment
log_type either 'articlefeedbackv5' or 'suppress', depending on whether or not we want the action to show up in central activity log/visible to all users
log_action the action being performed, e.g. 'feature', 'flag', ... (full list: maintenance/refreshLastActivity.php, $logActionToStatus)
log_timestamp self-explanatory, timestamp of the moment the action is logged - handled by core LogPage
log_user id of the user whose action is being logged (for automated actions, a special user 'articlefeedbackv5-default-user' is created) - handled by ApiArticleFeedbackv5Utils::logActivity based upon the User object tossed in
log_user_text name of the user whose action is being logged (for automated actions, a special user 'articlefeedbackv5-default-user' is created) - handled by ApiArticleFeedbackv5Utils::logActivity based upon the User object tossed in
log_namespace the article namespace of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_title the article title of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_page the article id of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_comment a (truncated) note attached to the action (when using fly-over panels that allow for notes, e.g. 'feature this post')
log_params so far, unused by AFTv5
log_deleted unused by AFTv5 - handled by core LogPage

aft_article_feedback table

Column Description
...
af_activity_count equivalent to counting an feedback entry's log type 'articlefeedbackv5' entries in the logging table - denormalized to improve performance/scaling
af_suppress_count equivalent to counting an feedback entry's log type 'suppress' (with log_action 'oversight', 'unoversight', 'decline', 'request', 'unrequest') entries in the logging table - denormalized to improve performance/scaling
...

Metrics / Clicktracking [edit]

AFTv5 uses the clicktracking extension to track how the extension is being used. All event names are prefixed with ext.articleFeedbackv5@{VERSION}, where {VERSION} is the value of $wgArticleFeedbackv5Tracking['version'].

Platforms [edit]

We aim to support the following web browsers for phases 1.0 and 1.5:

  • Internet Explorer 7+
  • Firefox 3+
  • Safari 5+
  • Opera 10+
  • Chrome 5+

NB: Support for IE7 was pulled from phase 1.0.

We will focus our testing on these top browser versions for Wikipedia:

  • IE 8 (17%)
  • Chrome 14 (16%)
  • Firefox 7 (11%)
  • IE 7 (7%)
  • IE 9 (6%)
  • Firefox 3 (5%)
  • Firefox 6 (2%)

These will be tested on these desktop platforms:

  • Windows (78%)
  • Mac (8%)
  • Linux (3%)

We are not currently planning to support IE6 or mobile platforms for phases 1.0 and 1.5, and will not show the forms at all on these unsupported platforms. In future versions, we will aim for 'graceful degradation' in unsupported platforms, and are working on a good definition for testing that objective.


Article Feedback Version 5 Specs: HubFeature RequirementsTechnical DesignTestingExtension Page
Metrics: Data and Metrics PlanClicktrackingVolume AnalysisQuality Assessment
Tools:Wikipedia AFTv5 CategoryWikipedia Data StreamOpen BugsRecent Code ChangesPDF Prototype
Community: MW talk pageIdeas LogWikipedia talk page