Article feedback/Version 5/Technical Design

This page describes technical designs for the Article Feedback Tool Version 5 (AFT V5).

See also: feature requirements page, project overview page, useful links, as well as data and metrics plan.

Overview

The front end of the AFTv5 extension follows the ArticleFeedback extension: on article pages, a set of javascript modules attach a new div to the bottom of the page and fill it with a feedback form. On submission, the form is replaced by a CTA (call to action). Most of this code has been rewritten to allow for the selection of one of a set of possible forms, and one of a set of possible CTAs. Right now, form options 1, 2, and 3 are available, and the edit CTA is the primary choice (falling back to the learn more CTA if the current user cannot edit the article in question).

The back end is entirely new: rather than displaying aggregate data, the Special page shows a feed of the responses to the article passed in to the url. Right now the back end is not available in production; however, it is the first priority for the next phase. (For now, aggregate data from production is available via Dario's tools.)

Database schema

AFTv5 Schema

More detailed information is available on the schema documentation page

There are three types of tables that compose AFTv5: data type tables (describing the available fields and options by form), feedback tables (one for posts, and one for the answers that compose it), and aggregate tables (keeping easily-accessible counts for the special page).

When a feedback form is submitted, a record is created in aft_article_feedback and also, depending on the feedback form, a number of rows may be added to aft_article_answer -- one row per answer.

Each answer is associated with a row in aft_article_field, which dictates the data type of the answer. Fields of type text have their answers stored in aa_response_text, for example. If the comment text is longer than 255 characters, the first 255 are stored in this field and the full text in aft_article_answer_text.

In the same transaction, each row in the appropriate aggregate tables is updated -- by rating or boolean value for aft_article_feedback_ratings_rollup, or by select option for aft_article_feedback_select_rollup. For each filter the new feedback post matches (both for its article and for the central feedback page), the row in aft_article_filter_count is updated.

Later, when editors and monitors perform various actions on the feedback, the associated filter counts are updated -- for example, hiding a comment would increment the hidden filter and decrement the visible filter.

Upgrading and Maintenance

As of June 5, 2012, AFT can be upgraded using maintenance/update.php. If you're currently using an earlier version of the code, and you don't mind losing data, you can manually delete the aft_* tables from your database. If you do mind losing data, you'll need to compare your schema to sql/ArticleFeedbackv5.sql. The sql/wmfupdate-*.sql files may be useful, depending on what version you have.

The upgrade hook is located in ArticleFeedbackv5.hooks.php, in the loadExtensionSchemaUpdates method. If you are bringing in new schema changes, you need to:

1. Create a SQL upgrade file in the sql directory
2. Add a $updater->addExtensionUpdate call to loadExtensionSchemaUpdates indicating what to check for and what file to run if it's missing.

There are two maintenance scripts available:

• maintenance/refreshFilterCounts.php resets the filter counts on the special page
• maintenance/refreshLastActivity.php pulls the activity on all feedback from the log and sets the af_last_*, af_activity_count, and af_suppress_count columns

There are also a few plain-sql scripts:

• sql/filter_count.sql is identical to the filter counts maintenance script, except that it doesn't wait for slaves
• sql/fix_logs.sql moves any log entries that should have been suppressed from log type articlefeedbackv5 to log type suppress and updates the activity counts
• sql/relevance_score.sql sets the relevance scores for all feedback (beware: untouched feedback gets a score of -50 rather than zero, as this script is intended as an initial setting, not an update)

Startup

AFT consists of three parts: a front-end form (displayed on pages in the main namespace), a link to the feedback page (displayed on talk pages), and the feedback page itself. Each of these can only appear when the main article falls within the configurable parameters:

• Not disabled via user preferences (AFT appears if other checks pass)
• Category blacklist (AFT never appears)
• Category whitelist (AFT always appears)
• Lottery (AFT appears if it's not in the category whitelist or blacklist, and the page ID falls within range allowed)

We also have to make sure that the user's browser is supported.

Ideally, we'd be performing all of these checks in the javascript, which has a reasonably short cache timeout compared to the back end, which only clears cache when the page is edited -- lower-traffic pages could take up to 30 days to clear. However, the talk and special namespaces don't have a way of retrieving the category information about their associated article from the javascript, so we must perform that part of the check in the back end.

The back end startup check is located in ArticleFeedbackv5.hooks.php, in the beforePageDisplay method. It finds the associated main article (talk/special), makes sure that it's a regular view page (not edit, printable, or old revision), and loads the appropriate javascript:

• Articles:
  • Sets aftv5Whitelist to true if the category was whitelisted
  • Loads module ext.articleFeedbackv5.startup
• Talk pages:
  • Sets aftv5Whitelist to true if the category was whitelisted
  • Sets aftv5PageId to the associated article's page ID
  • Loads module ext.articleFeedbackv5.talk
• Special page:
  • Sets aftv5Whitelist to true if the category was whitelisted
  • Sets aftv5PageId to the associated article's page ID (or zero for the central feedback page
  • Loads module ext.articleFeedbackv5.dashboard

NB: The front end startup check has been refactored as of https://gerrit.wikimedia.org/r/13549 into modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.verify.js; before that, it was repeated in each file.

1. Check to see if aftv5Whitelist is present.
2. If not, and it's an article page, run the whole check on the js side
3. If it's present and false, run the lottery check
4. If all this has resolved to AFT being enabled, check the user agent

On the article and talk pages, AFT being not enabled will just stop the relevant javascript from loading and the page will appear as normal. On the special page, the entire contents are removed and replaced with an error message saying that AFT is not enabled for the requested page -- or, if it was just the user agent check that failed, the error message will say that.

Front End

If startup is successful, the primary module (ext.articleFeedbackv5.js) is loaded. The primary module attaches a new div to the bottom of the page and invokes the jQuery plugin (jquery.articleFeedbackv5.js) on it -- if the user scrolls to the bottom of the page, they'll see the form there; if they click the toolbox link or one of the optional prominent links, they'll see it in a overlay window. The particular form the user sees, and the presence/location of the optional link, are chosen via bucketing.

If the user hits the submit button, it runs an error check and, if there are no issues, invokes an AJAX request. Upon successful return, the form is replaced with a CTA (call to action).

Flow

1. Startup phase (modules/ext.articleFeedbackv5/ext.articleFeedbackv5.startup.js)

   Determines whether to display the tool:

   • the user agent is supported
   • the page is an article
   • the category is whitelisted/blacklisted

   If all is well, it loads the main module.

2. Main phase (modules/ext.articleFeedbackv5/ext.articleFeedbackv5.js)

   Creates a new div, inserts it at the bottom of the article, and invokes the jQuery plugin on it

   Selects a trigger link option and adds it to the page (if any)

   * NB: The click event calls the plugin's open-as-modal event

3. Init phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.init)

   Sets some state variables, chooses a display option, and binds the "appear" event to the holder div

4. Load phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.load)

   Sets up the bottom of the page and overlay containers, builds the selected form, and puts it in the appropriate container

5. Submit phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm)

   Runs validation, locks the form, and sends off an ajax request

6. Submit response phase (modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm)

   On success, selects a CTA and loads it into the appropriate container, and removes the feedback link(s)

   On error, sets an error state and unlocks the form

7. CTA close (when displayed in modal)

   If the user closes the CTA, it is entirely removed from the page and cannot be retrieved.

Query string options

You can choose a form and/or a trigger link option and avoid bucketing by passing the following in the url:

URL variable	Possible values	Values active on enwiki	What it does
aftv5_link	A, B, C, D, E, F, G, H, or X	X	Show a particular trigger link
aftv5_form	0, 1, 4, or 6	6	Show a particular form
aftv5_cta	0, 1, 2, 3 , 4, 5 or 6	4, 5, or 6	Show a particular cta after the form is submitted

Form Bucketing Process

Bucketing mechanism flowchart

The decision of what form to display works like this:

1. If the plugin is in the debug state, and a form has been requested in the url, and that form is known, select it.
2. If there's a form-bucket cookie set, select it.
3. Otherwise, call mw.user.bucket.

The decision of what link to display works like this:

1. If the display bucket is 0 or 5, select - (nothing).
2. If the bucketing object is missing from the config, select -.
3. If the plugin is in the debug state, and a link has been requested in the url, and that link is known, select it.
4. Otherwise, call mw.user.bucket.

Note that the form selection doesn't check for a missing bucketing config object -- it should - TODO.

Submit AJAX call

This call is made when the user submits any of the feedback forms.

Call can be found at:
modules/jquery.articleFeedbackv5/jquery.articleFeedbackv5.js, method $.articleFeedbackv5.submitForm

Response can be found at:
api/ApiArticleFeedbackv5.php

Parameters:

Name	Type	Description
pageid	Integer	Page ID to submit feedback for
revid	Integer	Revision ID to submit feedback for
anontoken	String	Token for anonymous users
bucket	Integer	Which form was shown to the user
experiment	String	The name of the current experiment
link	String	Which link, if any, the user clicked on to get to the widget (-, A through H, and TBX)
found	Boolean	Whether the user found what they were looking for (Form option 1 and 6)
comment	String	The text the user entered (Form options 1, 2, 3 and 6)
tag	Integer	The type of comment entered, matching an id from aft_article_field_option (Form option 2)
rating	Integer	The helpfulness rating, on a scale of 1 to 5 (Form option 3)
trustworthy	Integer	The trustworthiness rating, on a scale of 1 to 5 (Form option 5)
objective	Integer	The objectivity rating, on a scale of 1 to 5 (Form option 5)
complete	Integer	The completeness rating, on a scale of 1 to 5 (Form option 5)
wellwritten	Integer	The well-written rating, on a scale of 1 to 5 (Form option 5)
expertise-general	Boolean	Whether the user self-identified as an expert (Form option 5)
expertise-studies	Boolean	Whether the user said their expertise came from academic work (Form option 5)
expertise-profession	Boolean	Whether the user said their expertise came from professional work (Form option 5)
expertise-hobby	Boolean	Whether the user said their expertise came from a hobby (Form option 5)
expertise-other	Boolean	Whether the user said their expertise came from something else (Form option 5)
email	String	The user's email address, if they want to be contacted (Form option 5)

Success response:
JSON object with one key, articlefeedbackv5, containing this object:

Key	Type	Description
result	String	Always "Success"
feedback_id	Integer	The id of the new row in aft_article_feedback
cta_id	Integer	The index of the selected CTA
aft_url	String	The link to /Special:ArticleFeedbackv5 - for use in CTA5
permalink	String	The parmalink URL to this feedback entry

Error response:
JSON object with one key, error, containing the token for the appropriate error message

Abuse Filtering

AFTv5 abuse filtering

AFT makes use of $wgSpamRegex, SpamBlacklist, and AbuseFilter. You can turn abuse filtering on by setting $wgArticleFeedbackv5AbuseFiltering to true. If any of these three are available they will be used to filter out abusive feedback. See flowchart for details.

Actions

AFT makes use of AbuseFilter's "disallow" and "warn" actions, and it provides custom actions to automatically flag incoming feedback:

Action	Source	Description	Message Key	Implemented
disallow	AbuseFilter	Prevents the posting of feedback	articlefeedbackv5-error-abuse	Yes
warn	AbuseFilter	Shows a message on your first attempt, but proceeds with other actions on your second attempt	Any key you set on the rule, but abusefilter-warning-feedback is available	Yes
aftv5flagabuse	AFTv5	Flags the incoming feedback as abuse	none	Yes, but commented out for 4/4 release
aftv5hide	AFTv5	Hides the incoming feedback	none	Yes, but commented out for 4/4 release
aftv5requestoversight	AFTv5	Requests oversight on the incoming feedback	none	Yes, but commented out for 4/4 release

Creating Filters

AFT makes use of AbuseFilter's new "group" feature -- any filters set to "feedback" will be processed; other ones will be ignored. (If you're using an older version of AbuseFilter without the group feature, all filters will be processed.)

When you set up filters for AFT, you should:

1. Prefix the filter name with "Feedback: "
2. Set "feedback" as the group
3. Write conditions approprite for feedback
4. Use one of the action options above

Writing filters is just as risky for feedback as it is for edits. You should read through the AbuseFilter conditions documentation carefully before proceeding.

You will have access to the following variables when writing conditions:

Variable	Description
summary	constant, "Article Feedback 5"
action	constant, "feedback"
context	constant, "filter"
timestamp	time the feedback was posted
new_wikitext	comment text
new_size	comment length
user_editcount	posting user's edit count
user_name	posting user's name
user_emailconfirm	posting user's email confirmation timestamp
user_age	posting user's account age
user_groups	posting user's groups
article_articleid	id of the article posted to
article_namespace	namespace of the article posted to
article_text	title text of the article posted to
article_prefixedtext	prefixed title text of the article posted to
article_restrictions_create, _edit, _move, _upload	any restrictions on the article posted to (names of groups allowed)
article_recent_contributors	last ten editors of the article posted to

If you're adapting an edit filter to work with feedback, you should change conditions checking added_lines to use new_wikitext. You can remove any conditions related to the previous state of the article (e.g., removed_lines).

You can test these filters just as you would the edit filters, by watching the AbuseFilter log.

Special Page

Feedback list

The feedback list is loaded via an AJAX call, using the $.articleFeedbackv5special.loadFeedback function. This provides for paginated (incremental) display of feedback without the need to reload the entire page.

The $.articleFeedbackv5special.pullHighlight function pulls the highlighted post(s) separately.

Feedback post actions

Implementation of actions for the special page revolves around the $.articleFeedbackv5special.actions array of objects. Each array element represents an action, keyed by the action name, and contains an object with the following structure:

• hasTipsy - true if the action needs a flyover panel*tipsyHtml - html for the corresponding flyover panel
• click - click action
• apiFlagType - flag type for api call
• apiFlagDir - flag direction for api call (+/-1)
• onSuccess - callback to execute after action success. Callback parameters:
• id - respective post id
• data - any data returned by the AJAX call

The following actions are implemented:

JS object key	Flagging class name	Description	Increment filters	Decrement filters	Relevance change
helpful	helpful, increase	flag post as helpful	visible-helpful	visible-unhelpful	+1
reversehelpful	helpful, decrease	undo flag post as helpful	visible-helpful	visible-unhelpful	-1
unhelpful	unhelpful, increase	flag post as unhelpful	visible-helpful	visible-unhelpful	-1
reverseunhelpful	unhelpful, decrease	undo flag post as unhelpful	visible-helpful	visible-unhelpful	+1
abuse	abuse, increase	flag post as abuse	visible-abusive, hidden filters if auto-hidden	visible filters if auto-hidden	-5
unabuse	abuse, decrease	undo flag post as abuse	visible filters if auto-unhidden	visible-abusive, hidden filters if auto-unhidden	+5
feature	feature, increase	feature post	visible-featured	visible-unfeatured if previously unfeatured	+50
unfeature	feature, decrease	undo feature post	visible-unfeatured	visible-featured	-50
resolve	resolve, increase	mark post as resolved	visible-resolved	visible-unresolved if previously unresolved	-45
unresolve	resolve, decrease	undo mark post as resolved	visible-unresolved	visible-unresolved	+45
hide	hide, increase	hide post	hidden filters	visible filters	n/a
show	hide, decrease	undo hide post	visible	hidden filters	n/a
requestoversight	oversight, increase	request oversighting of post	all-requested, notdeleted-requested, hidden filters	all-unrequested and notdeleted-unrequested if previously unrequested, visible filters	n/a
unrequestoversight	oversight, decrease	undo request oversighting of post	all-unrequested, notdeleted-unrequested	all-requested, notdeleted-requested	n/a
oversight	delete, increase	oversight post	all-oversighted, hidden filters	notdeleted, and as applicable: notdeleted-hidden, notdeleted-unhidden, notdeleted-requested, notdeleted-unrequested, notdeleted-declined, all-unoversighted, visible filters	n/a
unoversight	delete, decrease	undo post oversight	all-unoversighted, and as applicable: notdeleted-hidden, notdeleted-unhidden, notdeleted-requested, notdeleted-unrequested, notdeleted-declined, visible filters	all-oversighted	n/a
declineoversight	resetoversight	decline oversight request	all-declined, notdeleted-declined	all-requested, notdeleted-requested	n/a
activity	n/a	view activity log	n/a	n/a	n/a
activity2	n/a	view activity log in permalink info section	n/a	n/a	n/a

• hidden filters are: all-hidden (increase), all-unhidden (decrease), notdeleted-hidden (increase), and notdeleted-unhidden (decrease)
• visible filters are: visible (increase), visible-abusive (increase if applicable), visible-comment (increase if applicable), visible-featured (increase if applicable), visible-helpful (increase if applicable), visible-relevant (increase if applicable), visible-resolved (increase if applicable), visible-unfeatured (increase if applicable), visible-unhelpful (increase if applicable), and visible-unresolved (increase if applicable)

The actual AJAX call for flagging actions is performed by the $.articleFeedbackv5special.flagFeedback function. The function locks the flagging actions for its duration to avoid multiple simultaneous requests.

User activity tracking

The user activity on the special page is stored client-side, via cookies. This functionality provides some limitation on actions repetition by the user.

TODO

1. Moderation tools - hide, promote (to talk page), or flag various pieces of feedback.
2. Sorting and filtering options - good comments first (how that's defined is unclear), newest first, etc. Basically, tools are needed to manage or at least hide the inevitable flood of spam "feedback".
3. A special page, on the level of the talk pages - add a link to this in the not-quite-top-level navigation on said talk page.
4. Need to work out which bits of this page are must-have-now vs must-have-eventually.
5. Phase 1 will have a limited version with:
   1. Filters: All, Visible only
   2. Moderation Tools: Hide this comment
   3. Sorting: By date only
   4. Pagination: Fixed at 50

Filters

The feedback page allows the user to see feedback grouped according to a set of filters. These are available according to permissions (e.g., to see hidden or deleted feedback), and each action performed may adjust the counts in aft_filter_counts:

Title	Visible filter	Not deleted filter	All filter	SQL
Most Relevant	visible-relevant	N/A	N/A	(af_has_comment IS TRUE OR af_net_helpfulness > 0 OR af_is_featured IS TRUE) AND af_relevance_score > $cutoff
All Comments	visible-comment	N/A	N/A	af_has_comment IS TRUE
Featured	visible-featured	N/A	N/A	af_is_featured IS TRUE
Helpful	visible-helpful	N/A	N/A	af_net_helpfulness > 0
All Visible	visible	N/A	N/A	N/A
Unhelpful	visible-unhelpful	N/A	N/A	af_net_helpfulness < 0
Flagged as Abusive	visible-abusive	N/A	N/A	af_abuse_count > 0
Un-featured	visible-unfeatured	N/A	N/A	af_is_unfeatured IS TRUE
Resolved	visible-resolved	N/A	N/A	af_is_resolved IS TRUE
Un-resolved	visible-resolved	N/A	N/A	af_is_unresolved IS TRUE
Hidden	N/A	notdeleted-hidden	all-hidden	af_is_hidden IS TRUE
All	N/A	notdeleted	N/A	N/A
Un-hidden	N/A	notdeleted-unhidden	all-unhidden	af_is_hidden IS TRUE
Oversight requested	N/A	notdeleted-requested	all-requested	af_oversight_count > 0
Oversight un-requested	N/A	notdeleted-unrequested	all-unrequested	af_is_unrequested IS TRUE
Oversight declined	N/A	notdeleted-declined	all-declined	af_is_declined IS TRUE
Oversighted	N/A	N/A	all-oversighted	af_is_deleted IS TRUE
Un-oversighted	N/A	N/A	all-unoversighted	af_is_undeleted IS TRUE
All (oversight)	N/A	N/A	all	N/A

• Filters in the visible column also have af_is_deleted IS FALSE AND af_is_hidden IS FALSE appended the the where clause.
• Filters in the not deleted column also have af_is_deleted IS FALSE appended the the where clause.
• Filters in the all column have nothing appended the the where clause.

Relevance

Relevance scores are adjusted according to actions taken on the feedback. The weight of each action is configurable ($wgArticleFeedbackv5RelevanceScoring). The default values are:

Action	Points change
vote helpful	+1
vote unhelpful	-1
feature	+50
unfeature	-50
mark resolved	-45
unmark resolved	+45
flag as abuse	-5
unflag as abuse	+5

Logging

Central activity log (https://en.wikipedia.org/w/index.php?title=Special%3ALog&type=articlefeedbackv5) calls upon class ArticleFeedbackv5LogFormatter to format the AFTv5 entries. This class extends from the default LogFormatter class and adds in some additional details to the entry (feedback text, page title & link, feedback n° & link)

The texts for all actions being logged are defined in ArticleFeedbackv5.php (as is the hook to the class that'll be formatting the log: ArticleFeedbackv5LogFormatter)

Note: We don't want some of the activity to show up, which is why certain actions are prefixed with suppress/ (see *core*/includes/logging/LogFormatter.php; "case 'suppress'") rather than articlefeedbackv5/. This way, we still have logs of all these actions, though not immediately visible.

The logging of all activity happens through a function call to ApiArticleFeedbackv5Utils::logActivity (ApiArticleFeedbackv5Utils.php) which will both insert the log entry, and increment the activity count on an entry (joining with the logging table is complex and not desired on such huge dataset).

The "view activity" link in the feedback page toolbox (both on central feedback page & permalink) also fetches its data from this logging table.

Database

ApiArticleFeedbackv5Utils::logActivity is the wrapper method to call when inserting log entries. This method performs some additional checks and will build the final data to be inserted.

logging table

Column	Description
log_id	auto-increment
log_type	either 'articlefeedbackv5' or 'suppress', depending on whether or not we want the action to show up in central activity log/visible to all users
log_action	the action being performed, e.g. 'feature', 'flag', ... (full list: maintenance/refreshLastActivity.php, $logActionToStatus)
log_timestamp	self-explanatory, timestamp of the moment the action is logged - handled by core LogPage
log_user	id of the user whose action is being logged (for automated actions, a special user 'articlefeedbackv5-default-user' is created) - handled by ApiArticleFeedbackv5Utils::logActivity based upon the User object tossed in
log_user_text	name of the user whose action is being logged (for automated actions, a special user 'articlefeedbackv5-default-user' is created) - handled by ApiArticleFeedbackv5Utils::logActivity based upon the User object tossed in
log_namespace	the article namespace of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_title	the article title of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_page	the article id of the feedback permalink - handled by ApiArticleFeedbackv5Utils::logActivity based upon the Title object tossed in
log_comment	a (truncated) note attached to the action (when using fly-over panels that allow for notes, e.g. 'feature this post')
log_params	so far, unused by AFTv5
log_deleted	unused by AFTv5 - handled by core LogPage

aft_article_feedback table

Column	Description
...
af_activity_count	equivalent to counting an feedback entry's log type 'articlefeedbackv5' entries in the logging table - denormalized to improve performance/scaling
af_suppress_count	equivalent to counting an feedback entry's log type 'suppress' (with log_action 'oversight', 'unoversight', 'decline', 'request', 'unrequest') entries in the logging table - denormalized to improve performance/scaling
...

Metrics / Clicktracking

AFTv5 uses the clicktracking extension to track how the extension is being used. All event names are prefixed with ext.articleFeedbackv5@{VERSION}, where {VERSION} is the value of $wgArticleFeedbackv5Tracking['version'].

Platforms

We aim to support the following web browsers for phases 1.0 and 1.5:

• Internet Explorer 7+
• Firefox 3+
• Safari 5+
• Opera 10+
• Chrome 5+

NB: Support for IE7 was pulled from phase 1.0.

We will focus our testing on these top browser versions for Wikipedia:

• IE 8 (17%)
• Chrome 14 (16%)
• Firefox 7 (11%)
• IE 7 (7%)
• IE 9 (6%)
• Firefox 3 (5%)
• Firefox 6 (2%)

These will be tested on these desktop platforms:

• Windows (78%)
• Mac (8%)
• Linux (3%)

We are not currently planning to support IE6 or mobile platforms for phases 1.0 and 1.5, and will not show the forms at all on these unsupported platforms. In future versions, we will aim for 'graceful degradation' in unsupported platforms, and are working on a good definition for testing that objective.