This feature is a test of whether giving post-edit feedback to Wikipedia editors enhances their experience, and which kind of feedback is best at motivating them to continue editing.
Prior to this project, one of two things happened when a user saved an edit:
- if the edit is successful, the page reloads in normal "read" mode
- the edit fails due to an error such as an edit conflict or loss of session data
There was no system feedback which explicitly confirms that the edit was successful and is immediately live, though if they are aware of it, editors may check the revision history.
After an edit is saved successfully, a notification appeared on the page along with the read-mode of the page.
Confirmation and gratitude (test one)
In the first phase of this experiment, a sample of logged-in editors received a message every time they successfully save an edit. Depending on which experimental bucket they fell into, they received either a dry confirmation that their edit was saved, or a thank you message for making an edit. These notifications will disappear after three seconds and were be dismissable, with the intent that can editors simply scan the page after editing and use the presence of the notification as confirmation of their edit. Since this feature was primarily designed to support new editors unfamiliar with wiki editing, this confirmation message will only be delivered to editors who registered after the start of the experiment.
Results and future plans
The results from the test above showed that new editors who were told "your edit was saved" had a marginal/insignificant increase in the volume of edits made, with no significant increase in the burden they inflicted on experienced Wikipedians.
Based on this data, we built and deployed a confirmation message, via Extension:PostEdit, into MediaWiki, both for the current wikitext editor. We also recommended to the team building the upcoming VisualEditor that it incorporate a confirmation.
Historical feedback (test two)
In the second phase of this experiment, a sample of logged-in editors received a notification when they reach their first, tenth, 50th, and 100th edits. The purpose of this notification was to thank editors for reaching milestones in the contribution history, as an alternative to the current experience where they receive no information about how much work they have done, without looking it up using a specialized tool.
This notification looked slightly different than the previous one delivered confirming every edit, with the intent that newly-registered editors will feel they have accomplished something significant on Wikipedia. Since this feature is only designed to support new editors unfamiliar with wiki editing, it will only appear to those with less than 100 edits before the start of the experiment.
All messages will utilize the same general design, but the message text and edit count numbers will be specific to the milestone:
- First edit milestone: "Success! You made your first edit to Wikipedia."
- Fifth edit milestone: "You've made five edits. Keep up the good work!"
- 10th edit milestone: "That's your first 10 edits. Nice!"
- 25th edit milestone: "25 edits! You're doing great. "
- 50th edit milestone: "You’ve made 50 edits!"
- 100th edit milestone: "Congratulations! You've completed 100 edits."
Results and future plans
The results from the test above showed no statistically significant difference in the productivity or quality of editors who received the feedback message about editing milestones, compared to a control group. This was somewhat surprising given that experiments with wiki contributors outside Wikipedia concluded that historical feedback was one of the strongest motivators. Considering those prior results, the failure of this experiment may have been due to a variety of reasons, including that unlike the previous confirmation, the message appeared only intermittently and was easier to miss entirely.
In any case, we are not currently recommending that a feature to support this implementation of the feedback message be built. Edit counter-like notifications may eventually be incorporated in to the upcoming notifications system, but it is a low priority compared to many other types of notifications.
Regarding future tests, it may still be fruitful to test either more "sticky" implementations of feedback that would otherwise be intermittent, like the historical feedback previously tried, or other kinds such as social ranking.
- Confirmation and gratitude (test one)
- A message will be displayed to users in the test every time they successfully save an edit and the page is reloaded in read mode.
- Users will either see the confirmation message or the gratitude message, never both.
- The message will appear in a notification box that is overlaid on the page.
- The message will fade away after three seconds.
- The message will be visible even if the user is returned to a section via an anchor in the link.
- The message will be delivered in all namespaces.
- The message will only be delivered to users who registered after the start of the experiment.
- The message will not appear after page creation.
- Historical feedback (test two)
- A message will be displayed to users in the test after they save their first, fifth, 10th, 50th, and 100th edits. (Including if an editing milestone is page creation.)
- The message for each milestone will be unique.
- The messages will appear in a notification box that is overlaid on the page.
- The messages will fade away after no more than five seconds.
- The messages will be dismissible by the user.
- The messages will be visible even if the user is returned to a section via an anchor in the link.
- The messages will be delivered in all namespaces.
Experiment methodology and analysis requirements
- Research on editor milestones has suggested that even at 1,000 edits, not all contributors are aware of the offsite tools for finding out one's edit count.