API:Recent changes stream

From MediaWiki.org
Jump to: navigation, search
Plug-in Noun project 4032.svg This page is part of the Web APIs hub.


listen.hatnote.com

A core MediaWiki feature is that all changes to pages are visible: you can see who made what change to a page. On an active wiki, the overall pattern of activity is as interesting as the specific changes to pages: you can see spikes in overall activity, sudden interest in particular pages in response to news events, when editors are active, etc.

On MediaWiki wikis, the page Special:RecentChanges lists these changes. So you could reload this page and scan it to look for patterns. But there's a better way.

stream.wikimedia.org broadcasts all changes to all public wikis run by Wikimedia Foundation to clients over HTTP via wikitech:EventStreams.

How it works on Wikimedia wikis[edit]

For years, the only recent changes feed was IRCColourfulRCFeedFormatter.php, which sent formatted IRC (Internet Relay Chat) messages to an IRC daemon on irc.wikimedia.org that broadcasted changes on various IRC channels.


MediaWiki version: 1.29

In 2017, wikitech:EventStreams was launched to expose arbitrary stream data over HTTP. This service replaces RCStream (described below). It uses the same JSONRCFeedFormatter.php to format RecentChanges, but also has the ability to resume from a older stream position, and also to expose more streams than just RecentChanges. Manual:RCFeed describes the properties of the change, which include the title of the page, the editor's comment about the change, the old and new revision IDs, the old and new length of the page, etc.


MediaWiki version: 1.25

In 2014, developers added another feed, JSONRCFeedFormatter.php, which formats each recent change as JSON and sends it over Redis to a simple "RCStream" server at stream.wikimedia.org/rc.

The RCStream daemon uses Socket.IO library to broadcast 'change' events on the endpoint stream.wikimedia.org/rc. Socket.IO provides an interface on top of websockets in recent browsers, and falls back to alternative transports in earlier browsers.

This allows anyone to write a visualization or analysis of recent changes to wiki content, such as the level of activity, major additions or deletions, new images or other media, and so on. Developers have already written excellent tools that do this by parsing the old IRC messages, such as http://listen.hatnote.com, the audiovisualization, shown above. The reliable and fast RCStream server makes it easier.

Between February and July 2017, developers will be asked and assisted to move RCStream based clients to EventStreams. Eventually, RCStream will be deprecated and turned off.

Explore the recent changes stream[edit]

"EventStreams" on Wikitech describes the API.

See codepen.io/ottomata/pen/VKNyEw/ for a simple demo. It uses the EventSource library to connect with the stream and displays each change.

There are many implementations of EventSource clients. The EventStreams page shows an equivalent code samples in Python and NodeJS.

How it works in the MediaWiki code[edit]

When MediaWiki writes a change to the RecentChanges page, it also calls notifyRCFeeds(). The $wgRCFeeds lets you configure different kinds of recent changes feeds, with different kinds of formatting. includes/rcfeed in core has their implementation. If you run a wiki of your own and would like to adapt these visualizations and monitors to it, you can enable similar feeds to a similar client that publishes changes over HTTP, WebSockets or on an IRC channel. As usual, all aspects of WMF's setup are free and open source.

Alternatives: request changes through the MediaWiki API[edit]

The regular MediaWiki API has various modules that report on changes. These use a "pull" model, you make an API request asking for changes rather than consuming a stream of changes "pushed" over HTTP, WebSockets or IRC.

feedrecentchanges action[edit]

This module provides changes to pages as an RSS feed. The feedformat you specify (atom or RSS, both XML output) determines the format of the API response, overriding the format parameter

For example, to request a feed of 14 days of changes in Category:Open requests for comment, excluding edits marked as "bot" edits, in the Atom format:.

For more information, see Special:ApiHelp/feedcontributions.

recentchanges query module[edit]

You can use this to query for changes. It doesn't generate the same
← Older revision Revision as of some date
output as feedrecentchanges, but it provides the revision IDs of page changes and can provide other information. To make a similar kind of request as the feedrecentchanges example above, use a generator of categorymembers=Category:Open request for comment. (Broken?)

Query recentchanges for pages in Category:Open requests for comment, excluding bot edits, in recent time.

For more information, see Special:ApiHelp/query+recentchanges.

Real-Time Recent Changes is a JavaScript tool that uses the recentchanges query module to monitor recent changes of a wiki in real-time.

See also[edit]