Extension:XMLRC

From MediaWiki.org
Jump to: navigation, search
MediaWiki extensions manual
Crystal Clear action run.png
XMLRC

Release status: experimental

Implementation Hook
Description Sends notifications about changes as XML, via UDP, XMPP or other means.
Author(s) Daniel Kinzler, copyright Wikimedia Deutschland e.V. (Duesentriebtalk)
MediaWiki 1.16
PHP 5
License GPL
Download SVN root
README
log
Hooks used
RecentChange_save

Translate the XMLRC extension if it is available at translatewiki.net

Check usage and version matrix; code metrics

XMLRC is designed to push notifications about changes on the wiki to listeners, in a way that makes it easy to process this information. To this end, XMLRC sends UDP-packets containing XML formated notifications. The XML contains <rc>-tags exactly like those generated by the web API when asked to list recentchanges. This way, anyone interested in changes on the wiki can simply listen to these notifications instead of constantly polling the web API.

Architecture[edit | edit source]

Usually, XMLRC will send out UDP packets containing XML tags that describe recent changes items. Such UDP packets can easily be received and processed directly on the local network. However, for further distribution to multiple listeners over the internet, UDP isn't practical. For this reason, XMLRC contains udp2xmpp.py, a bridge service that can pass the XML messages from the UDP packets to an XMPP chat or chat group (MUC). XMPP allows messages to carry any XML elements as extra payload in messages, so it is easy to attach the full <rc>-tag to the message while providing a human readable version of the notification as the "normal" chat message body. The human-readable messages are similar to the ones used for the traditional IRC-based notifications.

So, a normal Jabber chat message could look something like this:

<message xmlns="jabber:client" to="john@jabber.org/mars" 
         from="enwiki@conference.jabber.toolserver.org/xmlrc" 
         id="23" type="groupchat">
  <body>Hello John</body>
</message>

An XMPP message stanza generated by udp2xmpp.py however would contain an <rc> tag, as returned by the we API, in addition to a human readable notification similar to the ones used on IRC:

<message xmlns="jabber:client" to="john@jabber.org/mars"
         from="enwiki@conference.jabber.toolserver.org/xmlrc"
         id="38" type="groupchat">
  <body>[[List of Knight's Cross recipients: Z]]  
        http://en.wikipedia.org/w/index.php?diff=378715833&amp;rcid=389791968&amp;oldid=378714319&amp;title=List+of+Knight%27s+Cross+recipients%3A+Z * 
        MisterBee1966 * (+1203) /* Recipients */</body>
  <rc comment="/* Recipients */ " newlen="26554" rcid="389791968"
      pageid="8089657" title="List of Knight's Cross recipients: Z"
      timestamp="2010-08-13T14:08:49Z" wikiid="enwiki" 
      revid="378715833" old_revid="378714319" user="MisterBee1966"
      ns="0" type="edit" oldlen="25351">
    <tags />
  </rc>
</message>

Any XMPP client aware of the meaning of the <rc> tag can thus easily access all the details about that recentchanges item.

The reason not to use XMPP directly, but rather take a detour via UDP, is this: PHP scripts such as MediaWiki are not persistent, they start afresh for every HTTP request. This means that MediaWiki can't stay connected to the XMPP server, but would rather need to connect, authenticate, and join every time a page is saved. Potentially multiple times in parallel. This would be very slow (especially if the XMPP server fails to respond), and would put considerable strain on the XMPP server. Using UDP removes keeps MediaWiki's execution independent of the state of any other service, and is cheap in terms of system resources. Handling all XMPP traffic via a single bridge process is much more efficient.


So, the flow of information looks like this:

MediaWiki + XMLRC
   \--(UDP)--> udp2xmpp.py
                  \--(XMPP)--> XMPP server ..... XMPP server 
                                                  \--(XMPP)--> XMPP client 
                                                              (e.g. rcclient.py)

Note however that UDP does not guarantee delivery of packets, or packet order. XMLRC does not try to compensate for this. This means that notifications sent via UDP may be lost or delivered in an order different from how they were sent. This is rare when using UDP in a local network with few switches in between. If you experience packet loss, this usually indicates that either a) the network is saturated or b) the process that is receiving the packets is too slow (that is, the input buffer overflows).

If however the UDP packets gets delivered correctly, and the XML bit gets relayed to XMPP by udp2xmpp.py, the extra <rc> element on the XMPP chat messages can be used easily by any client aware of it. XMLRC provides a basic python library, rcclient.py, demonstrating this. rcclient.py can also be run directly as a program, it then simply echos all RC messages to the console. This may be useful for debugging.

For testing purposes, XMLRC also provides rc2udp.py, a generator service that polls the web API of some wiki periodically, and emits UDP packets for each change it received. This way, a UDP-to-XMPP bridge setup can be tested without having to actually install the XMLRC extension into the target wiki. For production use, this is however not recommended.

Installation[edit | edit source]

To make XMLRC emit UDP-packets about changes, add this to your LocalSettings.php file:

require_once( "$IP/extensions/XMLRC/XMLRC.php"  );
 
$wgXMLRCTransport = array(
  'class' => 'XMLRC_UDP',    # this means "send UDP packets"
  'port' => 4455,            # ...to port 4455
  'address' => '127.0.0.1',  # ...on the local machine
);

Configuration[edit | edit source]

$wgXMLRCTransport
defines which transport should be used to distribute the XML snippets generated for each change item. The value for this option is an associative array, with the key 'class' defining which transport class to use. All other items in the array are specific to the individual transport classes:
'class' => 'XMLRC_UDP'
send UDP packets to a given address and port. Parameters:
address
the IP address to send the UPD packets to
port
the port to send the UDP packets to
'class' => 'XMLRC_File'
writes XML to a file. Mostly useful for testing and debugging. Parameters:
file
the file to write to
$wgXMLRCProperties
defines which properties of the recent changes item should be included in the notification sent out by XMLRC. This value may be given as an array of property names, or a string where the property names are separated by the pipe character "|". The property names are the same used for the API when specifying the rcprops argument when listing recentchanges. The default value for wgXMLRCProperties is 'user|comment|flags|timestamp|title|ids|sizes|redirect|loginfo|tags'.

XMPP Server Setup[edit | edit source]

There are a lot of public jabber servers like http://jabber.org (see list at http://xmpp.org/services/). You can use pretty much any of them to host your MUC rooms for RC notifications, but when doing so for a busy wiki, you should probably ask them whether it's OK. In many cases, it might be better if you ran your own Jabber server for this purpose, there are several open source server implementations available [1].

Below is a list of things you may want to do or consider when setting up an XMPP server and MUC rooms for XMLRC. How these things work differs greately from server to server, though.

  • Decide whether you want to allow account creation. If you only want the server for handling XMLRC, it's probably best to disable account creation. People can use any Jabber account to access your chat rooms anyway.
  • Decide whether you want to use MUC chat rooms for making RC events public. If yes, configure your server to support MUC. By convention, you should set up a separate subdomain for this: if you server is on jabber.example.com, then there should be conference.jabber.example.com for MUC. If this is actually required depends on your server.
  • If using MUC, set up a room for each wiki. The room should probably be set to be permanent and moderated, so only XMLRC can talk in the room, and others can join, but not talk there. Note: In the future, XMLRC should include a utility to do this for you.
  • You probably also want to disable chat room creation by others.

That's about it. The standard setup of the server should be sufficient for it to allow public access to your XMLRC feed.

Future[edit | edit source]

  • Add a utility to easily create MUC rooms for publishing RC events
  • Add support for XMPP PubSub, which is more suitable for this application than MUC. However, PubSub is complex, not supported by all servers and only by a few client libraries.
  • Implement service that allows arbitrary queries to the MediaWiki API via Jabber (using iq requests, perhaps?)
  • use an XML namespace on the <rc> tag, see bugzilla:24781
  • improve support for nested tags in rcclient.RecentChange
  • investigate support for TIPC (sounds nice, but is it supported by networking gear, etc?)

See also[edit | edit source]