Extension:XMLRC

XMLRC is designed to push notifications about changes on the wiki to listeners, in a way that makes it easy to process this information. To this end, XMLRC sends UDP-packets containing XML formated notifications. The XML contains -tags exactly like those generated by the web API when asked to list recentchanges. This way, anyone interested in changes on the wiki can simply listen to these notifications instead of constantly polling the web API. UDP packets can easily be received and processed directly on the local network. However, for further distribution to multiple listeners over the internet, UDP isn't practical. For this reason, XMLRC contains a bridge service that can pass the XML messages from the UDP packets to an XMPP chat or chat group (MUC). XMPP allows messages to carry any XML elements as extra payload in messages, so it is easy to attach the full -tag to the message while providing a human readable version of the notification as the "normal" chat message body. The human-readable messages are similar to the ones used for the traditional IRC-based notifications.

The reason not to use XMPP directly, but rather take a detaour via UDP, is this: PHP scripts such as MediaWiki are not persistent, they start afresh for every HTTP request. This means that MediaWiki can't stay connected to the XMPP server, but would rather need to connect, authenticate, and join every time a page is saved. Potentially multiple times in parallel. This would be very slow (especially if the XMPP server fails to respond), and would put considerable strain on the XMPP server. Using UDP removes keeps MediaWiki's execution independant of the state of any other service, and is cheap in terms of system resources. Handling all XMPP traffic via a single bridge process is much more efficient.

 So, the flow of information looks like this:

MediaWiki + XMLRC \--(UDP)--> udp2xmpp.py                      \--(XMPP)--> XMPP server \--(XMPP)--> XMPP client (e.g. rcclient.py)

This extra payload on the XMPP chat messages can be used easily by any client aware of it. XMLRC provides a basic python library demonstrating this. rcclient.py can also be run directly as a program, it then simply echos all RC messages to the console. This may be useful for debugging.

For testing purposes, XMLRC comes with a generator service that polls the web API of some wiki periodically, and emity UDP packets for each change it received. This way, a UDP-to-XMPP bridge setup can be tested without having to actually install the XMLRC extension into the target wiki. For production use, this is however not recommended.

NOTE: UDP does not guarantee delivery of packets, or packet order. XMLRC does not try to compensate for this. This means that notifications sent via UDP may be lost or delivered in an order different from how they were sent. This is rare when using UDP in a local network with few switches in between. If you experience packet loss, this usually indicates that either a) the network is saturated or b) the process that is receiving the packets is too slow (that is, the input buffer overflows).

The XMLRC extension
This section describes the MediaWiki extension XMLRC proper. This extension generates an XML tag for every change done on the wiki (that is, anything that would appeal on Special:Recentchanges, and hands it to a transport object to distribute it. Currently, there are two transports implemented: one will send a UDP packet for every change, the other one simply writes them to a file.

Installation
To make XMLRC emit UDP-packets about changes, add this to your LocalSettings.php file:

Configuration

 * $wgXMLRCTransport: defines which transport should be used to distribute the XML snippets generated for each change item. The value for this option is an associative array, whit the key 'class' defining which transport class to use. All other items in the array are specific to the individual transport classes:
 * 'class' => 'XMLRC_UDP': send UDP packets to a given address and port. Parameters:
 * address: the IP address to send the UPD packets to
 * port: the port to send the UDP packets to
 * 'class' => 'XMLRC_File': writes XML to a file. Mostly useful for testing and debugging. Parameters:
 * file: the file to write to


 * $wgXMLRCProperties: defines which properties of the recent changes item should be included in the notification sent out by XMLRC. This value may be given as an array of property names, or a string where the property names are separated by the pipe character "|". The property names are the same used for the API when specifying the rcprops argument when listing recentchanges. The default value for wgXMLRCProperties is.

The UDP-to-XMPP bridge (udp2xmpp.py)
'''

An XMPP message stanza generated by udp2xmpp.py however would contain an  tag, as returned by the we API, in addition to a human readable notification dimiar to the ones used on IRC:

Any XMPP client aware of the meaning of the  tag can thus easily access all the details about that recentchanges item.

Installation
directory. You can place this directory anywhere on your system. In fact, for security reasons, it is recommended to move it to a location that is not accessible from the web or by the web server.

To run udp2xmpp.py, you need python 2.5 and the xmpppy library from . Make sure python can find the library, e.g. by adjusting the PYTHONPATH environment variable.

You also need an account on an XMPP (Jabber) server for use by udp2xmpp.py. The credentials for this account then need to be entered into the configuration file for udp2xmpp, see below.

Configuration
In order to configure udp2xmpp for use with your wiki, rename  to , and    to

The configuration filed uses the INI file syntax.

udp2xmpp.ini
udp2xmpp.ini contains the configuration for the bridge, and is structured into three sections:


 * [udp2xmpp] contains configuration for the bridge as such
 * wiki-info-file: the location of the  file. This may be given relative to the location of the   file. This may be overwritten by the --wiki-info command line option. If nither is given, udp2xmpp-wikis.ini will be looked for in the directory where udp2xmpp.py is located.
 * wiki-info-section: name of a wiki info section in this config file; must be the wikiid of the target wiki. If this is set, udp2xmpp-wikis.ini is ignored. This is useful if you only need a single wiki.
 * [UDP] contains configuration for receiving UDP packets
 * port: the UDP port to listen to. Default is 4455
 * interface: the IP address of the network interface to bind to. Default is 0.0.0.0, which means "all devices". Set it to 127.0.0.1 to only listen on the local loopback interface.
 * buffer-size: maximum expected size of UDP packets. This should be larger than the XML representation of any recent changes item could get, but not larget than the size the network stack can handle. Default is 8192. Increase this if you have problems with the XML getting cut off.
 * [XMPP] contains configuration for connectiong to an XMPP service
 * jid: the Jabber ID (JID) to use to connect to the XMPP service, e.g. mywiki-xmlrc@jabber.example.com
 * password: the password for logging into the XMPP service. NOTE: make sure this file is not accessible via the web!

udp2xmpp-wikis.ini
udp2xmpp-wikis.ini contains configuration for each wiki that shall be handled by the bridge. The name of each section must be the wikiid of the wiki for which it applies, as reported by the web API when asked for general site info. A wiki's ID, as defined internally by wfWikiID, is generally the name of the database the wiki uses ( $wgDBname ) plus, if applicable, the table prefix, separated by a dash ( $wgDBname-$wgDBprefix ). You can find these values in you LocalSettings.php.

Each section must specify the channel and channel-type options. In addition, base-url or both of page-url and script-url have to be defined:


 * base-url: the base URL of your wiki, equivalent to $wgScriptPath. Must end with "/".
 * page-url: the URL of pages in your wiki, with $1 as a placeholder for the page name. If not given, defaults to {base-url}index.php/$1
 * script-url: the URL of the main entry point for the wiki ("ugly" URL for index.php). If not given, defaults to {base-url}index.php
 * channel-type: type of the notification channel. can be 'jabber' or 'muc'.
 * channel: the channle's name/address. The meaning depends on the channel type, but for both 'jabber' and 'muc' this is the XMPP JID of the target user or chat group respectively.

If you only need to handle a single wiki, you can put the section for this wiki directly into udp2xmpp.ini and point wiki-info-section to that section. In that case, udp2xmpp-wikis.ini is ignored.

Running the bridge
When everything is configured, you can start udp2xmpp.py simply by typing:

python udp2xmpp.py

Since this is a long running process, it's advisable to put it into a screen session on a server.

udp2xmpp.py accepts the following command line arguments:


 * --help: print a help message an exit.
 * --config: configuration file to use. If not given, udp2xmpp.ini will be looked for the the directory where udp2xmpp.py is located.
 * --wiki-info: configuration file to use. If not given, udp2xmpp-wikis.ini will be looked for in the location given by the wiki-info-file option in udp2xmpp.ini. If that is also not provided, it will be looked for in the directory where udp2xmpp.py is located.
 * --debug: sets verbosity to debug level
 * --quiet: sets verbosity to quiet, only output warnings and errors.

udp2xmpp.py also accepts commands from standard input. Thes must start with "/":


 * /quit: terminates udp2xmpp
 * /send <...>: sends a message to *all* channels
 * /debug: sets verbosity to debug (noisy) level
 * /verbose: sets verbosity to normal (informative) level
 * /quiet: sets verbosity to quiet, only output warnings and errors.

The XMPP RC client (rcclient.py)
TBD

The UDP generator (rc2udp.py)
TBD

XMPP Server Setup
There are a lot of public jabber servers like http://jabber.org (see list at http://xmpp.org/services/). You can use pretty much any of them to host your MUC rooms for RC notifications, but when doing so for a busy wiki, you should probably ask them whether it's OK. In many cases, it might be better if you ran your own Jabber server for this purpose, there are several open source server implementations available.

Below is a list of things you may want to do or consider when setting up an XMPP server and MUC rooms for XMLRC. How these things work differs greately from server to server, though.


 * Decide whether you want to allow account creation. If you only want the server for handling XMLRC, it'S probably best to disable account creation. People can use any Jabber account to access your chat rooms anyway.
 * Decide whether you want to use MUC chat rooms for making RC events public. If yes, configure your server to support MUC. By convention, you should set up a separate subdomain for this: if you server is on jabber.example.com, then there should be conference.jabber.example.com for MUC. If this is actually required depends on your server.
 * If using MUC, set up a room for each wiki. The room should probably be set to be permanent and moderated, so only XMLRC can talk in the room, and others can join, but not talk there. Note: In the future, XMLRC should include a utility to do this for you.
 * You probably also want to disable chat room creation by others.

That's about it. The standard setup of the server should be sufficient for it to allow public access to your XMLRC feed.

Future

 * Add a utility to easily create MUC rooms for publishing RC events
 * Add support for XMPP PubSub, which is more suitable for this application than MUC. However, PubSub is complex, not supported by all servers and only by a few client libraries.
 * Implement service that allows arbitrary queries to the MediaWiki API via Jabber (using iq requests, perhaps?)
 * use an XML namespace on the  tag, see 24781
 * improve support for nested tags in rcclient.RecentChange