Toolserver:Suggestions

If you have suggestions for things the Toolserver should offer, please add them to this page. You should describe what the suggestion is, and why it would be useful, but you do not need to describe how it should be implemented (unless you want to). Suggestions should be clearly described, and reasonably concise, but should contain enough detail that we can actually understand what it is you want.

Feel free to suggest anything, even if it would currently be against the rules, or you think it would be infeasible or too expensive to implement. Dream big!

Local MediaWiki api.php
Valhallasw has done some work on getting MediaWiki's api.php working on the Toolserver. The basis is working, but there are some security-related issues to think of.


 * What it should do
 * Provide api.php over (internal?) http, so users can easily query the database without using query limits (or using higher limits)
 * Provide FauxRequest interface for PHP users
 * (other, internal, calling method for other languages?)

See /home/valhallasw/src/tsapi (available under GPLv2). It is running under http://toolserver.org/~valhallasw/tsapi/api.php for requests from the toolserver. For example:
 * What has already been done

curl "http://toolserver.org/~valhallasw/tsapi/api.php?wiki_domain=en.wikipedia.org&action=query&prop=info&titles=Main%20Page&format=json"

The api is basically running, but I have not checked security-related issues (for instance hidden revisions).

The following things are accounted for:
 * Basic (database, language, namespace) wiki initialization for the given wiki_domain
 * Caching table (TSBagOStuff.php, which is an adaptation of BagOStuff.php to allow information for multiple wikis in the same table)
 * Disabling index selection (which is not possible with views)
 * Note: This has the disadvantage some queries become *very* slow. These should probably be forwarded to the real wiki?

The following things should still be done:
 * Security! (hidden revs etc)
 * More closely match the actual configuration of the wikis (using InitializeSettings and CommonSettings from noc?) Applicable to for example  (rights for anonymous users are not the same everywhere), but also extension specific configuration (FlaggedRevisions/AbuseFilter for example)
 * Passthrough for slow queries?
 * Does it run REL1_17, 1.17wmf1 or trunk ? It should run 1.17wmf1 (if posisble including it's extensions) and automatically svn update.
 * 1.17wmf1, without extensions. See /home/valhallasw/src/phase3. Valhallasw 16:02, 28 May 2011 (UTC)

Page view statistics
A page view stats database / API to replace the user-run service at http://stats.grok.se.

Direct page text access and search
Locally accessible MySQL database containing page text in a sane format.

Aside from accessing specific revisions's content, searching through them is another important point. Wikimedia's Lucene is great for simple searches by readers, but more advanced searches or even just exact matches are impossible (ie. searching the the exact phrase " " is not posisble, or " ").

Most awesome would be a way to perform an all-wiki search.

Thumbnail renderer
Require X server, screenshot utility, thumbnailer, and Firefox. This makes possible to use thumbnails to represent article in a visual browser of sorts.

High quality text-to-speech
To create high quality speech synthesis for Wikipedia articles and text stored at Wikisource. Useful for mobile devices, iPods, and children's educational apps.

Cross-domain media access
Creating a crossdomain.xml policy file on upload.wikimedia.org allows Flash and Java based tools to access Wikimedia Commons. More tools will be possible such as a Flash based Ogg Vorbis player 360 panorama viewers without the need for a local proxy server.

Toolserver analytics
As discussed before, Toolserver users would like to know how often their tools are being used.

krinkle has been working on a plan to implement this by nightly parsing the apache logs and periodically aggregating the results. Display would be through a toolserver tool that allows filtering by date, url(prefix) and more.

User authentication
A sane way to verify Wikimedia user credentials inside Toolserver tools. Currently hackishly implemented using TUSC.

The efforts by the Wikimedia Foundation in implementing OpenID support will likely solve this.

Mailing list archive search
This is particularly important for private lists. There's no way to search mailing list archives currently and Wikimedia won't be putting resources toward this anytime soon.

River worked on this in January/Februari 2011: http://news.tcx.org.uk/wikimedia.html Although the search appears to be unavailable.

Framework for scripts
Framework for PHP would probably be a good start but I guess you might think about Python and other languages used on TS.

This framework (or maybe a template project) would ideally allow you to just type in few functions calls to make a new simple tool. I guess Toolserver Intuition is a good start when it comes for i18n. What I see missing is:
 * database access base classes - specifically simple connection to various databases
 * caching - data caching with time interval settings
 * common templating system including basic template with some simple layout

The goal would be to have similar layout for many tools for greater user experience and easy creation of tools for developers. Sure some would still want to have fun creating from scratch and experimenting, but if you just want to create something simple then having no framework makes it a bit more painful then it needs to be.


 * I've started converting my personal PHP framework into something reusable. ToyEngine is a small, modular framework which simplifies creating tools on the Wikimedia Toolserver. It provides logging, cached wiki data, simple caching with expiry, and a database wrapper built on top of PDO. The database wrapper enables easy parameterized SQL, simplifies connecting and querying, and optimizes connections when iterating through multiple wiki databases. I put some code examples on the ToyEngine page. When this is more complete and I've converted a few tools, I'll stick the framework on github.


 * It doesn't handle templating, but that should probably be done with something like Smarty (and maybe some template files on github). Pathoschild 05:12, 12 February 2012 (UTC)