Service-template-node/GettingStarted

This guide will walk you through the most basic steps needed to create a new service based on the service-template-node project. As an example we'll build a micro-service that does nothing more than convert an SVG stored in MediaWiki to PNG, and return it the user. We will call this new service svg2png.

Creating the service

The service-template-node project is meant to act as a skeleton, providing new projects with all of the boiler plate needed to hit the ground running with a web service that conforms to best-practices. To create your new service, start by cloning.

$ git clone https://github.com/wikimedia/service-template-node.git svg2png

Project metadata

In the top-level of your new project is a file named package.json that contains important meta-data. After cloning the new repository, this meta-data naturally pertains to the service-template-node project, so open it with the editor of your choice, and customize it for our svg2png service. At a minimum, you should update the name, version, description, repository, author, bug tracking URL (bugs), and homepage.

{
  "name": "svg2png",
  "version": "0.1.0",
  "description": "A service for converting SVGs to PNGs",
  "main": "./app.js",
  "scripts": {
    ...
  },
  "repository": {
    "type": "git",
    "url": "git://github.com/eevans/svg2png.git"
  },
  "keywords": [
    "REST",
    "API",
    "images",
    "conversion",
    "MediaWiki"
  ],
  "author": "Eric Evans <eevans@wikimedia.org>",
  "contributors": [],
  "license": "Apache-2.0",
  "bugs": {
    "url": "https://github.com/eevans/svg2png/issues"
  },
  "homepage": "https://github.com/eevans/svg2png",

  ...

}

Note: A complete description of package.json is beyond the scope of this document; You are encouraged to consult the documentation for package.json.

Configuration

Our newly minted service reads its configuration at startup from a YAML configuration file. This file can be used to configure a number of the features inherited from service-runner, (logging, metrics, etc), but for the time-being, let's configure the service name, and a port number to listen on.

Open the included example config.dev.yaml and configure a service name and port:

- services:
  - name: svg2png
    conf:
      port: 8825

Dependencies

Finally, before we can begin implementing our service, we need to ensure that we have all required dependencies. service-template-node provided us with a list of sensible default dependencies in package.json, but we must explicitly install them ourselves. Let's do that now:

$ npm install
...

This will download the NodeJS modules specified in package.json, including any transitive dependencies, to a directory named node_modules where they can be loaded by our service.

Specific to our service though, is the need to convert SVG graphic files to PNG format, and fortunately for us there exist NodeJS bindings for the excellent librsvg library. The only caveat here, is that in addition to downloading the Javascript source, npm also needs to compile and link some architecture-specific code. This means that you must have librsvg installed on your host platform first.

On Debian-based systems, this is as easy as:

$ sudo apt-get install librsvg2-dev

With librsvg installed, we can now invoke npm to complete the installation.

$ npm install --save librsvg

Note: the --save argument above is a convenience that instructs npm to append librsvg to the list of dependencies in package.json.

Adding a route

The purpose of a route is to map a resource (URL) to the code responsible for processing its requests.

Let's think for a moment about our SVG-to-PNG conversion service, and what a route for conversions should look like.

By convention, we want each of our routes to be per-domain, so that services can operate against an arbitrary number of MediaWiki instances. And, we include a version so that we can make changes later without disrupting existing users. These conventions are so common that routes automatically loaded from the routes/ subdirectory support templated URL parameters for the domain and version out-of-the-box.

http://host:port/{domain}/{version}/

For our service, we also need the name of the SVG file to convert.

http://host:port/{domain}/{version}/png/{file}

Note: We could just append our filename parameter, assume that conversions to PNG are implicit, but adding the png element above buys us a little flexibility should we decide to extend the service to other formats later.

For example:

http://localhost:8825/commons.wikimedia.org/v1/png/Square_funny

Creating the route module

Create your new route module by copying the example routes/empty.js.template, to routes/svg2png-v1.js.

Next, edit routes/svg2png-v1.js, and change the exported function (at the bottom of the file), so that it looks something like:

module.exports = function(appObj) {

     app = appObj;

     // the returned object mounts the routes on
     // /{domain}/vX/mount/path
     return {
         path: '/png',
         api_version: 1,  // must be a number!                            
         router: router
     };
};

Now, find a convenient location in the body of the file to register a route, and its corresponding handler function.

var Rsvg = require('librsvg').Rsvg;
router.get('/:file', function(req, res) {

});

The call to router#get here sets up a route valid for requests made with the GET method. The first argument is appended to rest of the resource for this module to create the final URL http://host:port/{domain}/{version}/png/{file}. The second argument is a function that will be invoked on requests, and gets passed a copy of the request and response objects. Note the format of that first argument, :file is special and will match whatever comes after the /, and be made available as req.params.file to our handler function.

Oh one last thing, don't forget to import librsvg too, we'll soon need it!

We now have a handler that can execute code when GET requests are made to this endpoint, what is remaining is to:

• Fetch the image info for the file from the MediaWiki API for domain
• Use the URL obtained in the API response to fetch the SVG
• Convert the SVG to PNG, and return it to our user

Retrieving imageinfo

The service-template-node project includes a dependency on preq, a promised-based version of req. We'll use it to make the API request.

router.get('/:file', function(req, res) {
    return preq.post({
        uri: 'http://' + req.params.domain + '/w/api.php',
        body: {
            format: 'json',
            action: 'query',
            prop:   'imageinfo',
            iiprop: 'url',
            titles: 'File:' + req.params.file + '.svg'
        }
    })
});

Fetching the SVG

We'll use a continuation that fires when the API request is complete, to extract the file's URL from the response, and use it to fetch the SVG itself.

router.get('/:file', function(req, res) {
    return preq.post({
        uri: 'http://' + req.params.domain + '/w/api.php',
        body: {
            format: 'json',
            action: 'query',
            prop:   'imageinfo',
            iiprop: 'url',
            titles: 'File:' + req.params.file + '.svg'
        }
    })
    .then(function(apiRes) {
        var pages = apiRes.body.query.pages
        var pageId = Object.keys(pages)[0];
        var url = pages[pageId].imageinfo[0].url;
        return preq.get(url);
    })
});

Converting to PNG

Finally, a continuation that fires when the results are ready creates an Rsvg instance from the SVG content, sets the response's Content-Type header to image/png, and writes the converted data to the client.

router.get('/:file', function(req, res) {
    return preq.post({
        uri: 'http://' + req.params.domain + '/w/api.php',
        body: {
            format: 'json',
            action: 'query',
            prop:   'imageinfo',
            iiprop: 'url',
            titles: 'File:' + req.params.file + '.svg'
        }
    })
    .then(function(apiRes) {
        var pages = apiRes.body.query.pages
        var pageId = Object.keys(pages)[0];
        var url = pages[pageId].imageinfo[0].url;
        return preq.get(url);
    })
    .then(function(svgRes) {
        var rsvg = new Rsvg(svgRes.body);
        res.type('image/png');
        res.send(rsvg.render({
            format: 'png',
            width:  rsvg.width,
            height: rsvg.height
        }).data).end();
    });
});

Trying it out

That's it! Start the service...

$ nodejs service.js

...and try it out from your browser:

http://localhost:8825/commons.wikimedia.org/v1/png/Square_funny

Testing

Create a new file named test/features/v1/svg2png.js.

TODO: Describe test setup.

'use strict';
var preq   = require('preq');
var assert = require('../../utils/assert.js');
var server = require('../../utils/server.js');
describe('svg2png', function() {
    this.timeout(20000);
    before(function () { return server.start(); });
});

TODO: Add test case description

'use strict';
var preq   = require('preq');
var assert = require('../../utils/assert.js');
var server = require('../../utils/server.js');
describe('svg2png', function() {
    this.timeout(20000);
    before(function () { return server.start(); });

    it('should return the PNG', function() {
        return preq.get(
            server.config.uri + 'commons.wikimedia.org/v1/png/Square_funny'
        )
        .then(function(res) {
            assert.status(res, 200);
            assert.contentType(res, 'image/png');
            assert.deepEqual(Buffer.isBuffer(res.body), true, 'Unexpected body!');
    });
});

TODO: Document test running.

Next steps

TODO: Talk about next steps