Toolserver:OpenStreetMap server/Setup notes

Jump to navigation Jump to search
Wikimedia Community Logo-Toolserver.svg

This page was moved from the Toolserver wiki.
Toolserver has been replaced by Toolforge. As such, the instructions here may no longer work, but may still be of historical interest.
Please help by updating examples, links, template links, etc. If a page is still relevant, move it to a normal title and leave a redirect.

Cassini is no longer in use and its successor ptolemy is setup differently. Haggi 19:04, 12 August 2011 (UTC)

These are notes I (Ævar Arnfjörð Bjarmason) am keeping when setting up Cassini.


This is the software running on Cassini

From apt[edit]

Non-standard software installed from the package system:

  • aptitude install screen pv bzip2
  • aptitude install subversion
  • aptitude build-dep mapnik
  • aptitude install python-cairo-dev libcairomm-1.0-dev # for mapnik
  • aptitude install iotop
  • aptitude install unzip # To get a stupd CPAN package that used a .zip file instead of gz/bzip2
  • aptitude install build-essential libxml2-dev libgeos-dev libpq-dev libbz2-dev proj # for osm2pgsql
  • aptitude install postgresql-8.3-postgis
  • aptitude install apache2
  • aptitude build-dep libapache2-mod-python # to get apache/apr dev headers for mod_tile
  • aptitude install strace
  • aptitude install gdb
  • aptitude install git-core
  • aptitude install ttf-unifont # for i18n support in rendering
  • aptitude install gdal-bin # for e.g. hillshading
  • aptitude install libgdal1-dev # for building Mapnik with gdal support

From CPAN[edit]

Things installed from the CPAN. I'm using CPANPLUS (cpanplus) to install things. If this listing is incomplete /home/ravar/.cpanplus/install-logs/ should be the full listing.

  • WWW::Mechanize
  • HTML::TableParser::Grid
  • YAML::Syck
  • App::Ack

From source[edit]

Directories with stuff built from source, in /usr/local/src/


ravar@cassini:/usr/local/src$ svn co mapnik

Make it build with gdal and other goodies:

ravar@cassini:/usr/local/src/mapnik$ python scons/ INPUT_PLUGINS='all'

Build it:

ravar@cassini:/usr/local/src/mapnik$ python scons/ configure
ravar@cassini:/usr/local/src/mapnik$ python scons/
ravar@cassini:/usr/local/src/mapnik$ sudo python scons/ install

By default mapnik can't find its .so:

ravar@cassini:/usr/local/src/mapnik$ python
Python 2.5.2 (r252:60911, Jan  4 2009, 21:59:32)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mapnik
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.5/site-packages/mapnik/", line 54, in <module>
    from _mapnik import *
ImportError: cannot open shared object file: No such file or directory

You need to add /usr/local/lib64 to /etc/

ravar@cassini:~$ diff -ru /etc/ /etc/
--- /etc/    2009-07-21 14:54:36.000000000 +0000
+++ /etc/     2009-07-21 14:54:36.000000000 +0000
@@ -1,3 +1,4 @@
 # Multiarch support

Then run:

sudo ldconfig
aptitude install ttf-unifont

Make renderd / mapnik load it:

sudo ln -s /usr/share/fonts/truetype/unifont/unifont.ttf /usr/local/lib64/mapnik/fonts/unifont.ttf


We're using various stuff from the OSM SVN, copied with:

ravar@cassini:/usr/local/src$ svn co osm
ravar@cassini:/usr/local/src/osm/applications/utils/export/osm2pgsql$ make
ravar@cassini:/usr/local/src/osm/applications/utils/export/osm2pgsql$ sudo make install


These are our tools for setting up or own rendering/stuff hosted in MediaWiki SVN:

ravar@cassini:/usr/local/src$ svn co
sudo aptitude install python-yaml
sudo ln -s /usr/local/src/osm-tools/cgi-bin/export /usr/lib/cgi-bin/export
sudo ln -s /usr/local/src/osm-tools/cgi-bin/export-config.yml /sql/etc/export-config.yml


Create the renderd.conf:

# perl --languages /sql/misc-data/wikipedia-languages.yml > /etc/renderd.conf

Create the directories renderd emits to:

avar@cassini:/sql/mod_tile$ mkdir $( perl /usr/local/src/osm-tools/osm2pgsql-style/ | perl -MYAML::Syck -E 'say join q[ ], sort keys %{ LoadFile(*STDIN) }' )



We have a copy of the Planet.osm file in /sql/planet.osm/

TODO: Keep it automatically up-to-date:


PostgreSQL has been setup to store its data on the /sql partition:

ravar@cassini:/sql$ sudo ack -a /sql /etc/postgresql
38:data_directory = '/sql/postgresql/8.3/main'          # use data in another directory

And for allowing password auth from localhost:

ravar@cassini:~$ sudo tail -n9 /etc/postgresql/8.3/main/pg_hba.conf
# "local" is for Unix domain socket connections only
#local   all         all                               ident sameuser
# IPv4 local connections:
#host    all         all          md5
# IPv6 local connections:
#host    all         all         ::1/128               md5

local   all         all                               trust
host    all         all          trust

Imported databases[edit]

At first I wasn't using --slim, which is less usable, see:

TODO: Re-import with intarray support to support OSC updating: see [1]

With i18n support[edit]
createdb -E UTF8 -O gis gis
createlang plpgsql gis
psql -d gis -f /usr/share/postgresql-8.3-postgis/lwpostgis.sql
echo "ALTER TABLE geometry_columns OWNER TO gis; ALTER TABLE spatial_ref_sys OWNER TO gis;"  | psql -d gis

Create language list & style:

perl /usr/local/src/osm-tools/osm2pgsql-style/ > /sql/misc-data/languages.yml
perl /usr/local/src/osm-tools/osm2pgsql-style/ --style  /usr/local/src/osm/applications/utils/export/osm2pgsql/ --languages /sql/misc-data/languages.yml > /sql/planet.osm/

Import the planet:

osm2pgsql --slim -d gis -U avar -W -S ./ planet-090715.osm


As documented on the OSM wiki:

avar@cassini:/sql/world_boundaries/packed$ wget
avar@cassini:/sql/world_boundaries/packed$ wget
avar@cassini:/sql/world_boundaries/packed$ wget
avar@cassini:/sql/world_boundaries/unpacked$ unzip ../packed/ 
Archive:  ../packed/
  inflating: coastlines/processed_p.dbf  
  inflating: coastlines/processed_p.index  
  inflating: coastlines/processed_p.shp  
  inflating: coastlines/processed_p.shx  
avar@cassini:/sql/world_boundaries/unpacked$ tar xjvf ../packed/shoreline_300.tar.bz2 
avar@cassini:/sql/world_boundaries/unpacked$ tar xzvf ../packed/world_boundaries-spherical.tgz 
avar@cassini:/sql/world_boundaries/unpacked$ mv coastlines/* world_boundaries/* .
avar@cassini:/sql/world_boundaries/unpacked$ rmdir coastlines world_boundaries
avar@cassini:/sql/world_boundaries/unpacked$ ls -l
total 534M
-rw-r--r-- 1 avar users 3.4M 2007-03-10 10:05 builtup_area.dbf
-rw-r--r-- 1 avar users 273K 2007-03-10 10:05 builtup_area.index
-rw-r--r-- 1 avar users  513 2007-03-10 10:05 builtup_area.prj
-rw-r--r-- 1 avar users  13M 2007-03-10 10:05 builtup_area.shp
-rw-r--r-- 1 avar users 285K 2007-03-10 10:05 builtup_area.shx
-rw-r--r-- 1 avar users  16K 2007-03-10 10:07 places.dbf
-rw-r--r-- 1 avar users  333 2007-03-10 10:07 places.prj
-rw-r--r-- 1 avar users 7.0K 2007-03-10 10:07 places.shp
-rw-r--r-- 1 avar users 2.1K 2007-03-10 10:07 places.shx
-rw-r--r-- 1 avar users 4.3M 2008-08-10 20:48 processed_p.dbf
-rw-r--r-- 1 avar users 2.0M 2008-08-10 21:59 processed_p.index
-rw-r--r-- 1 avar users 357M 2008-08-10 20:48 processed_p.shp
-rw-r--r-- 1 avar users 3.4M 2008-08-10 20:48 processed_p.shx
-rw-r--r-- 1 avar users 3.0M 2009-07-10 04:01 shoreline_300.dbf
-rw-r--r-- 1 avar users 1.3M 2009-07-10 04:01 shoreline_300.index
-rw-r--r-- 1 avar users  86M 2009-07-10 04:01 shoreline_300.shp
-rw-r--r-- 1 avar users 2.0M 2009-07-10 04:01 shoreline_300.shx
-rw-r--r-- 1 avar users 4.0M 2007-03-09 11:31 world_bnd_m.dbf
-rw-r--r-- 1 avar users 246K 2007-09-09 19:28 world_bnd_m.index
-rw-r--r-- 1 avar users  355 2007-03-09 11:31 world_bnd_m.prj
-rw-r--r-- 1 avar users  47M 2007-03-09 11:32 world_bnd_m.shp
-rw-r--r-- 1 avar users 244K 2007-03-09 11:32 world_bnd_m.shx
-rw-r--r-- 1 avar users 659K 2008-03-30 21:42 world_boundaries_m.dbf
-rw-r--r-- 1 avar users  85K 2008-03-30 21:48 world_boundaries_m.index
-rw-r--r-- 1 avar users  355 2008-03-30 21:51 world_boundaries_m.prj
-rw-r--r-- 1 avar users 6.4M 2008-03-30 21:42 world_boundaries_m.shp
-rw-r--r-- 1 avar users  30K 2008-03-30 21:42 world_boundaries_m.shx


Create a HTML page to browse each of our 279 languages:

avar@cassini:/var/www/tile-browse$ for lang in $(perl /usr/local/src/osm-tools/osm2pgsql-style/ | perl -MYAML::Syck -E 'say join q[ ], sort keys %{ LoadFile(*STDIN) }'); do cp .browse-template.html browse-$lang.html && lang=$lang perl -pi -e 's[(?<=osm-like/)(\w+)][$ENV{lang}]' browse-$lang.html; done

Category:OpenStreetMap --Cseki 04:22, 19 November 2009 (UTC)