Create your own tile server and map client

7 min readJun 6, 2018

This document is drafted from my experience after strenuous tinkering with multiple tutorials to create a tile server and a client that uses this tile server, basically after this you will have your own Maps, like Google does. This document lists high level steps to set up your tile server by pointing to existing excellent detailed tutorials for each step.

These are the major steps involved:

Get the data for maps
Process and save data into Postgres database
Create tiles from Postgres
Create a client that makes specific requests to get tile data

The server is responsible for serving tiles (nothing but PNG images) to any client requesting map information for a location. Each tile is 256 X 256 px in dimensions. A map has multiple zoom levels for every location, with 1 being birds eye view of the entire world and 18 being the deepest zoom level. So at zoom level 1, we have 1 tile to serve, for the entire world which is an image of 256 px by 256 px in size. If we zoom in to level 2, we have four tiles each again 256 px by 256 px. So every time we zoom in, we divide the tile from upper layer into 4 pieces. Based on client resolution and location requested, we return the corresponding tiles to the client.

Open Street Map (OSM) is a free open source data source of map data which is updated regularly. Visit https://wiki.openstreetmap.org/wiki/Main_Page to know more about this service. This site distributes data in a format called .osm which describes every location on earth in an XML format. Go ahead and download this file (~20 MB) and extract it (uncompress) to get .osm file. Entities like roads, buildings, monuments and everything is described in XML! This file is for a country called Turkmenistan, which is in Asia. Like this we can download .osm files for every country from http://download.geofabrik.de/. For entire planet osm file, visit http://planet.openstreetmap.org/, however lets us first generate tiles for small countries and then progressively move on.

Here we save the raw XML data into a spatial database so that another service can query this database and generate tiles. The recommended choice of spatial database is Postgres with spatial extensions. Check my articles on how to install and setup Postgres on a MAC and how to add spatial extensions to it. For this we have to install an application called osm2pgsql. Steps vary based on your operating system. If you are on Ubuntu or Fedora you can check the steps on their github page. If you are on a MAC, you can easily install it using homebrew tool. You can either search online for steps or follow this tutorial. If you are on Windows, you can use the tutorial from OSM Wiki. The OSM wiki tutorial has installation steps for every OS, however it is not updated regularly. This is the major step and takes a while and tinkering based on your environment.

Once osm2pgsql is installed, we use its options to save data into Postgres. So lets begin by creating a database for this data in Postgres. All commands are tested for MAC, however the description for each step is the intent for the command which can be used to execute similar commands on other platforms.

Start the Postgres server: pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start
Create a database called gis, with spatial extensions: psql -d gis -c ‘CREATE EXTENSION hstore; CREATE EXTENSION postgis;’

Once these are done you can see all the spatial functions that are added to gis database. One way to check if creation worked properly is to install a GUI client for Postgres like PgAdmin. Using it we can see external and conceptual schemas for each database, including our gis.

Now things are getting together. We have our raw OSM data in XML and Postgres ready to save spatial data. Once we save the data into Postgres, the next step is to generate tiles by making spatial queries to Postgres. But how do the tiles get the beautiful colors we see on map from this raw XML? We need to attach style information for all the entities — roads, buildings, streets etc. Think of them like CSS for HTML files. As there are hundreds of types of entities, writing styles for each can be tedious. Hence we download existing styles from https://github.com/gravitystorm/openstreetmap-carto. Besides colors and fonts, we also need shape files which are like guiding principles for converting XML to geometry shapes (lines, circles, rectangles etc.). Follow the steps to download and install shapefiles mentioned here. If the shell script doesn’t work for you, use the Manual installation instructions below it.

Once shapefiles are downloaded and installed we are now ready to run osm2pgsql. It has many options which can be configured according to your server capacity and input file size. Basic command usage would be:

osm2pgsql -d gis ~/path/to/data.osm.pbf — style openstreetmap-carto.style

and a more elaborate command would be,

./osm2pgsql -U postgres -H localhost -d gis -W “.\california-latest.osm.pbf” — style openstreetmap-carto.style — slim — number-processes 8

Option -H prompts for password to connect to postgres user. Alternative option is to create an environment variable called PGPASS and set its value as password for postgres user. We have now successfully saved our raw data into Postgres! Congratulations!

We now create an intermediate stylefile which is used by tile generation script in the next step. For this clone or download mapnik-stylesheets repository. Navigate to the directory and run,

pip install python-mapnik # thanks to Hüseyin Çapan for reporting it./generate_xml.py osm.xml — dbname gis — host localhost — user postgres — accept-none > out.xml

This step assumes Python is installed on your machine. We will use out.xml file to create tiles in the next step. The big step is done :)

Now we are all set to generate the image files, that is tiles which are sent to the client. We will use another script which uses our Postgres data and our intermediate stylesheet, out.xml to create tiles for all zoom levels. One thing to keep in mind is, we might have data loaded into Postgres only for a small region and not for entire planet, so generating tiles for entire earth is wasteful as the script generates a blank tile if data is not found for a particular region. We will use the script generate_tiles.py from mapnik-stylesheets project downloaded in the previous step. By default the script runs for various countries and so we have edit the script to make it run for our specific region.

Open generate_tiles.py in any editor and modify the bounding boxes to match your region in the __main__ region at the end of the file. To get the coordinates of any region, I use Open Street Map’s export feature and manually select a region as shown in the below screenshot. Then note down the coordinates from left panel and mention in the bounding box

Also set minZoom and maxZoom vales between 0 to 18 which indicates the zoom levels for which tiles are to be generated. You can remove other bounding boxes so that tiles are not generated for them. The contents of __main__ region should be something like,

if __name__ == "__main__":
    home = os.environ['HOME']
    try:
        mapfile = os.environ['MAPNIK_MAP_FILE']
    except KeyError:
        mapfile = home + "/svn.openstreetmap.org/applications/rendering/mapnik/osm-local.xml"
    try:
        tile_dir = os.environ['MAPNIK_TILE_DIR']
    except KeyError:
        tile_dir = home + "/osm/tiles/"if not tile_dir.endswith('/'):
        tile_dir = tile_dir + '/'    #---------------------------------------------------------------
    #
    # Change the following for different bounding boxes and zoom levels
    #    minZoom = 10
    maxZoom = 16
    bbox = (-2, 50.0,1.0,52.0)
    render_tiles(bbox, mapfile, tile_dir, minZoom, maxZoom)

To start generating tiles,

Create an environment variable called MAPNIK_MAP_FILE with the path to out.xml we generated in the previous step
Create an environment variable called MAPNIK_TILE_DIR with a path to where you want the tiles to be saved
Start generating tiles — Run python generate_tiles.py, in a terminal / command prompt (in the directory where this file exists)

This will take a while based on the size of data and number of zoom levels you mentioned.

Congratulations, you now have tiles of your own. Next up, create a client to render these tiles on a web browser.

This is the simplest step of all. Here we create a simple REST server using Python Flask and a HTML file which responds to user actions and talks to this server.

Server

Install Flask following the steps from here. Then create a file called server.py and paste the below code. The client we are about to create, requests tiles in the form /zoom/x/y.png. Luckily the script we used to create the tiles above generates the tiles in the exact folder structure. So all we do is, return the corresponding image file following the path from the request object. Modify the lines marked in red to match your paths.

import os.path
from flask import Flask, send_fileapp = Flask(__name__, static_url_path='/static')@app.route('/tiles/<zoom>/<y>/<x>', methods=['GET', 'POST'])
def tiles(zoom, y, x):
    default = '_path_to_default_tile\\tiles\\0\\11\\333\\831.png' # this is a blank tile, change to whatever you want
    filename = '_path_to_tiles\\tiles\\0\\%s\\%s\\%s.png' % (zoom, x, y)
    if os.path.isfile(filename):
        return send_file(filename)
    else:
        return send_file(default)@app.route('/', methods=['GET', 'POST'])
def index():
    return app.send_static_file('index.html')if __name__ == '__main__':
    app.run(debug=False, host='localhost', port=8080)

Client

Here we will use a JavaScript library called leaflet.js, which takes care of user interaction and requests the above server for the required tiles. Download the leaflet.js and leaflet.css files from here and add them to the same directory as server.py. To get a quick idea about this library, follow this getting started guide to render tiles from a third party server. Now that we have our own tile server and tiles, we need not use the third party server! Create a file called index.html in the same folder as server.py and paste the below code. Add required CSS to match your look and feel. The map is generated to <div id=’map’></div> placeholder, so dont forget to add some width and height.

<html><head>
    <title>My Maps</title>
    <meta name='viewport' content='initial-scale=1,maximum-scale=1,user-scalable=no'/>
    <script type="text/javascript" src='leaflet.js'></script>
    <link rel="stylesheet" type="text/css" href="leaflet.css">
</head><body>
<div id='map'></div>
<script type="text/javascript">
    var map = L.map('map', {
        center: [40, -110], // change to center at the region you generated the tiles for
        zoom: 5,
        subdomains: []
    });    L.tileLayer('http://localhost:8080/tiles/{z}/{y}/{x}', {
        maxZoom: 18,
        attribution: '(C) 2016 Nitin Pasumarthy'
    }).addTo(map);
</script>
</body></html>

That’s it! You made it. Please add your valuable comments or questions below. Share the article and enjoy!!!

Create your own tile server and map client

Written by Nitin Pasumarthy