This is a follow-up blog post after the State Of the Map US 2017 conference held in Denver.
The process of conflation in GIS is defined as the act of merging two data layers to create one layer containing the features and attributes of both original layers.
Cygnus is a tool that compares external data with OSM, giving you a result file in JOSM XML format with all the changes. The comparison is made in a non-destructive way, so no OSM ways are ever deleted or degraded.
Workflow
NOTE – The license compatibility between the local data file and OSM has to be taken into account before adding anything in OSM. Also, please follow the OSM import procedures if you are planning to add external data to OSM.
First of all, you need to have a shapefile with local data in WGS84 spatial reference. This shapefile has to be filtered in different ways, depending on the tags you want to compare. For example, if you want to compare oneways, make sure to have a flow-direction/oneway/etc. attribute in the shapefile.
Translation
The first thing that has to be taken care of is to assure a proper attribute translation. I created a simple example for this exercise. I don’t want to get neck-deep in too many technical details so the main focus remains the process as a whole. I kept the attribute information for this example straightforward:
In order to create an OSM file from this data, I wrote a simple translation file that will be used together with ogr2osm.
Next, run the below command to obtain the OSM file.
python ogr2osm.py simple_streets.shp -t simple_translation.py -o simple_output.osm
Finally, I converted the OSM file to PBF using osmosis, because Cygnus requires a PBF file as input.
Cygnus goes to work!
Now that you have gone through the pre-processing of the local data file, we can offer it to Cygnus for processing. Note that your upload needs to be small-ish – the spatial extent needs to be smaller than 50×50 km and the file needs to be 20MB or smaller in size.
The interface of the Cygnus service is very simple – there are just two pages:
- the home page where you add new jobs
- the job queue page where you can see your progress and download the result
If your input file was uploaded successfully, Cygnus will go to work. Your job will be added to the back of the queue. When it’s your turn, Cygnus will read your PBF file, and download the OSM data for the same extent, using Overpass API. It will then compare your upload with the existing OSM data and produce the output file that you can download from the job queue.
NOTE – Everyone’s jobs are listed here, so be careful not to touch other users’ stuff.
Process the output in JOSM
Once Cygnus gives us the output, we can open it in JOSM and inspect it. This is by far the most important, and time consuming, step. Even though Cygnus does a best effort to connect ways where needed, it acts conservatively so it will not snap ways together that do not belong together.
Here are a few ways that got properly connected to the existing highway=secondary:
But there are situations where the distance was too far so Cygnus did not snap:
In this case, you need to manually connect the ways if that is appropriate.
When you are finally satisfied with your manually post-processed conflation result, you can go ahead and merge it with the OSM data and upload it!
