ImproveOSM plugin – new features

The Telenav OSM team just released a new version of the ImproveOsm plugin.  We have added a location search box (helpful feature for reaching an area to be mapped) and a button for downloading previously selected ways.

Location search box

A new feature we added to the ImproveOSM plugin is the location search box.  It enables the user to visualize the desired geographical point at a higher zoom level after entering its coordinates in the text field.                     The input of the search box should contain the values for latitude and longitude separated by a comma. If the plugin doesn’t understand your input, an intuitive message is displayed in order to help you. This works similar to the JOSM Jump to position but the search box brings advantages such as the single field for data collecting and more validations for the latitude-longitude values.



Download way button

Another feature is the download way button for the Traffic Flow Direction layer. This enables the user to download the ways of the selected road segments in a new Data layer. This option is only available if the Traffic Flow Detection Layer is active. Furthermore, the button becomes enabled when one or more road segments are selected.

Facebooktwittergoogle_plus

Detecting Traffic Signs in OpenStreetCam

OpenStreetCam’s mission is to help you improve OSM with street-view imagery. Photos taken with regular smartphones seem to be good enough for capturing map features like traffic signs, lanes or crosswalks. However, browsing the 120 million+ photos in OSC to find relevant things to map will take a while. The human factor is fundamental to OSM’s culture and we don’t see that changing, but we want to make editing street related attributes more efficient with automation.

We’re happy to announce a beta release of the traffic signs recognition on OpenStreetCam photos, made possible with machine learning. We processed a few million photos and detected around 500.000 traffic signs so far, currently available for tracks in several areas in United States and Canada. We’re working on extending the training sets and optimize the processing so that the area’s soon expanded.

What’s new from a user perspective: the track page on openstreetcam.org will now show detected traffic signs when available:

There’s a preview list of all detections in the track, detection overlays on photos and, of course, filters. Filters might now get a rep as something really exciting, but we’re excited about one of ours: the OSM status. Here’s why: after detecting a sign we compare it to the corresponding OSM feature and check if they’re consistent. Based on that, filtering is available.

For a practical example, let’s take speed limits: Instead of manually cross checking every detection with the maxspeed tag in OSM, one can only review detections where presumably maxspeed is not set or the value’s different in OSM. Just tick the Need review in OSM box.

Here are a few more examples of trips that have already been processed with our sign detections.

What’s next?

We’re busy working on a few things:

  • Scale the training sets and pipeline to extend the supported areas.
  • Traffic signs integration in the JOSM plugin.
  • Tagging new traffic signs support in the webpage.

If you like what we do and want to help:

  • First and foremost, you can use detections to improve OSM. If you’re seeing detections on tracks check them out, see what needs reviewing in OSM and edit. You can open iD or JOSM to photo’s location straight from the webpage.
  • Help us improve the traffic signs recognition. There’s a chance you will find some bad detections. You can review them and flag whether they’re good or bad, see the two buttons above the photo. We’re adding those reviews to training sets to improve recognitions, so please play nice.
  • Help us add these detections to the iD editor as well.

Tip: you can navigate between detections with Ctrl/Cmd + right/left arrows and confirm/invalidate with Ctrl/Cmd + up/down arrows. Goes pretty fast.

Facebooktwittergoogle_plus

New Features and Enhancements in Cygnus+

Cygnus is the Telenav Mapping conflation tool. We use it a lot internally to compare approved external data sources with existing OSM data, but there is also a public version. We outlined how it works in an earlier blog post. In this post, I want to highlight some of the newer features in Cygnus. These new features are based on the feedback from our team of Map Analysts, who use the tool in their day to day work.

Discarding Very Short Segments

Cygnus outputs the differences in geometry between existing OSM data and the spatial data that we want to use to improve OSM. Sometimes, when the differences are very tiny, Cygnus used to export very short ways. These are not really meaningful enhancements, and clutter up the result data. Therefore, we implemented a length filter. Ways shorter than a defined length threshold will not be included in the output. Based on experience, we set the default to 5 meters. In the internal (command line) version our team uses, this can be tweaked using a parameter. In the public web version, this is not yet possible. We can consider adding it if there is sufficient demand.

An example of Cygnus in action. It finds an opportunity for improvement (possibly incorrect street name) as well as a false positive (degraded road geometry)

Road Names

When comparing road geometry, Cygnus not only compares geometry, but also road names. An annoying side effect we noticed is that road names are often not exactly the same in OSM as they are in the external data we compare with. This does not mean that the external data is necessarily better. For example, OSM could say that the name of a road is “River Road”, and the external data source could say it is “River Rd”. This is not a meaningful difference, and we would want to exclude those in most cases. So we added a string distance based  threshold in Cygnus to filter out similar strings. It is set to a sensible default which, again, can be tweaked in the command line version we use internally, but not yet in the web version.

Another Cygnus improvement related to road names is to ignore name differences on certain types of ways: roundabouts and service roads. Roundabout ways in OSM do not have names by convention, unless the roundabout itself has a name, so they should generally not be added. Service roads technically can have names in OSM, but it is not common. In external data, they do sometimes have names, but if they do, it usually does not make sense to add them to OSM. Based on our experience, they often have descriptive names like ‘driveway’ or ‘access road’ in the source data.

Using Cygnus

You can use Cygnus yourself by going to http://cygnus.improve-osm.org/ and uploading your source data file. You need to do a fair amount of work to prepare the source data: translating the source attributes into valid OSM tags, and converting to OSM PBF. And always remember to consider carefully what you do with the result. Cygnus is not designed to be an automated import tool. Every suggested change should be manually reviewed.

Let us know how you have used, or would like to use Cygnus!Facebooktwittergoogle_plus

Cygnus – conflation at your fingertips!

This is a follow-up blogpost after the State Of the Map US 2017 conference held in Denver.

The process of conflation in GIS is defined as the act of merging two data layers to create one layer containing the features and attributes of both original layers.

Cygnus is a tool that compares external data with OSM, giving you a result file in JOSM XML format with all the changes. The comparison is made in a non-destructive way, so no OSM ways are ever deleted or degraded.

Workflow

NOTE – The license compatibility between the local data file and OSM has to be taken into account before adding anything in OSM. Also, please follow the OSM import procedures if you are planning to add external data to OSM.

First of all you need to have a shapefile with local data in WGS84 spatial reference. This shapefile has to be filtered in different ways, depending on the tags you want to compare. For example, if you want to compare oneways, make sure to have a flow-direction/oneway/etc. attribute in the shapefile.

Translation

The first thing that has to be taken care of is to assure a proper attribute translation. I created a simple example for this exercise. I don’t want to get neck-deep in too many technical details so the main focus remains the process as a whole. I kept the attribute information for this example straightforward:

In order to create an OSM file from this data, I wrote a simple translation file that will be used together with ogr2osm.

Next, run the below command to obtain the OSM file.

python ogr2osm.py simple_streets.shp -t simple_translation.py -o simple_output.osm

Finally, I converted the OSM file to PBF using osmosis, because Cygnus requires a PBF file as input.

Cygnus goes to work!

Now that you have gone through the pre-processing of the local data file, we can offer it to Cygnus for processing. Note that your upload needs to be small-ish – the spatial extent needs to be smaller than 50×50 km and the file needs to be 20MB or smaller in size.

The interface of the Cygnus service is very simple – there are just two pages:

  • the home page where you add new jobs
  • the job queue page where you can see your progress and download the result

If your input file was uploaded successfully, Cygnus will go to work. Your job will be added to the back of the queue. When it’s your turn, Cygnus will read your PBF file, and download the OSM data for the same extent, using Overpass API. It will then compare your upload with the existing OSM data and produce the output file that you can download from the job queue.

NOTE – Everyone’s jobs are listed here, so be careful not to touch other users’ stuff.

Process the output in JOSM

Once Cygnus gives us the output, we can open it in JOSM and inspect it. This is by far the most important, and time consumig, step. Even though Cygnus does a best effort to connect ways where needed, it acts conservatively so it will not snap ways together that do not belong together.

Here are a few ways that got properly connected to the existing highway=secondary:

But there are situations where the distance was too far so Cygnus did not snap:

In this case, you need to manually connect the ways if that is appropriate.

When you are finally satisfied with your manually post-processed conflation result, you can go ahead and merge it with the OSM data and upload it!Facebooktwittergoogle_plus

Check out the new tutorial section

Recently we have launched a video tutorial section on improve.osm.org, to help our users get up to speed with our tools and let you know what are all the possibilities and all the versatility of our products for the community’

So far, we have 2 videos. One is focused on doing a gentle introduction to Improve OSM, some sort of a guided tour of the Improve OSM web site for new users. This is the perfect starting point if you’re new here.

The second one is a step-by-step guide to fixing your first one-way street in OpenStreetMap using the Improve OSM web site. You can see it here.

We’ll prepare more tutorial videos soon. Let us know what other aspects of our products you’d like to be featured in the future.Facebooktwittergoogle_plus

Improve OSM Tool Highlight: Cygnus

Improve OSM is all about making OpenStreetMap better with the help of billions of GPS points that users of Scout apps and others contribute automatically. This allows us to identify missing roads, one-way streets and turn restrictions that are not in OSM yet. However, our OSM engineers think about other ways to help make OSM better as well. Looking at what is going on in the OpenStreetMap community, I see a lot of groups and individuals struggle with the challenge of using external data in OpenStreetMap. Many governments now open up data that was closed before. With proper caution and preparation, we may be able to use this data in OSM.

The technical challenge here is to combine the new, external data with what is already present in OSM. In the GIS world this is called conflation. There are some tools out there to help with this task, but they are hard to use. So we started to think about this from a practical point of view and a specific use case: what if you have an external database of just roads, and you want to add only the new roads to OSM?

cygnus-osm-merge

We developed a tool that can do just that, and we called it Cygnus. I wrote about Cygnus on my OSM diary before, announcing it here and then writing about it being used by OSM user MikeN here. We added a link to the Cygnus web site to the ‘Other tools’ section of the ImproveOSM web site.

We made Cygnus as easy to use as possible, but it is still not straightforward. It should not be. Conflation is hard, and what you are doing is importing data into OSM, which should be done with the utmost caution! If you want to get started using Cygnus, please get in touch. We are happy to help!Facebooktwittergoogle_plus