Hootenanny:
- A gathering at which folksingers entertain often with the audience joining in
Conflation:
- Fancy word for merge
Hootenanny is an open source map data conflation tool developed with machine learning techniques to facilitate automated and semi-automated conflation of critical Foundation GEOINT features in the topographic domain. In short, it merges multiple maps into a single seamless map.
Goals:
- Automatically combine geospatial features for decision making
- Allow for reviewing and manually resolving features which cannot be automatically matched with sufficient certainty
- Maintain geometry and attribute provenance for combined features
- Create up-to-date routable transportation networks from multiple sources
For installation from RPM, please follow the guide in the rpms repo. If you want to install from source without downloading the release, please follow VAGRANT.md.
Documentation is in the file named: Hootenanny - User Guide.pdf, located in the .tar.gz installation
file under the "docs" directory.
Hootenanny has specifically tailored conflation algorithms available for the following data types:
- Areas
- Buildings
- Points of Interest (POIs)
- Power Lines
- Railways
- Rivers
- Roads
Any feature whose type does not fit into the list above or has no type at all, will be conflated with Generic Geometry Conflation, which uses a simpler approach than the type specific conflation algorithms.
You can create your own custom conflation algorithms for additional feature types via Javascript or C++.
Conflation Workflows
A conflation workflow defines the manner in which two maps are merged together. Hootenanny has the following workflows:
- Reference Conflation (default; aka Vertical Conflation) - Keep the best of both maps while favoring the first
- Use this type of conflation when you want map output based on the best state of two maps while favoring the first one.
- Average Conflation - Keep an average of both maps
- Use this type of conflation when you consider both input maps equal in quality and want a result that is an average of the two.
- Currently, geometry averaging only applies to linear features but could be extended to point and polygon geometries. Point and polygon geometries are merged the same as in Reference Conflation.
- Average Conflation is currently not available from iD Editor.
- Horizontal Conflation (aka Cookie Cutter Conflation) - Completely replace a section
- Use this type of conflation if you have a specific region of your map that you would like to completely replace with a region from another map.
- Differential Conflation - Add new features that do not conflict
- Use this type of conflation when you want to fill holes in your map with data from another source without actually modifying any of the data in your map.
- There is an option available (
--include-tags) to additionally transfer tags to existing features in your map from matching features in another map where overlap occurs.
- Attribute Conflation - Transfer attributes over to existing geometries
- Use this type of conflation when one map's geometry is superior to that of a second map, but the attributes of the second map are superior to that of the first map.
Hootenanny leverages the OSM key value pair tag concept to support translation between various data schemas and supports the following schemas:
- Topographic Data Store (TDS)
- Multi-National Geospatial Co-Production Program (MGCP)
- Geonames
- OSM
- others
Users can define their own custom schema translations via Javascript or Python.
Hootenanny has the capability to let you selectively pick the features that are conflated from your data to save you from pre-conflation data wrangling.
Some examples:
- Conflate only buildings
- Conflate only restaurant buildings
- Conflate only restaurant buildings with "Subway" in the name
Hootenanny has several available filters that may be used to perform feature filtering during conflation.
Hootenanny has limited data validation reporting of its own, and JOSM validation routines may also be invoked via Hootenanny during batch processing of data.
Hootenanny has a variety of map cleaning capabilities to automatically correct erroneous data. Some of them are invoked automatically during a conflation job, however, you may also run them separately. In addition to its own built-in cleaning operations, Hootenanny is also integrated with the auto-fix capabilities of JOSM.
Although Hootenanny is configured by default to provide the best conflation results, at times you will need to tweak its wide range of configuration options in order to get the best conflated result for your input data.
Hootenanny's web user interface is built upon the open source Mapbox iD Editor, which provides an intuitive and user-friendly conflation experience.
Access to Hootenanny core capabilities are exposed through a web services API for those wishing to develop their own conflation clients. The web services use OAuth authentication.
Basic conflation example:
# conflate two datasets together
hoot conflate input1.osm input2.osm output.osm
Hootenanny has nodejs bindings available which expose core conflation capabilities for creating custom workflows.
Basic conflation example:
// conflate two datasets together
var hoot = require(process.env.HOOT_HOME + '/lib/HootJs');
var map = new hoot.OsmMap();
hoot.loadMap(map, "input1.osm", false, 1);
hoot.loadMap(map, "input2.osm", false, 2);
new hoot.UnifyingConflator().apply(map)
hoot.saveMap(map, "output.osm");
In addition to conflating map data, Hootenanny also provides these supporting capabilities that may aid conflation:
- Add missing type tags to features implicitly based on their name (experimental)
- Align the geometries of two maps more closely with each other
- Apply data transformation operations to a map
- Calculate the geospatial extent of a map
- Clean map data
- Compare maps with each other
- Convert maps between different geodata formats
- Create a covering shape that minimally covers all features in a map
- Crop a map to a geospatial extent
- De-duplicate or find duplicated features within maps (experimental)
- Derive changesets between maps and push through the OpenStreetMap API
- Detect spoken languages in a map's tag data (experimental)
- Explore tag data
- Filter a subset of features before conflation based on tag content and schema relationships
- Find street intersections by name
- Generate a task grid
- Generate pre-configured or custom map statistics
- Grab a random section of data from a map for testing against (experimental)
- Identify street intersection locations in a map
- Locate phone numbers geographically
- Replace or remove data completely within a bounding box via changeset and push the changes through the OpenStreetMap API
- Perturb map data for testing purposes
- Plot node density
- Process map postal address data
- Simplify ways by removing unnecessary nodes
- Snap unconnected ways to neighboring ways
- Sort map data
- Split a map into tiles
- Translate feature tags using standardized or user defined schemas
- Translate feature tags to English (experimental)
- Validate features