Spatial ETL

Spatial extract, transform, load (spatial ETL), also known as geospatial transformation and load (GTL), is a process for managing and manipulating geospatial data, for example map data. It is a type of extract, transform, load (ETL) process, with software tools and libraries specialised for geographical information.^[1]

A common use of spatial ETL is to convert geographical information from a data source into another format that can be more easily used, for example by importing it into GIS software.^[1] A tool may translate data directly from one format to another, or via an intermediate format. Intermediate formats are often used when data transformation must be carried out.

Origins and history

Although ETL tools for processing non-spatial data have existed for some time, ETL tools that can manage the unique characteristics of spatial data only emerged in the early 1990s.

Spatial ETL tools emerged in the GIS industry to enable interoperability (or the exchange of information) between the industry's diverse array of mapping applications and associated proprietary formats. However, spatial ETL tools are also becoming increasingly important in the realm of management information systems as a tool to help organizations integrate spatial data with their existing non-spatial databases, and also to leverage their spatial data assets to develop more competitive business strategies.

Traditionally, GIS applications have had the ability to read or import a limited number of spatial data formats, but with few specialist ETL transformation tools; the concept being to import data then carry out step-by-step transformation or analysis within the GIS application itself. Conversely, spatial ETL does not require the user to import or view the data, and generally carries out its tasks in a single predefined process.

With the push to achieve greater interoperability within the GIS industry, many existing GIS applications are now incorporating spatial ETL tools within their products; the ArcGIS Data Interoperability Extension being an example of this.^[2]

Transformation

The transformation phase of a spatial ETL process allows a variety of functions; some of these are similar to standard ETL, but some are unique to spatial data.^[3] Spatial data commonly consists of a geographic element and related attribute data; therefore spatial ETL transformations are often described as being either geometric transformations – transformation of the geographic element – or attribute transformations – transformations of the related attribute data.

Common geospatial transformations

Reprojection: the ability to convert spatial data between one coordinate system and another.
Spatial transformations: the ability to model spatial interactions and calculate spatial predicates
Topological transformations: the ability to create topological relationships between disparate datasets
Resymbolisation: the ability to change the cartographic characteristics of a feature, such as colour or line-style
Geocoding: the ability to convert attributes of tabular data into spatial data

Additional features

Desirable features of a spatial ETL application are:

Data comparison: Ability to carry out change detection and perform incremental updates
Conflict management: Ability to manage conflicts between multiple users of the same data
Data dissemination: Ability to publish data via the internet or deliver by email regardless of source format
Semantic processing: Ability to understand the rules of different data formats to minimize user input whilst preserving meaning

Uses

Spatial ETL has a number of distinct uses:

Data cleansing: The removal of errors within a dataset
Data merging: The bringing together of multiple datasets into a common framework – conflation is a good example of this
Data verification: The comparison of multiple datasets for verification and quality assurance purposes
Data conversion: Conversion between different data formats.

Examples of spatial ETL tools

FME (Feature Manipulation Engine)^[1]
GDAL (Geospatial Data Abstraction Library)^[1]

References

^ ^a ^b ^c ^d "What is ETL… and How Can it Turn You into a Geospatial Rock Star?". XYHt. 4 June 2020. Archived from the original on 6 November 2022. Retrieved 5 November 2022.
^ "Spatial ETL tools". Esri. Archived from the original on 11 April 2023. Retrieved 11 April 2023.
^ Miller, Harvey; Han, Jiawei, eds. (27 May 2009). Geographic Data Mining and Knowledge Discovery. CRC Press. p. 63. ISBN 9781420073980.

[xyht-1] "What is ETL… and How Can it Turn You into a Geospatial Rock Star?". XYHt. 4 June 2020. Archived from the original on 6 November 2022. Retrieved 5 November 2022.

[2] "Spatial ETL tools". Esri. Archived from the original on 11 April 2023. Retrieved 11 April 2023.

[3] Miller, Harvey; Han, Jiawei, eds. (27 May 2009). Geographic Data Mining and Knowledge Discovery. CRC Press. p. 63. ISBN 9781420073980.

[1]

[2]

[3]