Streaming pipelines allow you to transform data in stream layers. Examples of these pipelines include pipelines that process vehicle sensor data for information to use immediately or to archive for historical analysis. For more on stream layers, see the Data User Guide.
The figure below illustrates the streaming pipeline creation process.
When you create a streaming pipeline, you need to go through the following steps.
Identify one or more stream layers to serve as your data source. These may be versioned, volatile or streaming layers.
For more information on working with data catalogs, see the Data User Guide.
For example, HERE provides sensor data catalogs based on the Sensor Data Ingestion Interface (SDII). For more details on the SDII catalog data structures, see SDII Data Specification.
Create an output catalog.
The portal provides a UI for managing data catalogs. For more information on creating catalogs, see the Data User Guide.
The OLP CLI offers you command line options for managing data catalogs. For more information on OLP CLI data commands, see the OLP CLI User Guide.
If you prefer to use REST requests, the Data API provides a series of endpoints for managing data catalogs. For more information, see Data API Developer's Guide.
Create your pipeline and write to your output catalog.
The Location Library provides classes and methods for operations such as clustering, map matching, and other features. For more information on the Location Library, see the Location Library Developer's Guide.
You can use the Data Client Library to retrieve your input and write to your output. For more information on using the Data Client Library, see the Data Client Library Developer's Guide.