Create a Catalog

If you have data you want to bring to the HERE platform, you need to create a catalog to contain the data. A catalog is a collection of data that is logically managed as a single set. Catalogs usually contain layers that change together and depend on each other. For example, a catalog may contain a map that includes layers for road attributes, topology, and signs.

This topic provides an overview of how to create a catalog using the REST service config. For complete information on using the config service, see the API Reference.

Choose a Layer Type

Before creating a catalog you must decide which type of layers you want in the catalog: versioned, volatile, or streaming. Here's a summary of the characteristics of each layer type.

Layer Type Uses
Versioned Use a versioned layer for slowly-changing data where you want to provide access to the current version of the data as well as prior versions. An example use of a versioned layer is for historical weather data.
Volatile Use a volatile layer when you only need to provide access to the latest data. When data is updated, the prior data is overwritten and can no longer be accessed. An example use of a volatile layer is for live weather data.
Stream Use stream layers for a continuous stream of messages or events that can be consumed as they are published. An example use of a stream layer is for sensor data from vehicles.
Index Use index layers to index and store metadata and data in a way that is optimized for batch processing. An example use of an index layer is for archiving sensor data for use in a weekly data analytics job.

For complete information about layer types, see Layers.

Choose a Schema

Schemas define the organization of data in a layer, including the structure of the data and its content. When you create a layer, you can specify a schema for the layer. You can leverage any of the existing schemas available in the HERE platform, assuming you can find one which represents the same data type and construct of your data. If none of the existing schemas match your data, you can create your own schema. For instructions, see the Data User's Guide.

We recommend that you specify a schema for each layer in your catalog, although it is possible to not have a schema for a layer. For more information, see Schemas.

Create the Catalog

Once you have decided on the kind of layers you want the catalog to contain, you are ready to create the catalog.

  1. Obtain an authorization token. For instructions, see the Authentication and Authorization API Guide.
  2. Use the API Lookup service to get the API endpoint for the config API. For instructions, see the API Lookup Developer's Guide.
  3. Use this request to the config API to create the catalog. This request creates a catalog containing one new layer, but you can create more than one layer if needed. Typically you create new layers in the same request that creates the catalog, although it is possible to create a catalog with no layers, and create layers separately.

    Note

    For complete information on using the config service, see the API Reference.

    POST /<Base path for the config API from the API Lookup Service>/catalogs HTTP/1.1
    Host: <Hostname for the config API from the API Lookup Service>
    Content-Type: application/json
    Authorization: Bearer <Authorization Token>
    Cache-Control: no-cache
    {
     "description": "<Describe the content and purpose of the catalog>",
     "id": "<A unique identifier>",
     "name": "<Catalog name>",
     "notifications": {
     "enabled": false
     },
     "summary": "<A short description of the content and purpose of the catalog>",
     "layers": [
       {
         "id": "<A unique ID for the layer>",
         "name": "<A user-friendly name to use for display purposes>",
         "summary": "<A short description of the content and purpose of the layer>",
         "description": "<A long description of the content and purpose of the layer>",
         "schema": {
             "hrn": "<The schema's HERE Resource Name>"
         },
         "contentType": "<The media type>",
         "contentEncoding": "<Whether to compress data>",
         "partitioning": {
           "scheme": "<The method to use to divide data in the layer into partitions>",
         },
         "layerType": "<The type of layer: versioned, volatile, or stream>",
         "volume": {
           "volumeType": "durable",
           "maxMemoryPolicy": "failOnWrite",
           "packageType": "small"
         }
       }
     ]
    }
    

    Note: Catalog IDs

    Catalog IDs are publicly visible since they are part of the catalog HRN. Do not include private or company confidential information when you specify a catalog ID. Catalog names are private by default, which means you can add your private or confidential information in this field.

    You can specify configuration settings for volatile and stream layers. Versioned layers have no configuration settings. For more information, see:

    The response element href contains a URL where you can get the status of the catalog creation process. For example:

    {
     "title": "Catalog Creation",
     "href": "https://config.data.api.platform.hereolp.cn/config/v1/status/da167243-13cd-1312-1b72-2z5d2dea7460",
     "type": "urn:olp-types:status"
    }
    

    The URL returns the status while the catalog is being created. For example:

    {
     "status": "pending"
    }
    

    If you try creating a catalog using an ID that already exists, you will get a 409 Conflict response.

Once the catalog is created, the URL returns the catalog and layer metadata:

{
    "id": "<The identifier you specified in the create request>",
    "hrn": "<The HERE Resource Name of the catalog>",
    "name": "<The catalog name you specified in the request>",
    "summary": "<The summary you specified in the request>",
    "description": "<The description you specified in the request>",
    "owner": {
        "creator": {
            "id": "<The App ID associated with the credentials used in the create request> "
        },
        "organisation": {
            "id": "<The realm of the user ID used in the create request>"
        }
    },
    "tags": [],
    "billingTags": [],
    "created": "<The time stamp indicating when the catalog was created>",
    "layers": [
        {
            "id": "<The layer ID>",
            "hrn": "<The HERE Resource Name of the layer>",
            "name": "<The layer name>",
            "summary": "<The summary description of the purpose and content of the layer>",
            "owner": {
                "creator": {
                    "id": "<The App ID used to create the layer>"
                },
                "organisation": {
                    "id": "<The realm in which the layer was created>"
                }
            },
            "partitioningScheme": "<The partitioning scheme of the layer, HEREtile or generic>",
            "partitioning": {
                "tileLevels": [
                    0
                ],
                "scheme": "heretile"
            },
            "contentType": "application/json",
            "contentEncoding": "gzip",
            "volume": {
                "volumeType": "durable"
            },
            "tags": [],
            "billingTags": [],
            "created": "2018-01-29T16:16:55.402Z",
            "layerType": "<The layer type: versioned, volatile, or stream>"
        }
    ],
    "version": 0,
    "notifications": {
        "enabled": false
    }
}

With this metadata, specifically the layer HRNs, you can publish data to the catalog.

Note

It may take several minutes for a new catalog to appear in the list of catalogs returned by the Data Client Library, CLI, REST API, and the list displayed on the HERE platform portal.

Example

Here is an example of how to create a catalog. In this example, a new catalog named "My Traffic Catalog" with one layer named "Example Layer". They layer is a versioned layer.

HTTP
curl
POST /config/v1/catalogs HTTP/1.1
Host: config.data.api.platform.hereolp.cn
Content-Type: application/json
Authorization: Bearer <Authorization Token>
Cache-Control: no-cache

{
  "description": "This is a test catalog.",
  "id": "my-catalog",
  "name": "My Traffic Catalog",
  "notifications": {
  "enabled": false
  },
  "summary": "This is a test catalog.",
  "layers": [
    {
      "id": "my-layer",
      "name": "Example Layer",
      "summary": "The traffic incident layer contains information about events that are affecting the flow of traffic or that may be important for drivers to know.",
      "description": "This layer provides aggregated information about traffic incidents, including the type and location of each traffic incident, status, start and end time, and other relevant data. This data is useful to dynamically optimize route calculations. It also provides access to real-time traffic flow data in including information on speed and congestion for specific regions.",
      "schema": {
          "hrn": "hrn:example:schema:::traffic-schema"
      },
      "contentType": "application/json",
      "contentEncoding": "",
      "partitioning": {
        "scheme": "heretile"
      },
      "layerType": "versioned",
      "volume": {
        "volumeType": "durable"
      }
    }
  ]
}
curl -X POST https://config.data.api.platform.hereolp.cn/config/v1/catalogs \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <Authorization Token>' \
-H 'Cache-Control: no-cache' \
-d '{
  "description": "This is a test catalog.",
  "id": "my-catalog",
  "name": "My Traffic Catalog",
  "notifications": {
  "enabled": false
  },
  "summary": "This is a test catalog.",
  "layers": [
    {
      "id": "my-layer",
      "name": "Example Layer",
      "summary": "The traffic incident layer contains information about events that are affecting the flow of traffic or that may be important for drivers to know.",
      "description": "This layer provides aggregated information about traffic incidents, including the type and location of each traffic incident, status, start and end time, and other relevant data. This data is useful to dynamically optimize route calculations. It also provides access to real-time traffic flow data in including information on speed and congestion for specific regions.",
      "schema": {
          "hrn": "hrn:example:schema:::traffic-schema"
      },
      "contentType": "application/json",
      "contentEncoding": "",
      "partitioning": {
        "scheme": "heretile"
      },
      "layerType": "versioned",
      "volume": {
        "volumeType": "durable"
      }
    }
  ]
}'

Once the catalog is created, a request to the status URL returns:

{
    "id": "my-catalog",
    "hrn": "<Catalog HRN>",
    "name": "My Traffic Catalog",
    "summary": "This is a test catalog.",
    "description": "This is a test catalog.",
    "owner": {
        "creator": {
            "id": "Mejk6DMxAq7kKI5lt2T2"
        },
        "organisation": {
            "id": "olp-here-test"
        }
    },
    "tags": [],
    "billingTags": [],
    "created": "2018-01-29T16:16:55.402Z",
    "layers": [
        {
            "id": "my-layer",
            "hrn": "<Catalog HRN>",
            "name": "Example Layer",
            "summary": "The traffic incident layer contains information about events that are affecting the flow of traffic or that may be important for drivers to know.",
            "description": "This layer provides aggregated information about traffic incidents, including the type and location of each traffic incident, status, start and end time, and other relevant data. This data is useful to dynamically optimize route calculations. It also provides access to real-time traffic flow data in including information on speed and congestion for specific regions.",
            "owner": {
                "creator": {
                    "id": "Mejk09M4Aq71KI5l1t2T2"
                },
                "organisation": {
                    "id": "olp-here-test"
                }
            },
            "partitioningScheme": "heretile",
            "partitioning": {
                "tileLevels": [
                    0
                ],
                "scheme": "heretile"
            },
            "contentType": "application/json",
            "contentEncoding": "gzip",
            "volume": {
                "volumeType": "durable"
            },
            "tags": [],
            "billingTags": [],
            "created": "2018-01-29T16:16:55.402Z",
            "layerType": "versioned"
        }
    ],
    "version": 0,
    "notifications": {
        "enabled": false
    }
}

results matching ""

    No results matching ""