Hands On

Introducing Search to HERE XYZ

By Raymond Camden | 11 December 2019

Introducing Search to HERE XYZ

Recently we added a major new feature to HERE XYZ, the ability to perform searches against your geographic data. This is a fairly complex feature and while we document it of course, I've spent the last few days digging deep into this and wanted to share some additional context and sample code to (hopefully!) make this simpler for our developers. As always, you can leave me a comment below if you've got any follow up questions. All of the code shown here is also available on a (rough) GitHub repo I've setup for various different HERE APIs. I'll share that link at the end. Alright, ready for some awesome deep search talk? Buckle up...

Search via Tagging

Before we get into the new hotness that is property search, remember that previously we've supported searching by use of tagging. While technically not a "search" per se, by using tags with your features you can return subsets of your data. So for example, given a set of cats stored in HERE XYZ, you could use tags to represent different breeds of cats, or as a way to flag cats available for adoption. The existing search API endpoint supports passing in a list of tags.

So for example:

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?tags=ragdoll,fluffy

Would return features in match either ragdoll or fluffy tags. To filter by tags and require all of them, you could do:

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?tags=ragdoll,adoptable

As a way of matching both the breed and their ability to be adopted.

This works but isn't really flexible and requires you to ensure you properly tag your content.

Working with Search - Initial Setup

A wise woman once said - to search, you first need something to search. For my tests, I wrote a Node script that generated a GeoJSON file of cats. Each cat was located in a random city and had the following properties:

  • name: A random name.
  • gender: Either male or female.
  • age: A random number before 1 and 10.
  • breed: A random breed.
  • available: A boolean value representing if the cat is available for adoption.
  • skills: A nested object with two properties, weapons and cooking. Both have random values representing what weapons that cat is skilled with and what types of food that cat has experience with. Yes, cats.
  • totalFriends: An array of named cats that are friendly with this cat.

Here's two examples:


"properties": {
	"name": "Zombified Butterfly Wings",
	"gender": "female",
	"age": 5,
	"breed": "toyger",
	"available": true,
	"friends": [
		"Silly Kitten Zombie"
	],
	"skills": {
		"weapons": "katanas",
		"cooking": "cookies"
	}
}

"properties": {
	"name": "Lord Dumpster Fire",
	"gender": "female",
	"age": 3,
	"breed": "siamese",
	"available": false,
	"friends": [
		"Sir Juniper"
	],
	"skills": {
		"weapons": "broadswords",
		"cooking": "cakes"
	}
}

If you're interested in the script that generated this data, you can find it here: https://github.com/cfjedimaster/heredemos/blob/master/xyz_search/generateCatGeoJSON.js I took the output of this script, with different total number of features, and uploaded it into multiple spaces for testing.

So you have data in an HERE XYZ space. You've enabled the PRO set of features. What's next? First you need to figure out what properties you can search. This is a bit interesting and may be hard to follow at first, so let's take it slow.

First - you can search all properties that contain simple values, but not arrays. Arrays are actually marked as searchable (more on that in a second), but you can't search them yet.

Second - when you have more than ten thousand features in a space, HERE XYZ takes a look at your data, and based on advanced super-intelligent machine learning AI code, will make a determination if every property needs to be searchable. In other words, given five properties, it's possible HERE XYZ may decide that one property isn't used often and therefore shouldn't be searchable.

Luckily, we provide both an API to determine what's searchable as well as a way to change what's searchable as well. And in fact, you may specifically want to disable search on properties you know do not need it.

To determine what XYZ considers searchable in your space, make a call to the statistics endpoint with your space ID, like so:

https://xyz.api.here.com/hub/spaces/SPACE_ID/statistics

Here's a simple example of this:


/*

This script runs the statistics endpoint on a hard coded space (should be moved to .env, will do so now)
which reports on properties and their searchability.
*/

require('dotenv').config();

const fetch = require('node-fetch');
const ACCESS_TOKEN = process.env.ACCESS_TOKEN;
const SPACE_ID = process.env.SPACE_ID;

fetch(`https://xyz.api.here.com/hub/spaces/${SPACE_ID}/statistics?access_token=${ACCESS_TOKEN}`)
.then(res => res.json())
.then(res => { 
	console.log(JSON.stringify(res.properties, null, '\t'));
});

The result (when focused on properties as you see above) will be an object that looks like so:


{
	"value": [
		{
			"key": "breed",
			"datatype": "string",
			"count": 2000
		},
		{
			"key": "gender",
			"datatype": "string",
			"count": 2000
		},
		{
			"key": "name",
			"datatype": "string",
			"count": 2000
		}
	],
	"estimated": false, 
	"searchable": "ALL"
}

Note the final value, searchable. This will be one of three values:

  • ALL - Every property is searchable.
  • PARTIAL - Some properties are searchable.
  • NONE - No properties are searchable.

If you get PARTIAL as a result, how do you know what's searchable? In this case, the list of properties will include a flag for searchable properties:


{
		"key": "age",
		"datatype": "number",
		"count": 12000,
		"searchable": true
},
{
		"key": "available",
		"datatype": "boolean",
		"count": 12000
},
{
		"key": "breed",
		"datatype": "string",
		"count": 12000,
		"searchable": true
},
{
		"key": "friends",
		"datatype": "array",
		"count": 12000
}

In the example above, the presence of searchable on age and breed mean they can be searched, but available and friends can not. (Note - currently our docs are a bit outdated in that they show "searchable": false. The absence of the property implies falsehood. I've filed a bug report to make the docs right as I personally think the value should always be present. Again, this is a new feature so pardon the confusion!)

If you wish to change what can be searchable, you need to use the PATCH method against the space. This is covered in the documentation and the only thing I'd add is that you should consider using this both for times when you want to enable a property to be search but also when you want to disable a property as well. The less work XYZ has to do with your data the better it can perform! Note that the ability to change what is searchable is currently a PRO feature only

Alright, so just to be clear, we've covered how to enable search and how to figure out what properties can be searched. As well as how to possibly change those if we aren't happy. Now that we've done that, how do we search?

A basic search query looks like so (remember that SPACE_ID should be replaced with the ID of your search):

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?p.something=value

I'm asking the API to search against the something property and match against value. That's an equality check. You can also use other operators:

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?&p.age<=3

The above example will look for features where the age property has a value less than or equal to 3. You can use the following operators currently:

  • = for equals
  • != for not equals
  • >= or =gte= for greater than or equal to
  • > or =gt= for greater than
  • <= or =lte= for less than or equal to
  • < or =lt= for less than

You can search against multiple properties by passing multiple p. parameters.

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?p.age<=3&p.breed=ragdoll

The above example will do an AND search and require both properties to match the operators given. To do an OR search where you are ok with either value matching on a property, you can use this form:

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?p.breed=ragdoll,devon%20rex

The result set from your searches will be standard GeoJSON with the features array limited to items that match your choice. You can modify this slightly though by asking for a filtered set of properties.

As a final example, you can also perform a filter on the proeprties returned in the matched filters. To do this, specify the properties you want with the selection argument. For example:

https://xyz.api.here.com/hub/spaces/SPACE_ID/search?access_token=${ACCESS_TOKEN}&p.breed=ragdoll&selection=p.breed

This example will search against the breed property and filter the returned properties to just the breed value. Here's an example of how one result may look.


{
        "type": "Feature",
        "id": "9fa81d26b8064092f913aa1024b4e309",
        "geometry": {
                "type": "Point",
                "coordinates": [
                        -112.0740373,
                        33.4483771
                ]
        },
        "properties": {
                "breed": "ragdoll"
        }
}

Searching on the Client

So far you've seen how to use the REST-based API to work with search. You can apply the same property-based searches to the JavaScript library as well. You can either apply it at the constructor level:


new here.xyz.maps.providers.SpaceProvider({
    name: 'SpaceProvider',
    level: 1,
    space: 'SPACE_ID',
    credentials: {
        access_token: YOUR_ACCESS_TOKEN
    },
    propertySearch:{
        'name': {
            operator: '=',
            value: ['Max','Petra']
        },
        'age': {
            operator: '<',
            value: 32
        }
    }
})

You can also dynamically change property search values by using the setPropertySearch API. Let's consider a full example of how this could look. First, here's a simple implementation that renders our cat data.


//specify your credentials to spaces
var YOUR_ACCESS_TOKEN = "USE REAL VALUE"; 

// configure layers
var layers = [
	new here.xyz.maps.layers.TileLayer({
		name: 'Image Layer',
		min: 3,
		max: 10,
		provider: new here.xyz.maps.providers.ImageProvider({
			name: 'Live Map',
			url: 'https://1.mapcreator.tilehub.api.here.com/tilehub/wv_livemap_bc/png/sat/256/{QUADKEY}?access_token=' + YOUR_ACCESS_TOKEN
		})
	})
]

var mySpaceProvider = new here.xyz.maps.providers.SpaceProvider({
	// Name of the provider
	name: 'SpaceProvider',

	// Zoom level at which tiles are cached
	level: 3,

	// Space ID
	space: 'TjJw79XK',

	// User credential of the provider
	credentials: {
		access_token: YOUR_ACCESS_TOKEN
	}
});

var myLayer = new here.xyz.maps.layers.TileLayer({
	// Name of the layer
	name: 'mySpaceLayer',

	// Minimum zoom level
	min: 3,

	// Maximum zoom level
	max: 10,

	// Define provider for this layer
	provider: mySpaceProvider
})

window.display = new here.xyz.maps.Map(document.getElementById("map"), {
	zoomLevel: 4,
	center: {
		longitude: -96, latitude: 37.73
	},
	// add layers to display
	layers: layers
});

display.addLayer(myLayer);

Now, let's add a bit of interactivity. First, I'll add a dropdown:


<p>
	Breed: 
	<select id="breed">
		<option></option>
		<option value='persian'>Persian</option>
		<option value='bengal'>Bengal</option>
		<option value='siamese'>Siamese</option>
		<option value='maine coon'>Maine Coon</option>
		<option value='sphynx'>Sphynx</option>
		<option value='ragdoll'>Ragdoll</option>
		<option value='birman'>Birman</option>
		<option value='chartreux'>Chartreux</option>
		<option value='toyger'>Toyger</option>
		<option value='devon rex'>Devon Rex</option>
	</select>
</p>

Then JavaScript to listen to changes to the dropdown.


let breedSelect = document.querySelector('#breed');
breedSelect.addEventListener('change', (e) => {
	let currentVal = breedSelect.querySelector('option:checked').value;
	console.log(currentVal);
	if(currentVal === '') mySpaceProvider.setPropertySearch(null);
	else mySpaceProvider.setPropertySearch('breed', '=', currentVal);
}, false);

Notice the two branches based on whether or not the first, empty, option was selected. Clearing a search filter is done by passing null to setPropertySearch. Otherwise you pass the property, comparator, and the value. Here's an example of the map before a filter has been applied:

map1

And here's how it looks with a filter:

map2-1

Wrap Up

I hope this article helps you understand how to use this powerful new feature of XYZ. As always, if you've got questions, leave a comment below. You can find the complete set of demos I used here: https://github.com/cfjedimaster/heredemos/tree/master/xyz_search Note that you will need to make a .env file with your own key/space values in order to run the demos.