Supported runtime engines

Live Sense SDK supports multiple instances for executing the Live Sense models. Exact support depends upon the device and model being used.

ML Engine

TFLite is the default ML Engine and can run on any device that meets the minimum requirements.

The ML Engine may have multiple runtimes available. If none are specified, then the SDK will select the best runtime available for the current NNEngine.

Engine runtime support

Model runtime support

Default prioritization

Through experimentation and analysis of accuracy and inference time, a flow has been devised to prioritize the available options for engine and processor for each model. This flow is described in the diagram below.

The checks are performed internally to come up with the best possible configuration.

Usage

To use this feature and let the Live Sense SDK select the best possible option, the models must be initialized without specifying any options.

// Initialize RoadBasicsModel with configuration decided by the SDK
let rbModel = RoadBasicsModel(options: ModelOptions(), rbModelOptions: RoadBasicsModelOptions())

Specify the configuration explicitly

If you do not want to use the priority selected by the Live Sense SDK, you can overwrite it.

setUseDefaultOptions(val:Bool) of ModelOptions is true by default. You can update it to false for selecting options.

To specify the engine and processor explicitly from one of the supported options on which the model should run, define that option during the model initialization.

// Setting Road Basics to run on GPU

var modelOptions : ModelOptions = ModelOptions()
var modelOptions : ModelOptions = ModelOptions()
modelOptions.setUseCoreMLDelegate(val: true)
modelOptions.setUseDefaultOptions(val: false)
let rbModelOptions : RoadBasicsModelOptions = RoadBasicsModelOptions()
let rbModel = RoadBasicsModel(options: modelOptions, rbModelOptions: rbModelOptions)