Many machine learning applications, like Amazon’s Alexa Key Word detector or industrial predictive maintenance solutions, are deployed at the edge. Often applications are not deployed on expensive single-board computers but on smaller, less complex microcontrollers. Typically, these microcontrollers are based on Arm Cortex-M class processors like the Cortex-M4, Cortex-M3 or Cortex-M7. Arm Cortex-M class processors can implement machine learning networks that are not highly optimized. For microcontroller machine learning applications, increasing performance of the deployed inference engine is a balance between the following:
Cycle Bound – The number of processing cycles required.
Memory Bound – The number of memory accesses.
To provide additional context, a MobileNet implementation targeting a single-board computer using TensorFlow will require 3.3 Mbytes of memory and 300 Million Multiply Accumulate Cycles (MMACS). Performance using TensorFlow Lite is far beyond what a microcontroller could provide. A person-detect algorithm can be implemented in 250 Kbytes of memory and circa 7 million MACs. This is much more suitable for implementation in a microcontroller.
Of course, we still need to be able to gather data and create the inference application in order to implement a machine learning inference in a microcontroller. This is where Edge Impulse comes into its own. Founded in 2019, Edge Impulse is a development platform for machine learning on microcontrollers. As such, the web-based development environment enables capture of data, training of the neural network, and generation of the inference engine for deployment.
Getting started with Edge Impulse and creating your first ML application is amazingly simple. If you so desire, we can create our first application with a smart phone. If you do not want to use a smart phone for your first project, there are a several development boards which are fully supported. This means applications are provided for the development board which connect back to the Edge Impulse development platform, thereby enabling data from the board’s sensors to be captured and stored in the development platform. Once the solution is generated, an inference application will be created that demonstrates the machine learning algorithm just created.
This list includes some of the supported development boards.
ST B-L475E-IOT01A
Arduino Nano 33 BLE Sense
Eta Compute ECM3532 AI Sensor
Eta Compute ECM3532 AI Vision
Himax WE-I Plus
Nordic Semiconductor nRF52840 DK
Nordic Semiconductor nRF5340 DK
Silicon Labs Thunderboard Sense 2
OpenMV Cam H7 Plus
Arduino Portenta H7 + Vision Shield
For my first application, I used a ST B-L475E-IOT01A. Although the flow is the same for all boards, the accelerometer was used to capture data samples from the ST board when moved in several different directions or not at all, in order to detect a movement classification inference.
Creating the ML inference solution is simple. We can first create a spectral analysis block that converts the accelerometer inputs to frequency and power spectral density. The configuration of the spectral analysis can be controlled via a new page.
We can then add in the neural network. Again, this can be configured using a new page. Once we are ready, we can save the network and perform the training.
Once the training is completed, we can download the quantized network and visualize the quantized network in netron.app.
We can then download either a library for inclusion in a wider application or download a complete application for one of the supported boards.
Running the provided application on my ST board showed the correct prediction of movement as I moved the board about.
If you want to get up and running with Edge Impulse (and I highly recommend you do), check out this course on Machine Learning and Using Edge Impulse. I found it enjoyable and it covers a significant number of ML basics and how to deploy it using Edge Impulse you can even get a certificate.
You are going to see me doing a lot more with edge impulse over the coming weeks and months!