TinyML Will Drive More Advances in Embedded AI
One of the biggest challenges in developing AI applications has always been access to compute resources. This is one of the main reasons so many AI applications must run on the cloud; it’s the only way to get access to sufficient compute resources for processing data and integrating the results into further decision making in a larger AI-based application. Access to compute has also been a major bottleneck for AI development on microcontrollers and small FPGAs. Microcontrollers are particularly difficult because their sequential + combinational compute structure is very slow for most AI inference tasks.
TinyML is one of the solutions to this problem of implementing AI compute on small microcontrollers and FPGAs. It is both a development concept as well as an actual library available for Python. This article will go over some of the main ideas and development frameworks within the TinyML ecosystem, as well as some hardware options for model implementation.
TinyML Overview
TinyML is a conceptual framework describing implementation of machine learning and AI on small microcontrollers. Ideally, these devices should have the capability to train against new data and perform inference tasks without relying on a connection to the cloud or a data center. This means the models involved will be quite small due to the reduced compute resources available on these devices. Although compute is reduced, these systems can be used to perform highly specific AI inference tasks very quickly.
The key to making TinyML applications work on microcontrollers is that these frameworks should enable AI acceleration. At a high level, this involves reducing the amount of input data being used for inference, and shrinking model sizes to reduce the number of processing steps needed for inference. To perform these tasks, a development framework needs to be chosen for use in an AI-enabled application.
TinyML Development Frameworks
To begin building any application that leverages machine learning and AI, a development framework must be selected.
- TensorFlow Lite: TensorFlow Lite is an open-source machine learning framework developed by Google. TensorFlow Lite supports both C++ and Python programming languages, and the TFLite-Micro library (TensorFlow Lite for Microcontrollers) is designed specifically for use with microcontrollers.
- Arm CMSIS-NN: The Arm CMSIS-NN library is a collection of efficient neural network kernels developed by Arm for deployment on Cortex-M microcontrollers. The library is written in C++.
- uTensor: uTensor is an open-source, lightweight machine learning framework developed by uTensor for deployment on microcontrollers. uTensor supports both C++ and Python programming languages.
- Edge Impulse: Edge Impulse is an end-to-end machine learning platform that allows developers to create, train, and deploy machine learning models on embedded devices. Edge Impulse supports both C++ and Python programming languages.
- PyTorch Mobile: PyTorch Mobile is a machine learning framework developed by Facebook for deployment on mobile and embedded devices. PyTorch Mobile supports both C++ and Python programming languages.
- Keras: Keras is a high-level neural networks API written in Python. While Keras is primarily used for larger-scale machine learning applications, it also has support for deployment on embedded devices, including support for the TensorFlow Lite format.
Within these frameworks, users can implement the required AI acceleration steps needed to fully optimize their models and applications for deployment on edge devices and small microcontrollers. There is another option not listed within the above frameworks for use in Python: the TinyML library.
The TinyML Python Library
In addition to the development frameworks listed above for C/C++ and Python, there is a TinyML library that is specific to Python. To install the library and begin using it in your embedded projects, open a bash terminal and use the command:
|
pip install tinyml |
To import the library into a Python script or application, open your .py file and use the line:
|
import tinyml |
Within the TinyML library, there are many important functions that can be used for building and manipulating ML models for use in AI-driven applications. These are shown in the feature table below.
|
Data preprocessing |
|
|
Model training |
|
|
Model evaluation |
|
|
Model conversion |
Models can be converted to:
|
|
Device deployment |
|
|
Example models |
|
Hardware Platforms Supporting TinyML
There are many platforms that will support TinyML development frameworks, including the following popular options:
- STM32
- ESP32
- TI Sitara processors
- Raspberry Pi
- Arduino
- Xilinx FPGAs + Vivado IDE
- Efinix FPGAs + Efinity
- RISC-V cores instantiated in FPGAs
- Off-the-shelf RISC-V hardware
An important point here is that “support” simply refers to the fact that the proprietary compilers for these products support implementation of MicroPython and/or C/C++, either of which could include some element of AI from the frameworks/libraries mentioned above. Whether these chips can provide the compute required to support your specific application is another matter, and it requires some analysis and testing to determine.
When you’re developing applications for your embedded systems, make sure you use the complete set of system analysis tools from Cadence to evaluate systems functionality. Only Cadence offers a comprehensive set of circuit, IC, and PCB design tools for any application and any level of complexity. Cadence PCB design products also integrate with a multiphysics field solver for thermal analysis, including verification of thermally sensitive chip and package designs.
Subscribe to our newsletter for the latest updates. If you’re looking to learn more about how Cadence has the solution for you, talk to our team of experts.