Today microcontrollers can be found in almost any technical device, from washing machines to blood pressure meters and wearables. Researchers at the Fraunhofer Institute for Microelectronic Circuits and Systems IMS have developed AIfES, an artificial intelligence (AI) concept for microcontrollers and sensors that contains a completely configurable artificial neural network. AIfES is a platform-independent machine learning library which can be used to realize self-learning microelectronics requiring no connection to a cloud or to high-performance computers. The sensor-related AI system recognizes handwriting and gestures, enabling for example gesture control of input when the library is running on a wearable.
A wide variety of software solutions currently exist for machine learning, but as a rule they are only available for the PC and are based on the programming language Python. There is still no solution which makes it possible to execute and train neural networks on embedded systems such as microcontrollers. Nevertheless, it can be useful to conduct the training directly in the embedded system, for example when an implanted sensor is to calibrate itself. The vision is sensor-related AI that can be directly integrated in a sensor system. A team of researchers at Fraunhofer IMS has made this vision a reality in the form of AIfES (Artificial Intelligence for Embedded Systems), a machine learning library programmed in C that can run on microcontrollers, but also on other platforms such as PCs, Raspberry PI and Android. The library currently contains a completely configurable artificial neural network (ANN), which can also generate deep networks for deep learning when necessary. An ANN is an attempt to mathematically simulate the human brain using algorithms in order to make functional contexts learnable for the algorithms. AIfES has been optimized specifically for embedded systems.
“We’ve reduced the source code to a minimum, which means the ANN can be trained directly on the microcontroller or the sensor, i.e. the embedded system. In addition the source code is universally valid and can be compiled for almost any platform. Because the same algorithms are always used, an ANN generated for example on a PC can easily be ported to a microcontroller. Until now this has been impossible in this form with commercially available software solutions,” says Dr. Pierre Gembaczka, research associate at Fraunhofer IMS.
Protection of privacy
Another uniquely qualifying feature of the sensor-related AI from Fraunhofer IMS: until now artificial intelligence and neural networks have been used primarily for image processing and speech recognition, sometimes with the data leaving the local systems. For example, voice profiles are processed in the cloud on external servers, since the computing power of the local system is not always adequate. “It’s difficult to protect privacy in this process, and enormous amounts of data are transmitted. That’s why we’ve chosen a different approach and are turning away from machine learning processes in the cloud in favor of machine learning directly in the embedded system.
Since no sensitive data leave the system, data protection can be guaranteed and the amounts of data to be transferred are significantly reduced,” says Burkhard Heidemann, “Embedded Systems” group manager at Fraunhofer IMS. “Of course it’s not possible to implement giant deep learning models on an embedded system, so we’re increasing our efforts toward making an elegant feature extraction to reduce input signals.” By embedding the AI directly in the microcontroller, the researchers make it possible to equip a device with additional functions without the need for expensive hardware modifications.
AIfES demonstrator for handwriting recognition. All functions have been integrated on the Arduino UNO, which reads the sensor values of the touchpad, performs number recognition and outputs the result to the display. Credit: Fraunhofer-Gesellschaft
Reducing data
AIfES doesn’t focus on processing large amounts of data, instead transferring only the data needed to build very small neural networks. “We’re not following the trend toward processing big data; we’re sticking with the absolutely necessary data and are creating a kind of micro-intelligence in the embedded system that can resolve the task in question. We develop new feature extractions and new data pre-processing strategies for each problem so that we can realize the smallest possible ANN. This enables subsequent learning on the controller itself,” Gembaczka explains.
The approach has already been put into practice in the form of several demonstrators. If for example the research team implemented the recognition of handwritten numbers on an inexpensive 8-bit microcontroller (Arduino Uno). This was made technically possible by developing an innovative feature extraction method. Another demonstrator is capable of recognizing complex gestures made in the air. Here the IMS scientists have developed a system consisting of a microcontroller and an absolute orientation sensor that recognizes numbers written in the air. “One possible application here would be operation of a wearable,” the researchers point out. “In order for this type of communication to work, various persons write the numbers one through nine several times. The neural network receives this training data, learns from it and in the next step identifies the numbers independently. And almost any figure can be trained, not only numbers.” This eliminates the need to control the device using speech recognition: The wearable can be controlled with gestures and the user’s privacy remains protected.
There are practically no limits to the potential applications of AIfES: For example, a wristband with integrated gesture recognition could be used to control indoor lighting. And not only can AIfES recognize gestures, it can also monitor how well the gestureshave been made. Exercises and movements in physical therapy and fitness can be evaluated without the need for a coach or therapist. Privacy is maintained since no camera or cloud is used. AIfES can be used in a variety of fields such as automotive, medicine, Smart Home and Industrie 4.0.
Decentralized AI
And there are more advantages to AIfES: The library makes it possible to decentralize computing power for example by allowing small embedded systems to receive data before processing and pass on the results to a superordinate system. This dramatically reduces the amount of data to be transferred. In addition, it’s possible to implement a network of small learning-capable systems which distribute tasks among themselves.
Deep learning
AIfES currently contains a neural network with a feedforward structure that also supports deep neural networks. “We programmed our solution so that we can describe a complete network with one single function,” says Gembaczka. The integration of additional network forms and structures is currently in development. Furthermore the researcher and his colleagues are developing hardware components for neural networks in addition to other learning algorithms and demonstrators. Fraunhofer IMS is currently working on a RISC-V microprocessor which will have a hardware accelerator specifically for neural networks. A special version of AIfES is being optimized for this hardware in order to optimally exploit the resource.