In a little over a decade since researchers uncovered novel techniques to improve its efficiency and effectiveness, deep learning has become a practical technology that now underpins a number of applications that need artificial intelligence (AI).
Many of these applications are hosted in the cloud on powerful servers, as tasks sometimes involve the processing of data-rich sources such as images, videos, and audio. Those servers often call on the additional performance of acceleration hardware, which range from graphics processing units to custom devices. These become particularly important for the numerically-intensive process, during which a neural network is trained on new data.
Typically, the inferencing process, which uses a trained network to assess new data, is far less compute-intensive than training. There are also workloads that involve less data-intensive sources, such as sensor readings from IoT devices, where both training and inferencing can be performed on lower-performance hardware. As a result, systems designers are finding that workloads do not necessarily need to be located in the cloud once the AI model has been trained, though many current services still do for business reasons. Instead, the trained model can be transferred to a local machine for processing closer to the source of the data.
AI Benefits At The Edge
There are a number of reasons to bring AI models (Figure 1) closer to the edge of the network, most notably privacy and user acceptance. For example, consumers using devices like smart speakers are becoming increasingly concerned about having their private conversations recorded and uploaded to cloud services.
In industrial control applications, which use AI for machine-condition monitoring or process optimization, concerns over the confidentiality of production data will similarly demand that as little as possible is transferred to the cloud. For many industrial applications, there are concerns about the reliability and speed of the connection to the cloud. Many systems, whether on a shop floor or situated in a remote location, do not have the high-bandwidth connections required to support cloud-based inferencing.
Control systems are also adversely affected by high communications latency. If AI models are used within closed-loop control systems, any delay in fetching an update from the cloud will lead to inaccuracies and instability. Some systems may use a mixture of cloud and local processing. Surveillance cameras, for example, will preserve network bandwidth by identifying immediate threats, locally but then call on the cloud to perform additional processing when local models cannot cope.
Battery life is also significantly increased when AI sits at the edge, as far less data is sent over the network, reducing both network and cloud costs.
AI In Predictive Maintenance
Predictive maintenance is currently the most popular use-case for connected-industry applications. It provides a high return on investment due to its ability to reduce the frequency of on-site inspections. Machine down-time can also be reduced by accurately identifying a component’s remaining useful time in service, making it possible to exploit its operational life without risking a failure during operation. Prediction of lifetime, based on operational data, also helps optimize maintenance schedules and accurately identify correct spares requirements. Customers using predictive maintenance already experience efficiency gains of 20 to 25 percent.
With predictive maintenance, sensors in a machine tool may detect changes in temperature, coupled with an increase in noise or vibration, picked up through a combination of microphones and accelerometers, as indicative of a potential problem. With traditional algorithmic techniques, it can be extremely difficult to relate combinations of real-time sensor readings to problems. AI-based models can interpret time-series data together with real-time inputs to determine the health of the system accurately. Such models can take advantage of techniques such as sensor fusion to determine whether the situation demands maintenance. The complexity and latency requirements of the model will determine the feasibility of running the model locally.
Approaches To Local Processing
There are two approaches to local processing. The first is to use the processing power available on the device itself. This depends on the complexity of the AI model and the available processing power. For low-power sensor nodes, it will not be possible to run a large deep-learning model even for just inferencing. As machine-learning algorithms can take many forms, it may not be necessary to run such a heavyweight task completely locally.
The second option is to offload processing to another device either completely or partially. For example, the device may run a simplified AI model that performs an initial analysis of the data. In the case of a surveillance system that listens for activity, this model may simply perform the job of determining whether the signal is background noise, such as wind, or due to an intruder passing nearby, or the sudden sound of a breaking window. More complex functions can be transferred to a gateway that concentrates data from multiple cameras and other security devices. The devices may provide some preprocessing to help streamline the job of the gateway model.
Powerful hardware, such as the Xilinx programmable-logic devices provided by the Ultra96 platform, can perform inferencing based on sophisticated deep-learning models and other complex machine-learning algorithms. The local gateway may even update the model by collecting data and performing training in batches when the inferencing workload is light. Alternatively, the gateway can collect data that is important to retraining and pass that to a cloud server daily, weekly, or monthly.
Learning Technology Options
Developers can choose which types of model to employ for a specific application. Early AI models made use of structures, such as decision trees or expert systems, which demanded a large amount of effort from domain specialists to relate combinations of inputs to probable causes and outcomes.
Machine learning enhanced decision trees by making techniques, such as random forests, possible. Such an approach builds multiple decision trees from training data and applies them in parallel to compute an average that represents the most likely outcome based on the training data. A random forest is an example of a supervised learning system; it relates data provided by the system’s developers to the inputs from which the machine-learning model learns relationships.
Deep learning is another example of a supervised learning technology, as it relies on the training data being labelled beforehand. In an image-classification system, for example, the model takes in data from the image and applies it to a layer of simulated neurons. The outputs from the first layer of neurons are passed successively through many more layers. Some of them combine the outputs from multiple neurons in the preceding layer to produce a single value that is passed onto the next. In this way, deep neural networks perform dimension reduction: a vital step when converting complex multidimensional data such as imagery or audio into a form that can be used to provide a final classification. The neural network is able to perform classification, because it can learn how different arrangements of coefficients in the neural layers respond to various images to provide outputs that match the training labels.
An unsupervised machine-learning system can find patterns in the combinations of sensor readings. These patterns can reduce the amount of data that is then fed to a supervised algorithm that is trained on machines operating under different types of stress and varying states of repair. As the clustering helps reduce the amount of data that needs to be transferred, one useful design pattern for implementation in IoT systems is to perform this activity on the device node. The model derived from supervised learning can run on the gateway or even in the cloud depending on the tradeoffs between latency, performance, and cost.
AI in Action
With machine learning as a technology, its implementation often requires in-depth knowledge and experience to make the most of the different forms of AI. Octonion has developed a software solution to this problem which is built into Avnet’s SmartEdge Agile device: allowing engineers to develop AI solutions for IoT systems that involve significant edge processing without hiring expensive help or acquiring in-depth expertise themselves.
The solution (Figure 2) runs at three levels: device, gateway, and cloud. The device hardware, the Avnet SmartEdge Agile, is an autonomous, low-power sensor node. This device provides access to a variety of sensor types that include accelerometers, gyroscopes, and magnetometers, in addition to sensors measuring pressure, temperature, humidity, and proximity. A microphone is available to record audio inputs.
The smart-sensor hardware communicates with a local gateway using BLE. Versions that support other IoT-oriented network technologies such as LoRaWAN, SigFox, cellular, and WiFi are under development. The gateway module can take the form of an Android or iOS smart device; or alternatively, the gateway software can run on a Raspberry Pi or similar Linux platform device that can provide access to the cloud. The cloud layer can be deployed on AWS, Microsoft Azure, or custom server solutions.
Coordination of the various layers is provided through Octonion’s Brainium software, which provides edge-to-cloud support in a zero-code environment. Security is key to all IoT deployment and an important element of the system to protect the privacy of the data used to train models as well as during inferencing. This includes AES encryption together with hardware-based tamper-resistant cryptography key storage, encryption of firmware images with digital signature validation, and TLS encryption to protect the messages from the edge into the cloud.
Octonion’s Brainium software environment provides a combination of unsupervised and supervised machine learning. For example, it can be trained on anomalies in the time series representation of raw sensor data. Additionally, it will cluster data from different scenarios to identify common data patterns. Those data patterns can be pulled into a flexible model builder within the AI Studio software so that developers can tune the model for the target use-case.
When harnessing smart data computation at the edge, consider the volume of data that needs to be transmitted from each edge device. Where it is passed to the cloud, high security should be built into the system, ensuring a system that provides all the tools required for rapid development and deployment of AI-enabled systems.
Filed Under: AI • machine learning, Product design