Machine vision (MV) for object recognition and tracking solutions can be an important part of smart agriculture, industrial automation, environmental monitoring, robotics, logistics, and similar applications. However, developing object recognition and tracking applications takes much more than a camera and some artificial intelligence (AI) software. A multi-sensor platform and comprehensive data fusion are needed.
The starting point is a color camera with a resolution of at least 2 megapixels (MP) plus the software to extract and analyze information from the images. Other hardware can include a time-of-flight (ToF) sensor for distance measurements, a six-axis inertial measurement unit (IMU) with a 3D accelerometer and 3D gyroscope for monitoring speed and motion, and an integrated microphone.
The system can be trained to recognize important details with those hardware tools and the proper AI and machine learning (ML) software. That frees designers to focus on the big picture when developing systems for applications like object recognition, asset tracking, and predictive maintenance that must fuse the data on distance, sound, movement, and vibration.
This article focuses on the capabilities of the Arduino Nicla Vision platform with secondary discussions of the Nicla Sense ME (motion and environment) and Nicla Voice platforms.
It also reviews the benefits of the Arduino Desktop integrated development environment (IDE) for local development work, the Arduino Web Editor for working online, and how the finished design can be integrated with the Arduino Cloud for logging, graphing, and analyzing sensor data, trigger events, and supporting general automation processes.
The compact 22.86 mm x 22.86 mm format of Arduino’s Nicla boards enables designers to easily integrate low-power AI and ML into existing systems. The three boards are rated for operation from -20 to +70 °C and use industrial-grade sensors.
The micro-USB connector makes the Nicla boards capable of working as peripheral devices. The same micro-USB connector can interface with a PC for program development and debugging. The boards are designed to operate on battery power or can be powered through the USB connector.
They also include a five-pin Eslov connector, named after Eslöv, a small town in Sweden. Eslov is a modified I2C interface, sometimes called the Eslov self-identification port. The Eslov connector includes the standard I2C pins for the serial data line (SDA), serial clock line (SCL), ground (GND), power (+5V), and an extra digital pin (Figure 1).
Figure 1: Nicla boards showing the silver-color micro-USB connector and the cream-color Eslov connector on the bottoms of each board. (Image source: DigiKey)
Nicla Vision
The Nicla Vision ABX00051 board is designed to process and analyze images on the edge in real-time. It includes a 2 MP color camera that features:
- On-chip 10-bit analog-to-digital converter (ADC)
- 75 μm pixel size
- Focal length of 2.2 mm
- F-value of 2.2 ±5%
- Viewing angle of 80°
The camera is supported by an STMicroelectronics’ STM32H747AII6 Dual Arm Cortex processor, combining an M7 core of up to 480 MHz, with double-precision FPU and L1 cache and an M4 core of up to 240 MHz and an FPU. The MCU also includes 2 MB of Flash and 1 MB of RAM, plus a memory protection unit (MPU) and a full digital signal processing (DSP) instruction set.
The MCU is powerful enough to support data fusion and the complete sensor suite on the Nicla Vision that includes a smart six-axis motion sensor, integrated microphone, and ToF distance sensor, making it useful for asset tracking, image detection, object recognition, predictive maintenance scenarios, and more.
The Wi-Fi/Bluetooth module provides the connectivity needed for IoT functionality and cloud connectivity. Finally, security is supported by the integrated EdgeLock SE050 secure element (SE) crypto IC with enhanced Common Criteria EAL 6+ and FIPS 140-2 certified security. The SE includes an extended feature set for numerous IoT use cases and provides a root of trust at the IC level (Figure 2).
Figure 2: Locations of the main components on the Nicla Vision board. (Image source: Arduino)
Leveraging EdgeML and OpenMV
Nicla Vision is compatible with EdgeML (edge machine learning) and OpenMV (open machine vision). EdgeML and OpenMV can be used to rapidly develop edge sensing solutions for object recognition and tracking in a wide range of IoT MV use cases, including industrial automation, smart home, and health and safety.
EdgeML is optimized to run on resource-constrained devices like wireless IoT nodes. EdgeML applications have low latencies since inference is performed on the device, and no connection is needed to a server or the cloud. That limited need for communication also means less bandwidth is required for devices using EdgeML. Finally, EdgeML is designed to be implemented on battery-powered MCUs using only a few milliwatts of power.
Nicla Vision can be included into almost any project. It can be fully integrated with OpenMV and is compatible with all Arduino Portenta and MRK devices. It supports MicroPython and includes Wi-Fi and Bluetooth Low Energy (BLE) connectivity. The OpenMV Cam is a general-purpose MicroPython-powered MCU platform for computer vision processing.
Despite its powerful MV capabilities and extensive sensor suite, the Nicla Vision has a power consumption low enough to support compact battery-powered solutions (Figure 3). It features an average current consumption in deep sleep mode of only 374 μA and an average current consumption during image capture of 105 mA.
Motion and environment sensors
When additional environmental information is needed to provide context for a vision application, designers can turn to the Nicla Sense ME (Figure 3). The low-power Sense ME ABX00050 board can measure rotation, acceleration, pressure, humidity, temperature, air quality, and CO2 levels using four sensors from Bosch Sensortec:
- BHI260AP 6-axis inertial measurement unit (IMU) with integrated AI
- BMM150 magnetometer for gyroscope calibration in 9 DoF applications
- BMP390 pressure sensor
- BME688 gas sensor with AI and pressure, humidity, and temperature sensors
Figure 3: Nicla Vision with battery attached for wireless vision application. (Image source: Arduino)
Listening with a neural processor
Applications that need to listen and implement noise and vibration detection or low-power speech recognition can use the ABX00061 Nicla Voice board for always-on operation. It includes a special-purpose neural processor IC optimized for audio processing in battery-powered systems. The neural chip can run multiple deep neural networks (DNNs) using a variety of architectures like convoluted neural networks (CNNs), recurrent neural networks (RNNs), and fully connected networks.
Like the other Nicla boards, Nicla Voice is equipped with multiple sensors in addition to its high-performance MEMS microphone with an omnidirectional pickup pattern. Additional sensors on the Nicla Voice board include a six-axis IMU and a three-axis magnetometer.
Arduino IDE
The Arduino integrated development environment (IDE) is available on Windows, Linux, and macOS platforms. It includes a text console and text editor for writing code, a message area, a toolbar with buttons for common functions, and a series of menus for efficient navigation. It connects to the Arduino hardware to upload programs called sketches.
This IDE supports compiling and uploading sketches, file management, installing dependencies, etc. New features in the latest version include autocompletion, a built-in debugger, and syncing sketches with Arduino Cloud. Arduino also offers a cloud-based IDE called the web editor.
Arduino web editor
The Arduino Web Editor offers users several benefits compared with traditional IDEs, including:
- Being cloud-based, users can access this IDE from any device with an internet connection. Users are not tied to a single location.
- The IDE can be accessed from multiple locations to enable collaboration across geographically dispersed development teams.
- Cloud-based IDEs are scalable and can support large development teams almost without limitation. All development team members have complete access to the project.
- Arduino Web Editor is more secure than a desktop IDE. With the data stored in the cloud, it’s less likely to be lost or damaged because of a hardware failure. The data is encrypted, securing it from unauthorized access or interference.
- Automatic updates. The cloud environment ensures that the newest version of the IDE is being used, including all the latest tools and features, and eliminates the need for manual updates.
Arduino
In addition to supporting the Web Editor, the Arduino Cloud can provide secure wireless connectivity for boards. Features of the Arduino Cloud platform include:
- A backend service for synchronizing data from various Arduino boards and from Python and JavaScript clients
- A graphical dashboard and a mobile app for controlling and monitoring boards; for example, the mobile app can be used to read sensor data from remote boards, or the data can be pushed to a mobile device whenever a change in value exceeds a predetermined threshold (Figure 4)
- Tools to support large-scale automation installations
Figure 4: The Arduino Cloud can be used to connect mobile devices with remotely located sensor boards. (Image source: Arduino)
Conclusion
Arduino’s Nicle Vision can be the centerpiece for developing object recognition and tracking applications on the edge. Its sensor suite, which includes an IMU, ToF sensor, and microphone, enhances the utility of its MV capabilities. Compatibility with EdgeML, OpenMV, and MicroPython increases its utility and speeds development activities. Finally, it’s supported by the Arduino IDE and Web Developer platforms and can be easily integrated into the Arduino Cloud for secure connectivity
Learn more about Arduino ABX00051Sponsored content by DigiKey
Filed Under: SPONSORED CONTENT
