Augmented Reality Helps Build Aircraft Tanks

Walking through an unfamiliar city, getting directions or simulations of buildings that no longer exist – augmented reality is where virtual content and the real world come together. Scientists at Karlsruhe Institute of Technology (KIT) are developing an assistance system based on this technology which supports engineers in building and maintaining aircraft tanks. The system is currently being tested in industry.

In augmented reality applications (AR), a tablet, smartphone or special video glasses, also called a head-mounted display, provide additional information on reality. On the display, virtual content is mixed with the real world. One of the best-known AR applications is the game of Pokémon GO, a virtual hunt for virtual beings which “emerge” in all kinds of real-world locations. But the technology is gaining more and more in importance also in industry: “We are developing software that helps us build and maintain aircraft tanks. It aims to increase the flexibility of the employees, accelerate the workflow, and connect and optimize processes,” says Christian Tesch from the Institute of Anthropomatics and Robotics at KIT, Chair of Intelligent Sensor-Actuator-Systems (ISAS, headed by Prof. Uwe D. Hanebeck). Many commercial aircraft are at first not equipped for long flights and their fuel tanks are too small. So that they can still travel long distances, additional tanks are required which have to be maintained on a regular basis. To do that, so far engineers have had to climb into the tanks through a small opening. Often, however, they need both hands for assembling components, and new workers in particular also need instructions at the same time.

“The augmented reality glasses – at the moment we are using the HoloLens from Microsoft – display the work to be done in the engineers’ field of vision, who then have their hands free to install or repair components,” says Tesch. The glasses are equipped with cameras. Users scan special markers on the tank in advance using the cameras, which communicate the exact location and the size of the tank to the glasses. A transparent 3-D computer model from the inside of the tank is then projected onto the “real” tank; this means that engineers can also look into the closed tank from outside, understand the structure in detail, and get step-by-step instructions on how to install a pipe, for instance. What’s more, with the help of markings on the ground, the glasses show where to find the required components in the warehouse, the location of which the glasses also recognize thanks to markers that the system has already learned. “We connect the actual work on the tank with locating objects which helps us create an overall concept,” says Tesch.

Through ambient markers on the tank the AR glasses know where the tank is and how large it is, allowing them to project a 3-D model. Credit: Markus Breig, KIT

All calculations required for this concept take place directly in the glasses. Additional information about the tank condition, the work progress or the component stock can be supplied through an external database so that users are always up-to-date. They can operate the AR glasses using gesture and voice control.

Researchers are not only developing the system for AR glasses: “Many people today have a smartphone or a tablet; these devices have also found their way into day-to-day work. That’s why our software will also work with standard smartphones,” says Dr. Antonio Zea who is in charge of developing the software for mobile devices at the Institute of Anthropomatics and Robotics. In the next few years, the hardware for AR applications will probably also continue to be enhanced; AR glasses could get smaller and more affordable, making them even more versatile.