A new way of conceiving assembly instructions for the operator: augmented reality applied to the industrial sector. Parts and assembly stage automatic recognition, interactive instructions, animations superimposed on reality, speech synthesis and recognition, digital manual consultation, this all projected onto the reality. Fundamental when instructing a new operator in an autonomously manner, indispensable for refreshing experienced operators. By using vision technologies for object recognition, speech synthesis and recognition, interface with CAD and computer graphics for object virtualisation, the application allows the production manager to create assembly and repair instructions in a rapid and convenient manner. The assembly operator or the repair technician will see the instructions projected onto reality, through monitors or augmented reality glasses.

Download PDF version
Laboratory
Contacts
Mirko Falavigna
Specialization Area
Mechatronics and Materials
Interactive instructions for mechanical assembly
Innovative aspects

Augmented reality based on markers is not suitable for industry due to non-ideal environmental conditions and the need for a marker for every part to be assembled. This project avoids the problem using models derived from CAD, which therefore becomes the core element of the project, jointly with artificial vision technologies, for simplified development of assembly/repair stories.

Potential applications

This application offers the ability to: guide an operator so that he/she can learn an assembly or disassembly process for the first time: control routine assembly/disassembly operations which often lead to mistakes due to distraction; avoid costly journeys of skilled technicians on site, by using a local technician use this application for repair/replacement of a faulty part. In this case the updated information is transferred over the Internet.

Animations of augmented reality for assembly
Application example

Wiring an electrical panel with augmented reality

Application description and results

When wiring an automatic machine with the traditional manual, one may make electrical connection errors due to inexperience or distraction, which may lead to safety issues and are very costly to the company in terms of tests, validation and possible replacement of damaged components. By using this application, the operator follows the instructions on the monitor. Specifically, the work station is fitted with an all-in-one touchscreen PC and a video-camera. The application interfaces with the video-camera to allow the operator to show the cable code (OCR). When the code is recognised, the monitor which shows a photograph of the electrical panel displays the position in which the cable is to be connected. The operator may interact with the application either through speech recognition and synthesis and through the touchscreen monitor. The association between the cable number and position has been done previously, automatically analysing the electrical panel CAD and extracting the relevant codes and positions. In the event the engineering department needs to update a cable code, all it takes is to update the box of a table. Another way of indicating the cable position in the electrical panel is by interfacing with a laser pointer driven by two stepper motors so it may be freely rotated. In this way the operator sees on the panel where the cable is to be connected. We have obtained a significant reduction in the rate of errors, which generally take place also due to quick referencing to the paper manual, together with a decrease in the time required for the individual repeated operation. Reading the code on the cable, which is interpreted via OCR and in unique relation with the position on the electric panel, and the visual feedback given to the operator on where to connect the cable allow a controlled, thorough and traceable operation.

Involved partners

- Gruppo CMS (Marano sul Panaro - MO) - T3LAB

Implementation Time
The project has required a 6-man months commitment
Technology Readiness Level
TRL9 - Actual system proven in operational environment (competitive manufacturing in the case of key enabling technologies; or in space)
Exploitation

T3LAB sample project is in close collaboration with Gruppo CMS. However T3LAB is constantly looking for further collaborations and partners to experiment with augmented reality technologies in similar settings and pursue further development.

Recognition of the cable code and wiring instruction
Published on