How do you teach a robot to find scratches? With artificial intelligence

^{English subtitles are available in the YouTube player}

With the help of artificial intelligence, a new robot will automatically find scratches and errors on components and provide quality control that is more effective than the human eye. In a MADE research project, VOLA and B&O are developing a prototype in collaboration with the Danish Technological Institute (DTI).

Stainless taps and aluminium-coated hi-fi equipment are all products with glossy reflective surfaces that must not be scratched.

VOLA  manufactures kitchen taps, which have exactly this type of shiny surfaces. VOLA has always concentrated on quality and consistency. Today, quality control is performed by operators who manually inspect the items produced by the company.

- It is crucial for us to be part of such a project because we focus on quality and design. The challenge is that quality control is manual today, says Peter Krogh, Technical Manager at VOLA.

- We give people guidelines for inspecting items, but it will still always be an individual interpretation. We hope that we can achieve greater consistency on the issues once the solution is automated. There is great potential.

That's why the Danish Technological Institute's robot vision specialists are now trying to develop a method that can reliably detect scratches. This work is part of a MADE project together with VOLA and B&O and originates from MADE researcher Anne Juhler Hansen's work at AAU.

Due to the shiny and reflective nature of the products, you have to go beyond traditional inspection methods with simple camera technology.

DTI therefore makes use of a very special vision technology called deflectometry.

Lights, cameras and robots

Deflectometry is a way of effectively inspecting glossy and reflective items. Here, lights, cameras and robots are used to handle the item precisely during the inspection.

- Conceptually, it involves projecting a known pattern of light onto the item and then filming with a regular camera how that pattern appears, says Thomas Giselsson, consultant from DTI.

- If there are small recesses, holes, notches or scratches, you will be able to see them by processing the data you generate, he adds.

Deep learning to automate quality assurance

The benefits of the new system do not stop here.

Not only does deflectometry make it possible to identify defects that you would otherwise find difficult, you can also automate the whole process of finding defects and evaluating the quality. One of the methods for this is 'deep learning'.

- Initially, the operator helps the system by highlighting where the problem shows. When you have done this for a large amount of data, you will at some point be able to create a deep learning system that can make the same annotations on new items that the system has not seen before, says Thomas Giselsson.

- We should not have to tell the computer how to build the system. It can figure it out for itself. The system will then be able to see many of the problems we get people to identify today, he adds.

- This process is still in the experimental stage - and quality control is part of VOLA's development and implementation of new automation solutions, as are our new self-driving robots, which now manage parts of the internal logistics at the Horsens factory, explains Peter Krogh, Technical Manager at VOLA.

The project with VOLA, B&O and DTI is part of MADE research within the theme Sensor Technology and Production Data, funded by the Innovation Fund. In June 2020, a meeting has been agreed with VOLA, where DTI will discuss the options for introducing an automation plant in VOLA's factory in Horsens.

When is something a fault?

The project originates from MADE researcher Anne Juhler Hansen from Aalborg University (AAU), who researches visual quality control for B&O, VOLA and LEGO.

- There are many aspects to this work that crop up repeatedly across industries, but there are also many things that are different. It is about creating a common language and looking at what standards are in the area, explains Anne Juhler Hansen.

In her PhD thesis, she studied how to quantify faults and create boundary values. In other words, when is something a fault that is ugly enough that the item should be discarded?

- We humans are flawed and we have different opinions. Maybe we can see a scratch today that we would not see tomorrow. Therefore it can be a major challenge to define when a fault is significant.

An interaction between humans and machines

The system initiated by Anne Juhler Hansen is intended as an interaction between humans and machines.

We are not aiming for fully automated solutions initially, but to provide visual quality control staff with assistance in making decisions and improving quality control.

The researcher at Aalborg University is excited that GTSs and production companies are now furthering her ideas and methods so that more people can benefit.

- It will be extremely exciting to follow how companies will try to implement this. We call it aesthetic quality control and it’s really hard to define.

In future, computer must learn to think like humans and experience mistakes just as we do.

- I have worked in this area for several years, so I am very excited to see where it will lead. There is great potential to spread the solution to others. Automated quality control can ensure uniformity and take on tedious, repetitive work that requires sharp focus all the time, says Anne Juhler Hansen.

​