Many manufacturing processes operate using fixed or hard automated equipment that performs production tasks with limited sensory input. For more complex applications, simple cameras or sensors can detect the presence, position, size or thickness of an object. When the object is more complex, has fewer constraints, or needs to be evaluated for its appearance, a machine vision solution can be applied. This blog post will review three applications to gain insight into the role of machine vision in advanced automation.
In many manufacturing processes, counting objects or features is often important to ensure part quality or manage inventory. While this may sound trivial, it is not a practical task for humans when large amounts of data are involved. For such tasks to be automated through machine vision, object segmentation is the first step, and this can be facilitated by the proper application of lighting and imaging techniques.
The goal of image acquisition is to illuminate and capture an image of the object in a way that enhances the contrast between the features to be detected and the background. Machine vision software is then used to segment and detect the features or objects of interest. The measured attributes of each detected object can then be used to determine its quality or identity.
Characterizing weld porosity
Take the detection and evaluation of weld porosity as an example. The shape of the part, the variable contours of the weld channel, and reflective metal surfaces make uniform illumination a challenge. Fortunately, pores don't reflect much light - they look dark.
Welds have a variety of dark areas that can be segmented by machine vision. Pores in the weld have a characteristic size range and shape that can be used to ignore dark areas that do not match the characteristics of the pores. Once porosity is detected, the number and density of pores (number per inch) in the weld can be used to indicate whether the welding process is acceptable or whether operator or control system intervention is required.
Counting Tubes
A related example is counting the number of tubes in an image captured from the end of a crate; inventory control requires accurate counting. Challenges include variable illumination and variable perspective of the tube ends in the image. The tube end is characterized by its dark interior surrounded by the bright circular surface of the tube wall.
Splitting the dark area with a circle of the expected diameter will detect most of the tubes. Note, however, the bright reflections of some of the tube interiors near the bottom of the crate - image processing operations can merge these small features with the tube interior regions for robust detection and counting.
Detecting Complex Shape Damage
Consider detecting surface damage to propeller blades. Damage can range from narrow scratches to large wear spots; there are no standards to characterize the expected size or shape of the damaged area. In addition, the complex shapes of propeller blades present a challenge to the optimal illumination used to enhance damage contrast.
In the illumination configuration used for the leftmost (darkest) image, damage was barely perceptible. The two alternating illumination directions provided good contrast between damaged and undamaged blade areas, but the contrast flipped between the two configurations. Due to the localized surfaces and the direction of damage relative to the imaging system, different regions of the propeller blade will exhibit different responses as shown - meaning there is no single optimal illumination configuration.
The high degree of variability between damage shape, size and contrast makes automated detection using program-programmed methods challenging, as used in the weld porosity and tube counting examples. The Institute developed the inspection system using machine learning techniques. A convolutional neural network (CNN) recognizes potentially damaged areas in an image. A secondary deep neural network classifies the image as containing (or not containing) damage based on the eigenvalues generated by the CNN. These networks are trained using a large number of images where the damaged areas have been identified manually.
Beyond Monochrome Vision
The three examples above illustrate some monochrome machine vision applications. Things get even more interesting when using color contrast or using the invisible portion of the spectrum. For example, monochrome cameras are sensitive to near-infrared (NIR) wavelengths, allowing features that would normally be invisible or distracting to be exploited or removed by using or rejecting that band with a spectral filter.
Standard color cameras use overlapping broadband red, green, and blue filters; using narrowband RGB LED illuminators instead of broadband white illumination improves color discrimination. Multi-spectral cameras offer highly sensitive color discrimination compared to color cameras and can include NIR bands. The fluorescent properties of certain inks, dyes and adhesives can be exploited by using UV irradiation with appropriate spectral filters. And don't forget polarization! Infrared imaging (longwave, midwave, shortwave) can be used to measure surface temperatures, detect subsurface features/defects, detect hydrocarbon gases, and more.