Enhanced Low-Illumination Image Defect Detection Using Machine Vision

In real life, due to various factors such as lighting conditions and camera performance, the image acquisition quality is often unsatisfactory, and low-illumination images generally have defects, such as noise, blur, defects, etc. These defects not only affect the quality and clarity of the image but also may cause errors in the analysis results. Therefore, how to accurately and quickly detect and repair the defects in low-illumination images has become one of the current research hotspots [1]. Research on the defect detection method of low-illumination images is conducive to improving image quality and the accuracy of information extraction, especially in the scene where automatic image analysis and recognition are required. Accurate defect detection can effectively improve the robustness and performance of image processing algorithms [2], [3]. For example, in the industrial production line, by detecting the defect points in the low-illumination image, the product quality can be quickly detected and checked, thus significantly improving the production efficiency and ensuring the improvement of the product qualification rate. In the field of medical imaging, accurate detection of defects in low-illumination images can help doctors diagnose diseases more accurately, improving the accuracy of medical diagnosis [4]. Therefore, research on the defect detection method of low-illumination images is of great significance. It can not only improve the level of image processing and analysis technology but also promote the development and application of related fields.

The current research on image defect detection methods is mainly aimed at improving detection accuracy and robustness. Among them, Zhuang et al. [5] proposed a neural network model based on deformable dense convolution, which uses a deformable convolution mechanism to achieve effective target description. The offset between the convolutional pixel and the image center pixel in the x and y directions was set as a new learning function to improve the adaptive deformation ability of the perceptual field. In addition, in order to ensure that the defects in the image are not lost when modeling, a method based on dense connection was proposed. The experimental data shows that the model has a high accuracy in the task of single-class target detection, but the rate of missed detection is high in the low-clarity environment. Liu and Zhou [6] proposed an image singularity detection method based on contourlet transform and fuzzy logic, which combines continuous wave transform and fuzzy logic to detect singularity and output the detection results of image singularity. The experimental results show that this method can enhance the detection accuracy, but under the condition of low brightness, the defect feature information is similar to the background pattern and the background information has strong interference, which easily leads to low structural similarity (SSIM) of the detection results and low fidelity of visual information. Xiao et al. [7] proposed an image defect detection method based on contrastive learning, which adopts a dual branch encoder decoder structure and ResNet50 as the backbone network to process defect images and template images with the same center position but different sizes (template images have slightly higher resolution). In the design, the attention module LACB is used to divide patches and utilize attention mechanisms to locally compare the feature differences between the two images, in order to adapt to the small-scale watermark offset caused by paper flexibility changes. Therefore, the defect feature map is designed to be smaller than the template feature map, allowing the features to shift within a certain range. Input the comparison results of the first three layers into the decoder, and the final layer into ASPP to integrate global information. The decoder fuses multi-scale feature maps across layers to reduce information loss, enhance different feature responses, and accurately segment defects. However, the local comparison mechanism of the LACB module ignores global contextual information, resulting in insufficient detection accuracy for complex defects, especially in cases where the watermark offset range is large or the shape is complex, the detection effect may significantly decrease. Hu et al. [8] proposed an unsupervised learning surface defect detection method based on image repair. In the detection process, a lost area was generated manually, and then the lost part was predicted according to the established network model. Finally, the defects were extracted and separated through the SSIM evaluation and residual analysis of the reconstructed image and the image to be measured. The experimental results show that the method performs well in detecting defects on regular texture surfaces, but the effect is not ideal in processing noisy images. Therefore, the ability of defect detection needs to be further improved.

In order to overcome the problems of the existing methods, a method was developed in this study to detect defects in low-illumination images with the help of machine vision technology as follows:

(a) In the low-illumination environment, the image is often disturbed by noise and shadow; the proposed method decomposes the image by noise estimation, which effectively reduces the impact of noise on the subsequent processing and improves the quality of the image.

(b) Traditional image processing is mostly based on the red, green, and blue (RGB) color space, but under low-illumination conditions, the information in the RGB space may not be accurate enough. The proposed method converts an image from the RGB color space to the HSI space, which can better deal with brightness and color information and is conducive to distinguishing different parts of the image.

(c) In the HSI space, this method uses local changes to distinguish the target part (such as defects) and the background part. This strategy can separate defects from the surrounding environment more accurately, laying the foundation for subsequent defect detection.

(d) The proposed method adopts the energy LBP operator to detect the defect points in the segmented image, which can capture the local structure and texture changes in the image and has high sensitivity and accuracy to the defect points in the low-illumination image.

(e) Comparative analysis of the proposed method with the existing methods through experiments verifies the application effect of the proposed method.

The accuracy of traditional detection methods is often limited by workers' experience and subjective judgment, resulting in instability of detection results [9], [10]. For this reason, this study introduces machine vision technology for defect detection of low-illumination images, and collects low-illumination images through this technology. In terms of low-illumination image acquisition based on machine vision technology, computers and other electronic hardware were used as the core tools to capture the image information of target objects through specific image acquisition devices, such as Complementary Metal-Oxide-Semiconductor (CMOS) or Charge-Coupled Device (CCD) sensors. In the low-illumination environment, the image quality is usually low and the noise interference is strong. Therefore, a series of image processing technologies, such as image enhancement and feature extraction, are needed to optimize the image quality and improve the visibility and recognition accuracy of target objects. These technologies can effectively enhance the contrast of the image and enhance the edge information, thereby accurately extracting and recognizing the features of the target object under low-illumination conditions. By setting the objective function and using the optimization algorithm to decompose the image, this study further removed the noise, retained key information, and finally achieved reliable acquisition and processing of low-illumination images.

The working principle of the technique is shown in Figure 1.

As can be seen from Figure 1, the principle of machine vision image acquisition is to capture light and form images through the optical imaging technology and different types of sensors to convert these images into digital signals. This process involves the refraction of light, image focusing and sensor photoelectric conversion steps. In machine vision systems, image acquisition is the first step in the process, in which precise control of lighting conditions becomes a key step, while the selection of suitable lenses and sensors is also critical. By optimizing these steps, clear and accurate image data can be obtained to provide strong support for subsequent applications such as defect detection.

Since the current defect detection methods for low-illumination images are often used to detect the image as a whole, it may lead to insufficient feature extraction of defects, making it difficult to accurately identify and locate the defects. In addition, it is susceptible to misjudgment due to the influence of the overall information of the low-illumination image, which reduces the accuracy and stability of the defect detection. In addition, ignoring the multi-frequency characteristics of the image may lead to the loss of small-scale detail information, which affects the accuracy of the final detection results. For this reason, the low-illumination images acquired in Section 2.1 were decomposed based on noise evaluation before defect detection [11], [12], aiming to split the image into sub-images or feature maps of different frequencies for subsequent analysis and processing. The decomposition can help to extract the subtle defect point information in the image and locate and identify the defect points more accurately on different frequency spaces, thus improving the accuracy and reliability of the defect point detection method for low-illumination images.

Images captured by industrial cameras often have problems such as uneven illumination, low contrast and more noise. Directly adjusting image illumination often aggravates the impact of noise, making noise suppression or removal more complex. Therefore, the image decomposition method was used in this study to separate the different structure information of the image and achieve effective noise suppression in the detail layer (including the fine features and significant defects of the image). The input low-illumination image was set to $I(x, y)$, which contains the base layer $I_a(x, y)$ and detail layers $I_b(x, y)$ as follows:

Subsequently, the base layer, i.e., the main structural information of the image, was obtained by minimizing a specific objective function of ${ }_{I_a}^{\min }\left\|I_a(x, y)-I(x, y)\right\|_2^2+\eta\left|\nabla I_a(x, y)\right|$.

To ensure that noise can be effectively separated from the high-frequency detail to the detail layer, global noise estimation was implemented. In a static environment, the noise model is Gaussian noise, which can characterize the statistical properties of multiple random noise. However, under dynamic light conditions, the noise properties become more complex. In particular, when the light intensity changes dramatically, it may lead to a significant increase in salt and pepper noise, which appears as random bright or dark spots in the image, whose statistical properties are significantly different from Gaussian noise. To meet this challenge, a noise ($\sigma$) estimation formula for many different noise scenarios was proposed in this study as follows:

where, $W$ and $H$ are the width and height, respectively, for image $I(x, y)$; and $M$ is the convolutional template.

Eq. (2) allows a rough estimation of the amplitude of the high-frequency components, and the detail layer can be obtained by calculating the residuals as follows:

Under low-illumination conditions, defects or defects in images are often not easy to be directly observed by the naked eye, which poses a challenge to image defect detection. By segmenting a low-illumination image, the image can be divided into different regions or sets of pixels to better extract and analyze image contents at different illumination levels, highlight defects or possible defects, and mark their locations and shapes for subsequent defect detection and analysis [13], [14]. At the same time, segmentation can further help reduce the background interference in the image and improve the accuracy and stability of defect detection. Therefore, based on the decomposition of low-illumination images, the segmentation process was further carried out on low-illumination images.

Traditional methods are often limited by the lack of sensitivity, difficulty in parameter selection and sensitivity to noise, resulting in poor image segmentation results, while adaptive thresholding segmentation methods have strong adaptivity, which can reduce parameter dependence and achieve good anti-noise performance. By determining the threshold value according to the local features of the image, the adaptive thresholding method can better cope with the needs of image segmentation under low-illumination conditions, improve the segmentation accuracy, reduce the burden of manual parameter selection, and effectively reduce the interference of noise on the segmentation results, thereby better satisfying the requirements of low-illumination image segmentation. Therefore, the adaptive thresholding method was adopted in this study to segment low-illumination images.

Because the RGB color space is vulnerable to the interference of the correlation between colors during image segmentation, the image was first converted from the RGB space to the HSI space in this study to optimize the segmentation accuracy in the case of uneven light. In the preprocessing phase, the image was converted to different color channels and stored. On this basis, the HSI color space conversion was performed on the image. To achieve this conversion, the RGB values were first normalized to the range of [0,1], and the converted H value range was set at 0°$\sim$360° while keeping the values of S and I within the range of [0,1]. The conversion formula from RGB to HSI is as follows:

The converted image was analyzed as a whole to determine the optimal threshold value $\phi$. Then, the color channel components of the image were processed by partitioning. On the basis of partitioning, adaptive thresholding was further solved to achieve more accurate image segmentation.

In order to effectively segment image foreground and background points, i.e., the target and the background parts, it is necessary to make judgments based on local variations. Therefore, the optimal threshold that can maximize the distinction between these two parts was calculated to achieve accurate segmentation of low-illumination images [15], [16]. The overall flow of the algorithm is as follows:

(a) After determining the optimal threshold $\phi$ for the image by calculation, each color channel component was partitioned according to the threshold value to extract the HSI channel component of the corresponding region.

(b) A horizontal sliding window was set up and operated by sliding pixel by pixel in the order from left to right.

(c) After setting the initial gray level within each sliding window as $N$ and the statistical for the number of pixel points of gray level $i$ as $n_i$, the average value of H and S components of each block $H_{ij}$ and $S_{ij}$ was calculated. Finally, the grey-scale histograms were normalized to obtain the normalized results.

(d) For target and background, the probability of their occurrence is as follows:

Among them, $t$ is the image grayscale.

The average gray value for each category was calculated according to Eqs. (8) and (9).

where, $\eta_t$ is the cumulative value at the time for a gray scale of $t$. Then the whole grayscale cumulative value is as follows:

For the two types of pixels, target and background, the variances within each of them are as follows:

(e) In order to effectively differentiate between target and background, the interclass variance was used as the objective function:

By traversing the gray level $0 \sim N-1$, a specific gray level that maximizes the above objective function was found and identified as the optimal segmentation threshold for the region.

(f) Considering the weight of H and S channel components, the objective function was optimized to obtain the optimal threshold as follows:

where, $w_h$ and $w_s$ are the weights of H and S channel components in the threshold value.

(g) After determining the optimal threshold, all pixels in the binarized processing window were judged. If the variance of a window is below the threshold, it is considered a blank area. To more effectively address the challenge of image segmentation under low-illumination conditions, an adaptive approach was adopted in this study to calculate local thresholds based on the brightness distribution in different regions of the image, which can significantly improve the accuracy and reliability of defect detection. In order to accurately find out the illumination conditions and corresponding threshold values of the algorithm, the Receiver Operating Characteristic (ROC) curve was introduced in this study as an evaluation tool. The specific practice is to treat the samples under different lighting conditions as objects to be classified, and record the false positive rate (FPR) and true positive rate (TPR) data in each threshold classification result as the threshold gradually increases from 0 to 1. The ROC curves were plotted from the data, with FPR as the horizontal axis and TPR as the longitudinal axis. The Area Under the Curve (AUC) enclosed by the curve is an important indicator to measure the accuracy of the algorithm, and its value range is between 0 and 1. A larger AUC value indicates better algorithm performance. In particular, when the AUC value reaches 1, the TPR reaches 100% and the FPR is 0%. Figure 2 shows the ROC curves drawn under different light intensity conditions.

According to Figure 2, when the FPR is less than 60%, the rising rate of the ROC curve increases with the increase of light intensity; when the FPR is greater than 60%, all ROC curves stabilize. However, when the light intensity becomes higher, the value of TPR is relatively small. Therefore, the optimal light intensity can be determined according to the actual work requirements. The mean gray level of the optimal light intensity selected in this study is 66.25 and the corresponding AUC is 0.96.

After in-depth analysis of low-illumination images, it is found that the defect points often exhibit irregular and non-uniform characteristics and they vary in size from a few pixels to tens of pixels. Traditional detection methods cannot effectively distinguish between defect and normal points. Therefore, it is necessary to find an efficient image detection method for detecting defect points in low-illumination images. By mapping different pixel values into binary codes, the energy LBP operator can measure and extract the local texture energy information in the image, thus realizing the detection of defect points in low-illumination images [17], [18]. Therefore, the energy LBP operator was chosen in this study for low-illumination image defect detection.

In this method, the definition of the ELBP operator is as follows:

where, $d_i$ is the energy value of the $i$-th pixel in the neighborhood, $d_c$ is the energy value of the center pixel, and $L$ is the number of side pixels of the rectangular neighborhood block.

This method is mainly used to extract the feature vectors of the energy feature image and the detection pane for low-illumination image defect detection. In this process, the selection of the defect detection pane size is crucial, which is the unit area reflecting the accuracy of defect detection. The basic repeating unit of the image in low-illumination image defect detection determines the size of the defect detection pane, and according to experience, the detection pane of uniform texture is determined by Eq. (18):

where, $W$ is the number of pixels on the edge of the detection cell pane. To ensure that the detection window contains sufficient feature information, the size of the cell panel was set to $16\times 16$ ($W=16$) when defects were detected in the low-illumination image. This specification panel was chosen for several reasons. First, the window size of $16\times 16$ provides a good balance between capturing enough detail and avoiding calculation redundancy. Smaller windows may miss key features, while larger windows may introduce unnecessary background information and increase computational complexity. Secondly, this size complies with the size range of the feature extraction window commonly used in image processing, which is conducive to being compatible with the existing image processing algorithms and hardware implementation to ensure the efficient operation of the algorithm [19], [20]. Therefore, choosing $16\times 16$ as the detection window size not only ensures the richness of the feature information but also takes into account the real-time nature and accuracy of the algorithm. Furthermore, in addition to satisfying the conditions of Eq. (18), the width of the defect detection window is the minimum width of the repeating unit in the image. If the size of the detection window is too small, it cannot capture the texture features completely, which may lead to the occurrence of false detection.

In order to determine whether the cell pane is a defect point area or not, it is necessary to compare the likelihood ratio of the cell pane with the threshold value. The log-likelihood ratio of the cell pane is calculated as follows:

where, $A$ is the feature vector of the defect-free point energy reference image obtained by training, and $B_r$ is the eigenvector of the $r$-th cell pane.

On the basis of the above processing, the specific steps for the detection of defect points in low-illumination images are given as follows:

(a) Calculation of the feature vectors of the defect-free point energy reference image.

(b) After chunking the defect-free point energy reference image and calculating the feature vector of each cell pane and the log-likelihood ratio of each chunk, the threshold value of the defect-free point cell pane was obtained as follows:

(c) In the detection stage, after similarly chunking the low-illumination energy image containing defect points to be detected and calculating the feature vector of each cell pane, the likelihood ratio of each chunk was calculated. If $T_r>T$, the corresponding cell pane was classified as a defect point. Otherwise, the cell pane was identified as a non-defect region.

Machine vision technology plays a crucial role in the experimental verification of the effectiveness of low-illumination image defect detection methods. The low-illumination image of the experimental sample can be captured by deploying a high-resolution industrial camera and equipping it with an appropriate low-illumination adaptive lens and light source. In this study, by deploying a high-resolution MV-SC2004M industrial camera equipped with an appropriate low-illumination adaptive lens and light source, the low-illumination image of the experimental sample was captured. The technical parameters of the MV-SC2004M industrial camera are shown in Table 1.

Calibrating the camera using a calibration board and making a precise adjustment of the camera according to the calibration results are a key step to ensure that image geometric distortion and aberrations are corrected. In order to obtain low-illumination images, in the process of image acquisition, measures such as turning off some lamps and using dimming light sources to dim the overall environment were taken in this study. At the same time, the number of surrounding light sources was minimized to avoid strong light interference and ensure the low-illumination characteristics of the image. Furthermore, in terms of sample size statistics, a sufficient number of image samples was collected to cover various possible shooting conditions and scenarios, thus improving the accuracy and reliability of the calibration results. In terms of data diversity, attention was paid to collecting image data from different angles, distances and lighting conditions to ensure that the calibration algorithm can learn various distortion characteristics of the camera and effectively correct them. Implementation of these measures further improved the imaging quality of the camera and met the needs of various application scenarios.

(a) Peak signal-to-noise ratio (PSNR)

The higher the value of PSNR, the smaller the distortion of the image, thus reflecting higher image quality. The calculation formula is as follows:

where, $M A X_i^2$ is the maximum image color pixel value, and $M S E$ is the mean square error.

(b) SSIM

SSIM estimates the information difference between the sample image and the reference image from the human visual perception system behavior indicators to obtain an objective evaluation. The calculation formula is as follows:

where, $l(x, y)$, $v(x, y)$ and $z(x, y)$ are the mean, variance and covariance, respectively; and $\alpha$, $\beta$ and $\gamma$ are the proportions given to brightness, contrast and structure, respectively.

(c) Visual information fidelity (VIF)

VIF is a measure used for image quality inspection, which can quantify the information existing in the reference image and how much reference information can be extracted from the deformed image. The calculation formula is as follows:

where, $I\left(C_r ; G^k \mid Q_r^k\right)$ and $I\left(C_r ; S^k \mid Q_r^k\right)$ are the information extracted from specific subbands of the target image and the image to be measured, respectively; and $k$ is the number of subbands.

The method presents the noise decomposition of low-illumination images, as shown in Figure 3.

According to Figure 3, before the noise decomposition, the original image may contain various sources of noise, which appear as irregular pixel changes in the image, masking the key details in the image and reducing the visual quality of the image. After the noise decomposition processing, the noise component in the image is effectively separated. This means that the boundary between the useful information in the image and the noise becomes more clear and regular. After the noise decomposition, it can be observed that the image details are better preserved, and the overall visual effect of the image is also significantly improved. This processing not only helps to improve the clarity of the image but also provides a more accurate and reliable basis for the subsequent image analysis and processing steps.

Based on the above, the PSNR, SSIM and VIF of the method proposed in this study and the methods proposed by Zhuang et al. [5] and Liu and Zhou [6] were calculated from Eqs. (21) to (23). The calculation results are shown in Table 2 to Table 4.

According to the data provided in Table 2, it can be clearly seen that the PSNR of the method proposed in this study reaches 12.54, which is 2.49 and 1.24 higher than the methods proposed by Zhuang et al. [5] and Liu and Zhou [6], respectively, indicating that the method proposed in this study has excellent performance in maintaining image quality and can more accurately restore the original image information. According to the data in Table 3, the SSIM value obtained by the method proposed in this study is closer to 1, showing higher image structure retention capability. It shows that this proposed method can improve the image quality and has less damage to the structure information of the image. In image detection, the greater the fidelity of visual information, the better. High fidelity means that the content and details of the original image can be kept as much as possible after the image is processed so that the human eye can accurately perceive the quality and information of the image. By comparing the data in Table 4, it can be seen that the VIF of the method proposed in this study is higher, which indicates that the detection result is closer to the original image, and the image quality and visualization effect are better.

It can be seen from the above table that the method proposed in this study is better than the traditional methods under the three evaluation indicators of PSNR, SSIM and VIF, which verifies the application effect of the proposed method.

False detection rate can reflect the accuracy and reliability of the detection method in the detection process. Specifically, the leakage rate indicates the proportion of defect points that the detection method fails to detect successfully in the case of the actual existence of defect points. Therefore, the lower the leakage rate is, the better the performance of the detection method is, and the more accurately the defect points in the image can be recognized. The specific calculation formula is as follows:

where, $f$ denotes the false detection rate, $n_{F P}$ denotes the number of normal pixel points misclassified as defects, and $n_{T P}$ indicates the number of undetected defect points.

False detection rate was calculated using Eq. (24) and the results are shown in Figure 4.

Analysis of Figure 4 shows that, with the increase of the number of sample images, the leakage rate of the three methods shows an increasing trend, but the leakage rate of the method proposed in this study is lower compared with that of the methods proposed by Zhuang et al. [5] and Liu and Zhou [6]. In addition, the leakage rate is always lower than 8\%, while the highest value of the leakage rate of the defect points in the methods proposed by Zhuang et al. [5] and Liu and Zhou [6] reaches more than 13\%. It shows that the method proposed in this study is still able to accurately identify most of the defect points under the complex lighting conditions, which reflects the high detection accuracy and reliability.

In order to verify the applicability of the method proposed in this study, two other comparison methods were introduced in this experiment. First, the MV-SC2004M industrial camera was used to collect the images of spiral material. Then the low-light enhancement method based on Transformer and the target detection method based on the You Only Look Once (YOLO) series were used to detect the defect points in the collected images. The experimental results are shown in Figure 5.

By analyzing the results of Figure 5, it is found that all three methods can achieve effective defect detection to some extent. However, in the detection of impurities and scratch defects, both the methods based on Transformer and the YOLO series are missing problems when compared with the method presented in this study. In contrast, the proposed method is able to comprehensively and effectively detect all impurities and scratches, which fully verifies the superiority and comprehensiveness of this method in defect detection.

To further verify the effectiveness of the method proposed in this study, the computational efficiency of RetinexNet, the Zero-Reference Deep Curve Estimation (Zero-DCE) method and the proposed method were compared. The experimental results are shown in the Table 5 below.

As can be seen from Table 5, the proposed method has significant advantages in terms of the average processing time and detection accuracy of low-illumination image defect detection tasks. The average processing time of the proposed method is only 1.1 seconds/slice, which is much lower than the 2.5 seconds/slice of the RetinexNet method and 1.8 seconds/slice of the Zero-DCE method, indicating the obvious advantages of computational efficiency. Meanwhile, the detection accuracy of the proposed method is 98%, which is higher than that of the RetinexNet method (85%) and the Zero-DCE method (88%), further verifying the superiority of the proposed method in defect detection accuracy. In conclusion, this method has high utility and application prospects in the field of low-illumination image defect detection.

In order to improve the detection effect, this study proposes a machine vision-based defect point detection method for low-illumination images, which realizes the effective detection of defect points in low-illumination images by combining the techniques of machine vision, noise estimation, color space transformation, local change segmentation and energy LBP operator, which is innovative and practical. In practical applications, there may be other types of information (e.g., temperature, pressure, etc.) in addition to image information, which may be helpful for defect detection. Therefore, the fusion of multi-source information will be investigated in the future to characterize the defect points more comprehensively and improve the accuracy of detection.