Real time detection system for rail surface defects based on machine vision

Abstract

The detection of rail surface defects is an important part of railway daily inspection, according to the requirements of modern railway automatic detection technology on real-time detection and adaptability. This paper presents a method for real-time detection of rail surface defects based on machine vision. According to the basic principle of machine vision, an image acquisition device equipped with LED auxiliary light source and shading box has been designed and the portable testing model is designed to carry on the field experiment. In view of the real-time requirement, the method of extracting the target area from the original image is carried out without image pre-processing. The surface defects of the rail are optimized based on morphological process and the characteristics of the defects are obtained by tracking the direction chain code. It is demonstrated that the maximum positioning time of this proposed method is 4.65 ms and its maximum positioning failure rate is 5%. The real-time detection speed of this proposed method can reach 2 m/s, which can carry out real-time detection of artificial hand walking. The time of processing each picture is up to 245.61 ms, which ensures the real-time performance of the portable track defect vision inspection system. To a certain extent, the system can replace manual inspection and carry out the digital management of track defects.

1 Introduction

The railway track, including rail, sleeper, and fastener, is the most important infrastructure of the railway. Due to the long-term impact of the train and the natural environment on the railway, a variety of defects will be formed on the rail, which will jeopardize the safety of the railway, such as the deformed rail and broken sleeper [1]. These changes on the surface of the railway are caused by defects and can be directly seen by the worker’s eyes; therefore, the patrol inspection method by sight can be used for first detecting the defects of the railway. At present, the track surface defect detection mainly relies on manual inspection, but this method is labor-intensive, has lower efficiency, and influences results easily; it cannot meet the needs of daily inspection of the modern railway, especially the China High-speed Railway (CRH). As the daily inspecting time of CRH is always scheduled quickly and at night to avoid disrupting the regular operation of the train, there is not enough light and the workers tire easily so that it is hard to get a creditable inspection result from workers’ eyes. Therefore, it is urgent to find faster and more efficient automatic detection equipment to carry out the real-time detection of the track in the operation line [2,3,4]. The research mainly aims at the common rail surface defects, such as cracks, peeling, scratches, folds, broken rails, scars, and other forms [5].

In recent years, much research on the automatic detection of railway faults have been launched in major universities and railway institutes. A method to achieve the direct extraction of the surface defects of rail by the improved value of minimizing the maximum between class variance has been proposed by Mao et al., but the accuracy of the suggested method needs to be improved [6]. Tang et al. used gray level projection to realize rapid segmentation of the rail surface region, but the algorithm has poor adaptability [7]. Yuan et al. use the improved Ostu algorithm to segment the rail image and obtains the rail surface area, but the threshold selection of this method is not universal [8]. He et al. used the mean background subtraction algorithm to extract the rail surface defects, but the robustness of the algorithm was weak [9].

Guo et al. studied the position correction method of the detection equipment in the long line, which mainly used the track of the track mark image to carry on the position through the linear search [10]. According to the characteristics of uneven illumination, occlusion, and complicated environment, Li et al. established the sparse representation model of the defect image [11].

The existing research mainly focuses on the accuracy of the surface defect detection system. However, according to the requirements of the system in field application, it is necessary to reduce the computational cost of the algorithm as much as possible, making the algorithm with better real-time detection and adaptability [12].

This paper presents a method based on machine vision for real-time detection of track surface defects and a basis system of machine vision detection. First, the system can ensure access to high-quality images from a stable light source environment by adding the hoods and LED auxiliary light source in the image acquisition equipment. Then, the H value of the color image is used to directly extract the original image, which can shorten the time of image pre-processing and is good for a target area with a small range. Based on morphological processing, the interference of a large amount of redundant information is removed and the contour of the direction chain code is used to quickly extract the defect shape feature to carry out the defect type identification. In order to ensure the adaptability of the system in a complex environment, the issues of adaptive dual threshold selection in edge detection combined with the histogram concavity analysis have been solved.

2 Rail surface defect detection system

The track surface defect detection model is shown in Fig. 1; it includes five parts: image acquisition; target area location; image enhancement; defect detection; and human–computer interaction [13]. In the image acquisition part, in order to reduce the influence of the external complex illumination condition on the track surface defect detection system, the shading hoods and the auxiliary LED light source were added to provide a stable light environment for image capture. The function of the target area location part was to rapidly locate the target area in the captured image so that it can save image processing time later. The image enhancement part was mostly to enhance the character of the target area. The defect detection part uses lower complexity operation methods for rapid detection of the defective track. The human–computer interaction part displays the real-time result, recording the detected result and giving an effective alert. These parts combine a whole device for fast detection of the defects of a railway track.

Fig. 1

Schematic diagram of detection system model

3 Rail area rapid locating method

In actual defect detection, the track defects are detected in real-time and the system has strict requirements for the detection time. Therefore, in the detection system, first, the rail area is quickly located from the detected image, thus saving time for subsequent processing. This paper proposes a method by using the target area hue mutation to rapidly located target areas in the detecting image.

Hue, saturation, and lightness (HSL) are a standard of color in the industry, including all the colors that human vision can perceive. Red/green/blue (RGB) color models in cylindrical coordinates is one method, where H (Hue, hue) is the first feature of color and is the most accurate criterion for distinguishing different colors. The hue characteristic is stable and varies little with the illumination condition. Under certain illumination condition, this feature can be considered as unaffected by the external illumination. The H (Hue, Hue) value in the HSL color space is easily calculated by using the RGB space in the color image. The conversion of RGB color space to HSL color space can be realized by Eq. 1 [14]:

$$ H=\left\{\begin{array}{c}\operatorname{arccos}\left\{\frac{\frac{\left[\left(R-G\right)+\left(R-B\right)\right]}{2}}{\sqrt{{\left(R-G\right)}^2+\left(R-G\right)\left(R-B\right)}}\right\}\kern3.1em B>G\\ {}2\pi -\operatorname{arccos}\left\{\frac{\frac{\left[\left(R-G\right)+\left(R-B\right)\right]}{2}}{\sqrt{{\left(R-G\right)}^2+\left(R-G\right)\left(R-B\right)}}\right\}\kern1em B\le G\end{array}\right. $$

(1)

3.1 Track image feature analysis

The track area usually consists of rail, ballast, sleeper, and fasteners; their characteristics and H values are different in the detecting image, as shown in Fig. 2.

• (1) In Fig. 2a, the value of H in the rail surface area is high as a whole and has small mutations.

• (2) In Fig. 2b, the value of H has a larger mutation in the stone ballast area.

• (3) In Fig. 2c, the H value curve of the elastic strip area is not stable and has large mutations. In addition, the H values in this area are generally small.

• (4) In Fig. 2d, the H value curve of the sleeper area is small and smooth.

Fig. 2

H value curve of each part of orbit. a H value curve of a straight line in the rail region. b H value of a straight line in the ballast region. c H value curve of a straight line in the fastener region. d H value curve of a straight line in the sleeper region

It is easy to see from above that the mutation of the H value curve of the rail itself can be used as the basis of judgment of the rail surface area.

3.2 Target region location algorithm based on H value feature

Assuming that the geometric position of each point in the track space corresponds to each pixel in the detected image, the track space is completely mapped to the pixel space of the detected image, assuming the camera is calibrated and normalized. The actual width of the rail can be expressed and calculated by the pixel number of the rail image.

First, the image coordinate system O _xy is established and the lower-left corner of the image is taken as the coordinate origin O; the two sides of the image are the x-axis and the y-axis, respectively, as shown in Fig. 3. If the image size is u * v, and (x, y) is used to represent a certain pixel in the image, the range of values of x and y are {x|0 ≤ x ≤ u, x ∈ N⁺} and {y|0 ≤ y ≤ v, y ∈ N⁺}, so that the H value in the HSL space of the corresponding point is h(x, y).

Fig. 3

Sketch map of the image coordinate system

If all the characteristics of the H value curve in each column of the whole image are analyzed, it has lower efficiency and is not conducive to judge the rail surface boundary. To this end, the captured images are divided vertically along the positive direction of the x-axis and the H value mutation points on the bisector are searched in the positive direction of the y-axis to reduce the search workload and improve the efficiency.

Then, define n as the number that an image be equally divided and the H value h(x _p , y) of the point (x _p , y) on the p-line. We use t to represent mutation of H value in an image; the value of t is defined as Eq. 2.

$$ \left\{\begin{array}{c}t=0,\kern0.5em h\left({x}_p,y\right)-h\left({x}_p,y-1\right)\le \frac{h\left({x}_p,y\right)+h\left({x}_p,y-1\right)}{2},\left\{p\left|0\le p\le n,p\in {\mathrm{N}}^{+}\right.\right\}\\ {}t=1,\kern0.5em h\left({x}_p,y\right)-h\left({x}_p,y-1\right)>\frac{h\left({x}_p,y\right)+h\left({x}_p,y-1\right)}{2},\left\{p{\left|0\le p\le n,p\in \mathrm{N}\right.}^{+}\right\}\end{array}\right. $$

(2)

In the case of t = 1, it is recorded as a mutation point on the p-th bisection line. If the p-th bisection line cannot find any mutation point, it is assumed that the unit step is $d=\frac{u_{\mathrm{c}}}{n\times 2\times 5}$ (where u_c is the camera horizontal pixel). To re-determine the mutation point, use one of the ten lines which are parallel to the bisection line in the range of about five steps on the left and right sides. When the entire straight line is judged and the mutation point of H value is found, the search for other lines in the set range is stopped. If we still cannot find the mutation point, in order to avoid falling into the search dilemma, set the sub-line point of the mutation point as (x _p , 0); it will be directly removed in the follow-up discrimination.

Due to the limited view field of the camera and the curvature of the turning train being far greater than the view field, the edge of the rail can be considered as a straight line in the camera view field and without considering the change in the relative position of the camera and the rail when the train turns.

Next, assume the mutation points of the two adjacent H value in the p-line are (x _p , y _q ) and (x _p , y_{q + 1}). Comparison of the pixel width of the rail with the mutation points are shown in Eq. 3.

$$ \left|{y}_{q+1}-{y}_q\right|\le {w}_{\mathrm{r}} $$

(3)

In Eq. 3, w_r is the amount of the pixels of rail width in the detecting image. If (x _p , y _q ) and (x _p , y_{q + 1}) satisfy the determination of the rail boundary points in Eq. 3, the (x _p , y _q ) is the lower boundary point of the rail surface and the (x _p , y_{q + 1}) is the upper boundary point of the rail.

Through linear fitting of the n boundary points, it is easy to obtain the upper and lower edges of the rails so that the rail area in the image can be quickly determined, as shown in Fig. 4.

Fig. 4

Sketch of boundary fitting results

4 Rail surface defect detection

4.1 Rail surface area image enhancement

In the process of image acquisition, it is easy to interfere with the influence of external light, foreign body occlusion, car body shaking, and other factors, the noise will inevitably exist in the original image [15]. The noise will directly or indirectly affect the detection effect of the algorithm, so it is necessary to filter the image.

Median filtering is used to sort the gray values of each pixel in the filter window and select the median to filter noise [16]. Since median filtering requires a large amount of sorting and long computation time [17], a weighted fast median filtering method is adopted in this paper. The filtering process of this method is as follows.

Assuming the median filter window moves along the column direction, the pixel gray values of c₁, c₂, and c₃ are removed from the filter window and the pixel gray values of d₁, d₂, and d₃ are moved into the filter window. The schematic diagram of filtering process is shown as Fig. 5.

Fig. 5

Schematic diagram of filtering process. a Position relation before filtering. b Position relation after filtering

The gray level series and the correlation of adjacent pixels are shown in Eq. 4:

$$ \left\{\begin{array}{l}{c}_1={d}_1\\ {}{c}_2={d}_2\\ {}{c}_3={d}_3\end{array}\right. $$

(4)

• (1) If Eq. 4 is established, the output value is unchanged; if the equation is not established, then the next step is to be carried out.

• (2) The mean and the median of all pixels in the current window are f₁(x, y) and f₂(x, y), respectively.

• (3) Give different weights to f₁(x, y) and f₂(x, y), and the gray update value of the central pixel of the filtering window can get through Eq. 5:

$$ {f}^{\hbox{'}}\left(x,y\right)={f}_1\left(x,y\right)\times 0.7+{f}_2\left(x,y\right)\times 0.3 $$

(5)

• (4) Filtering window center to move to (x, y + 1) for the next round of computing.

By using the method of weighted fast filter, the rail surface area can be quickly filtered to remove noise, which can improve the operation speed of compared with the median filter 76.4–79.7% [16].

4.2 Morphological processing based on stroke coding

Experts in cognitive psychology, human contour features are more sensitive than the color and texture features; the stability and robustness of the recognition is high [18]. So, we first use the canny algorithm to convert the image into a binary image and get the image contour. However, when the image is processed into a binary image, the edge of track defect is prone to burrs and edges [19]. This seriously affects the detection accuracy of the portable track defect vision inspection system [20].

In the process of morphological transformation of a binary image, the closed operation can fill the smooth contours of the gap [21]. This is due to the fact that closed operations are essentially dilation operations and then etching operations. Therefore, the closed operation can be used to fill the fracture in the contour line, eliminate the isolated small holes in the target feature, and smooth the protrusion on the edge. But the expansion and corrosion of the closed operation need to analyze each pixel, so it needs to occupy a large amount of computing time and memory. So, the optimization of the closed operation process is the key to improve the speed of the image processing algorithm. In this paper, a morphological processing method based on stroke coding is used to eliminate the burr and the growth line break (as shown in Fig. 6) and to avoid the processing of a large amount of redundant information.

Fig. 6

Sketch map of burr and disconnection

A set of pixels with the same gray level in each row is called a stroke and each stroke only records the gray level and the pixel length. Therefore, run length encoding can be used to compile the key information region of image. By using the range coding to compress the gray level region effectively, the intersection and union of the key information areas will be simplified.

Meanwhile, morphological dilation operations can be regarded as transposed structural elements applied to the operation of Minkowski addition [22] and the corresponding expansion image region is obtained by calculation. In this way, the optimization process of morphological closed operation can be realized by using the intersection and union operations of run length encoding and the commutative law and distributive law of Minkowski addition, thus improving the processing speed of morphological closed operation.

According to the experimental results, it is indicated that the morphological processing based on the optimization of stroke coding can be applied to a real-time detection system. After the optimization of the image processing, the broken line and the edge burr of the rail defects are eliminated completely, and a smooth and connected track defect region is obtained, as shown in Fig. 7.

Fig. 7

Morphological processing results based on stroke coding

4.3 Directional chain code contour tracking

By optimizing the edge details of the defect region, the contour of the target becomes obvious. Based on the relationship between the defect region and the boundary, it is convenient to follow up the measurement by using the chain code to track the contour information [23]. This paper uses the direction chain code tracking algorithm to extract the contour information of the defect region.

This method is based on contour point information tracking, which can avoid searching the whole image, improving the efficiency of the algorithm. The chain code tracking process is as follows:

• (1) The center pixel L is encoded in an anti-clockwise direction, which is coded to eight adjacent points, as shown in Fig. 8.

• (2) The rail surface area image is scanned; when a contour point is obtained, the line scan is stopped and the coordinates of the point (x₁, y₁) recorded.

• (3) According to the direction of chain code tracking, (x₁, y₁) is the center and the adjacent points are scanned. If the defect contour point (x₂, y₂) is scanned, tracking is stopped and the recording point (x₁, y₁) of the direction of chain code value for the scan to the outline of the code, and then continue to step 3; if the contour points are not scanned, the contour tracking is stopped.

• (4) With the background color filling point (x₁, y₁), point (x₂, y₂) as the center, carry out step 2.

Fig. 8

The direction chain code of pixel L

The above steps will get all chain code sequences of the contour defect, realizing the accurate extraction of the rail surface defect area and the sequence of contour lines is helpful for subsequent processing and measurement. Some obtained contours of rail surface area defect are shown as Fig. 9.

Fig. 9

Rail surface defect profile screenshot. a Defect A. b Defect B. c Defect C

Because of the complex shape of rail surface defect, the single feature description parameter cannot describe the defect exactly and comprehensively. For this reason, the defects are described and recorded considering the combination of various parameters [24]. The description parameters of the defect features are shown in Table 1.

Table 1 Defect feature description parameters

Different types of defects have different effects on the composition of track performance. In the process of recording the characteristic information of defects, it can be simply classified so as to predict the influence of defects. In this paper, the defect categories are divided into two categories [25], scar and crackle, according to the aspect ratio. The classification of defects and the recording of characteristic information are shown in Table 2.

Table 2 Defect characteristic information

5 Experimental discussion and results

5.1 System implementation

According to the track defect detection method based on machine vision, this paper designs a portable vision inspection system for track defects and develops a portable track defect vision inspection prototype, as shown in Fig. 10. The prototype components can be quickly assembled and disassembled; the daily inspection and recording work of the rail surface defects can be completed by single hand push mode.

Fig. 10

Prototype of track defect detection system

The device of defect detecting consists of shading hoods, auxiliary lights, cameras, computer, battery, and portable trolley, as shown in Fig. 11. The camera works in trigger mode with LED light. The image processing algorithms can be generalized to two steps, namely image enhancement and image analysis. Image enhancement is used to pre-process the original image and segment the image to highlight the feature information, and to weaken or eliminate the interference of background information. In image analysis, by extracting the features (statistical features, shape features, geometric features, texture features, etc.) of the object under test, the process of decision-making is obtained by using the default criterion. In practical applications, image enhancement usually takes up a large proportion of system processing time and improved image enhancement can reduce the computing time of the vision detection system.

Fig. 11

Track surface fault inspection system

In this paper, we used LabVIEW to write the software system; the track defect recognition system interface is shown in Fig. 12. The system for the real-time display of an image picture, in the form of text to display the current picture detection demerit, curve form record test results. The track defect recognition system can continuously collect, process the image and record, save the detection results, the original text data format contains the measuring time, the number of image defects, and historical data. The horizontal coordinate of the data curve is the measuring time and the measuring data change dynamically with the measuring time.

Fig. 12

Surface defect visual inspection system interface

5.2 Target location experiment

In order to analyze the influence of different equal values of n on the recognition speed and recognition failure rate, and test the influence of changing condition of large span illumination intensity on test result, we carried out experiments with the fast recognition algorithm for rail surface area. These experiments were carried out under ideal conditions (light intensity suitable, uniform illumination), with the acquisition of 100 track images, taking different n values to locate, calculating the average processing time and extraction failure rate of each group of pictures. The statistical results are shown in Fig. 13.

Fig. 13

The testing results of different n

By measuring the location time and the failure rate, when n = 8, the best effect can be achieved, the location time is 4.65 ms, and the failure rate is 5%.

Whether the target area can be extracted accurately is very important to the detection system and because the portable track defect visual inspection system is mainly used for outdoor rail inspection. Therefore, it is necessary to have a strong adaptability to the change of illumination intensity. In order to verify the validity of the proposed algorithm under different light intensity and verify the difference between the performance of the algorithm based on H value and the traditional Otsu method, a comparative experiment is set up: considering three groups of light intensity range corresponding to night, morning/afternoon, and noon, respectively (500–1000 lx, 1000–10,000 lx, 10,000–100,000 lx), and two types of track were collected (wood, concrete sleeper). The result of extraction failure rate and extraction time is shown in Fig. 14.

Fig. 14

Test results of different light intensity. a Extraction failure rate. b Extraction time

It can be seen from Fig. 14 that the location algorithm based on H value is better than the location algorithm based on the maximum interclass difference method in the two parameters of the extraction failure rate and the extraction time; in different light conditions, the positioning method based on H value has good adaptability. The experimental results show that the algorithm can be applied to track inspection in complex environment and lay a good detection foundation for the portable track defect visual inspection system.

5.3 Testing experiment of rail surface defect

Due to the lack of an existing rail line that contains continuous scars and crack defects, select different lines, respectively, at different speeds through the defect area, to detect the accuracy of statistics. Among them, 20 of the scar defects and 20 crack defects were tested by 0.5 m/s, 1 m/s, 2 m/s, 4 m/s, and 6 m/s to get the best detection speed and the detection accuracy of the rail surface defects.

The experimental results (are shown in Table 3) show that the portable track defect vision detection system can work under the speed of 2 m/s (that is consistent with the people normal walking speed of 5 km/h), realizing the real-time detection of track defects.

Table 3 Rail surface defect detection results

6 Conclusion

This paper proposes a rail surface real-time visual detection method, focusing on the target location and rail surface defects contour extraction, to achieve the detection requirements through the use of the structural characteristics of the existing target to simplify the calculation. LabVIEW was used to write the algorithm program to achieve a portable track defect visual inspection system, based on the machine vision of the portable track inspection car to carry out the field experiment. Through the test of the target area location under the different sub-value, the best value of the target area location algorithm is determined. Using the image acquisition site and part of the manual synthesis, defect images verify the effectiveness of rail surface defects detection and detection time meets the requirements of real-time detection, can be in a certain extent instead of manual inspection. Due to the limited time, the portable track defect visual inspection system is only a preliminary realization of the track defect detection automation. Further research is needed: measures for preventing the influence of the uneven light on the rail surface on the defects of the rail and the station’s throat area and rapid discrimination of the occlusion; measures to complete the whole line real-time detection and other issues under the existing train speed; and measures to better facilitate the application of real-time detection of existing lines.