Apple stem/calyx detection based on phase-shifting algorithm

Abstract: Surface defect is one of the most important indexes for quality inspection of apple fruits. However, it is always difficult to distinguish between the surface defect and the stem/calyx of apples using traditional machine vision from the color information, due to the high similarity of the intensity distribution. This study aims to prevent the disturbance of the apple stem/calyx on the surface defect recognition. An effective apple stem/calyx detection was proposed using three-step phase-shifting algorithm, in terms of the three-dimensional (3D) information instead of color information. A typical fringe projection system was built with a digital projector and an industrial camera for the apple stem/calyx detection. The digital projector was used to sequentially illuminate three phase-shifting fringe patterns onto the apple sample during image acquisition. At the same time, the industrial camera was used to synchronously capture three fringe images that modulated by the apple surface. There were the outstandingly different concavity and convexity of the modulated fringe images within the stem/calyx regions from the normal. The reason was that the stem/calyx regions on the apple surface were usually concave relative to the other normal regions. The experimental analysis showed that the modulated fringe images were bending the left within the stem/calyx regions, while bending the right within the normal regions. Three-step phase-shifting algorithm was then utilized to recover the wrapped phase of the modulated fringe images that used to indicate the bending direction. Several shifted wrapped phases were computed from the original wrapped phase with the assistance of the remainder operation. The binary fringes were then obtained by simply applying threshold segmentation on these shifted wrapped phases. The convex residuals on the right side of these binary fringes were extracted using connected component labeling and two-dimensional convex hull algorithms. As such, the entire region of apple stem/calyx was detected to combine all convex residuals of these binary fringes. The total 684 group of modulated fringe images were captured from the apple samples to detect the stem/calyx of different apples with the different sizes, colors or poses, in order to validate the performance of the improved model. These modulated fringe images were mainly divided into four types: normal apple with stem/calyx, normal apple without stem/calyx, defective apple with stem/calyx, and defective apple without stem/calyx. The experiment results indicated that the proposed detection was correctly identified the apple stem/calyx under different positions and angles, even though the apple samples with the outstanding surface defects. Statistical results showed that the overall recognition rate reached 99.12%, and the recognition rate were all beyond 98.30% for the four types of the modulated fringe images. In addition, the average processing time was only about 0.479 s on MATLAB platform, fully meeting the requirement of online detection. Compared with the traditional machine vision using color information, the wrapped phase relating to 3D information was very insensitive to the surface color, indicating more suitable to distinguish the surface defect and the stem/calyx of apples. Moreover, the three fringe patterns were only required to recover the wrapped phase, compared with traditional structured light that require to reconstruct the 3D information of the apple surface. Consequently, the better performance was achieved in the higher recognition rate, the faster image acquisition speed, and the less duration of image processing. The great potential can be expected as the simple setup, high accuracy, and high speed for the stem/calyx detection of various fruits, such as apples, pears, and peaches. The finding can also provide the technical support for the quality inspection of apple surface.