Fusion of K-means and Ncut algorithm to realize segmentation and reconstruction of two overlapped apples without blocking by branches and leaves

Overlapped apples are common in natural scenes, and they can seriously affect the execution of picking task. Accurate segmentation of overlapped apples is one of the key problems that picking robot must solve. As the existing overlapped apples segmentation methods could not retain the contour of overlapped parts, a new method of segmenting overlapped apples was proposed in this paper. The steps of the algorithm were as follows: Firstly, the image was processed with morphological opening operation using disk-shaped structural element with the radius of 5 pixels so as to ensure the data information consistent in small area and to make the image more smooth. In order to distinguish the target and the background automatically, the image was then transformed from RGB color space into L*a*b color space. Then K-means clustering method was used to segment images to extract apple region. Secondly, in order to extract accurate and entire contour of apple region and keep the contour of overlapped part simultaneously, Normalized Cut (Ncut) method was adopted. To ensure the accuracy of Ncut algorithm, the cluster number was chosen as 5. Thirdly, contour segmentation method was used to extract contour of each apple. The procedures of contour segmentation method were as follows: 1) The extracted contours were connected by simply using eroding operation in order to avoid disconnection at intersection of contours of 2 adjacent apples; 2) Refine the connected contour and search three cross point, then break off the contour at three cross point by assigning zero at three cross point, and 3 contours could be got; 3) Add the 2 contours whose bending directions were opposite, and this was the contour of unblocked apple and the other contour was the contour of blocked apple. Lastly, Spline interpolation method was used to reconstruct the contour of blocked apples. In order to verify the validity of this algorithm, 20 images of adjacent apples were used to conduct the experiment, and the result was compared with the comparison method. The comparison method is a method that uses the connection line of 2 concave points to segment overlapped apples, and then utilizes the Hough transform method to reconstruct the contour of apples. The experimental results showed that for unblocked apples in apple images, average segmentation error of the presented method was 3.15%, 7.73% less than that of comparison method (10.88%). Average overlap ratio of the presented method was 96.08% and was increased by 9.71% compared to comparison method (90.85%). In addition, this method could keep the complete contour information of unblocked apples and thus improved segmentation accuracy effectively. For blocked apples, average segmentation error and average overlap ratio were 5.24% and 93.81%, respectively. The segmentation error was decreased by 11.35% and average overlap ratio was increased by 12.74% compared to comparison method, which indicated that the method could reconstruct contour of blocked apples well. In conclusion, the presented algorithm is feasible to segment and reconstruct 2 overlapped apples without blocking by branches and leaves. However, for the images of overlapped apple blocked by branches and leaves, the images of more than 2 overlapped apples, and the images of 2 overlapped apples unblocked by branches and leaves with unclear contour at overlapped part, this method cannot complete the segmentation, and thus further research would be needed.