Kinect scanning plant depth image restoration based on K-means and K-nearest neighbor algorithms

Abstract: The Kinect sensor scanning images for agricultural plants are vulnerable to field light conditions and background noise, etc. In addition, the amount of data of the color image and depth image affect the efficiency and accuracy of the plant area, which leads to the difficulty of meeting the requirement for the Kinect sensor in agricultural plant detection. For the above problems, considering the influence of the light conditions and complex background information in agricultural environment on the quality of the plant detection and the depth data acquisition, in this paper, we proposed a plant depth detection image restoration method based on K-means and K-nearest neighbor. We also developed a novel method of image restoration to reduce the impact of background information to improve the accuracy of the color image segmentation, and to enhance the accuracy of depth data. Firstly, a RGB threshold segmentation algorithm was applied to original RGB-formatted plant color images to extract plant target areas from backgrounds. Three components R, G, and B were respectively separated from RGB color space, and the difference between G and R or B was primary extract of the plant area information. Meanwhile, for the color characteristic of the environment, a K-means clustering segmentation algorithm was performed on the extracted plant target areas to remove background noise and enhance target contours. Secondly, to fix the errors of the depth data and meet the requirements of the agricultural plant detection operations, the color image and depth image were registered to restore the suspicious pixels depth data based on K-nearest neighbor algorithm. Then, a K-Nearest Neighbor algorithm was presented to recovery the black hole pixels for depth images. Finally, we acquired the depth data of target plant from the detected images. Compared with conventional RGB threshold segmentation method and K-means algorithm method, the proposed method can be used to solve the problem of the color image noise. The experiment results showed that, the segmentation error can be reduced by 12.12% with RGB threshold segmentation method, and 41.48% with K-means algorithm method. The average segmentation error can be up to 12.33% by using RGB threshold segmentation first and then the K-means algorithm. Furthermore, the proposed method can be used to restore the depth data, and can significantly reduce the effect of the backgrounds. Thus it had a good improvement to the edge sharpness of the depth data, and the accuracy of the empty point depth data of single frame. The result of this study can be a reference for agricultural plant detection and 3D reconstruction, precision of target spraying.