Kinect scanning plant depth image restoration based on K-means and K-nearest neighbor algorithms

Shen Yue; Xu Hui; Liu Hui; Li Ning

doi:10.11975/j.issn.1002-6819.2016.19.026

Volume 32 Issue 19

Sep. 2016

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2016 > 32(19): 188-194. > DOI: 10.11975/j.issn.1002-6819.2016.19.026

Shen Yue, Xu Hui, Liu Hui, Li Ning. Kinect scanning plant depth image restoration based on K-means and K-nearest neighbor algorithms[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(19): 188-194. DOI: 10.11975/j.issn.1002-6819.2016.19.026

Citation:

PDF (1630 KB)

Kinect scanning plant depth image restoration based on K-means and K-nearest neighbor algorithms

School of Electrical and Information Engineering, Jiangsu University, Zhenjiang 212013, China

More Information

Received Date: February 29, 2016
Revised Date: August 22, 2016
Published Date: September 30, 2016

Graphical Abstract

Abstract

Abstract

Abstract: The Kinect sensor scanning images for agricultural plants are vulnerable to field light conditions and background noise, etc. In addition, the amount of data of the color image and depth image affect the efficiency and accuracy of the plant area, which leads to the difficulty of meeting the requirement for the Kinect sensor in agricultural plant detection. For the above problems, considering the influence of the light conditions and complex background information in agricultural environment on the quality of the plant detection and the depth data acquisition, in this paper, we proposed a plant depth detection image restoration method based on K-means and K-nearest neighbor. We also developed a novel method of image restoration to reduce the impact of background information to improve the accuracy of the color image segmentation, and to enhance the accuracy of depth data. Firstly, a RGB threshold segmentation algorithm was applied to original RGB-formatted plant color images to extract plant target areas from backgrounds. Three components R, G, and B were respectively separated from RGB color space, and the difference between G and R or B was primary extract of the plant area information. Meanwhile, for the color characteristic of the environment, a K-means clustering segmentation algorithm was performed on the extracted plant target areas to remove background noise and enhance target contours. Secondly, to fix the errors of the depth data and meet the requirements of the agricultural plant detection operations, the color image and depth image were registered to restore the suspicious pixels depth data based on K-nearest neighbor algorithm. Then, a K-Nearest Neighbor algorithm was presented to recovery the black hole pixels for depth images. Finally, we acquired the depth data of target plant from the detected images. Compared with conventional RGB threshold segmentation method and K-means algorithm method, the proposed method can be used to solve the problem of the color image noise. The experiment results showed that, the segmentation error can be reduced by 12.12% with RGB threshold segmentation method, and 41.48% with K-means algorithm method. The average segmentation error can be up to 12.33% by using RGB threshold segmentation first and then the K-means algorithm. Furthermore, the proposed method can be used to restore the depth data, and can significantly reduce the effect of the backgrounds. Thus it had a good improvement to the edge sharpness of the depth data, and the accuracy of the empty point depth data of single frame. The result of this study can be a reference for agricultural plant detection and 3D reconstruction, precision of target spraying.
- Image processing,
- image segmentation,
- algorithms,
- plant detection,
- depth data,
- image restoration,
- K-means algorithm,
- K-nearest neighbor algorithm

FullText(HTML)

References (26)

References

[1]	Tong J, Zhou J, Liu L, et al. Scanning 3D full human bodies using Kinects[J]. IEEE Transactions on Visualization & Computer Graphics, 2012, 18(4): 643－650.
[2]	马旋，薛原，杨若瑜. 基于Kinect的人体实时三维重建及其应用[J]. 计算机辅助设计与图形学学报，2014，26(10)：1720－1726.Ma Xuan, Xue Yuan, Yang Ruoyu. Kinect-based real-time 3D reconstruction of human and its application[J]. Journal of Computer-Aided Design & Computer Graphics, 2014, 26(10): 1720－1726. (in Chinese with English abstract)
[3]	申晓霞，张桦，高赞，等. 基于Kinect和金字塔特征的行为识别算法[J]. 光电子.激光,2014，25(2)：357－363.Shen Xiaoxia, Zhang Hua, Gao Zan, et al. Human behavior recohnition based on Kinect and pyramid features[J]. Journal of Optoelectronics Laser,2014, 25(2): 357－363. (in Chinese with English abstract)
[4]	Li, D, Xu, L, Tan C, et al. Digitization and visualization of greenhouse tomato plants in indoor environments[I]. Sensors 2015, 15(2): 4019－4051.
[5]	王辉，毛文华，刘刚，等. 基于视觉组合的苹果作业机器人识别与定位[J]. 农业机械学报，2012，43(12)：165－170.Wang Hui, Mao Wenhua, Liu Gang, et al. Identification and location system of multi-operation apple robot based on vision combination[J]. Transactions of the CSAM, 2012, 43(12): 165－170. (in Chinese with English abstract)
[6]	刘波，朱伟兴，杨建军，等. 基于深度图像和生猪骨架端点分析的生猪步频特征提取[J]. 农业工程学报，2014，30(10)：131－137.Liu Bo, Zhu Weixing, Yang Jianjun, et al. Extracting of pig gait frequency feature based on depth image and pig skeletonendpoints analysis[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(10): 131－137. (in Chinese with English abstract)
[7]	何东健，邵小宁，王丹，等. Kinect获取植物三维点云数据的去噪方法[J] . 农业机械学报，2016，47(1)：331－336.He Dongjian, Shao Xiaoning, Wang Dan, et al. An effective denoising method of plants' 3D point cloud data obtained by kinect[J]. Transactions of the CSAM, 2016, 47(1): 331－336. (in Chinese with English abstract)
[8]	徐欢，郑江滨. Kinect深度图像修复算法[J]. 微电子学与计算机，2013(11)：98－103.Xu Huan, Zheng Jiagbin. A Novel inpainting algorithm for kinect depth image[J]. Microelectronics & Computer, 2013(11): 98－103. (in Chinese with English abstract)
[9]	李应彬，冯杰，张华熊，等. 基于改进双边滤波的Kinect深度图像空洞修复算法研究[J]. 工业控制计算机，2013，26(11)：105－106，109.Li Yingbin, Feng Jie, Zhang Huaxiong, et al. New algorithm of depth hole filling based on intensive bilateral filter[J]. Industrial Control Computer, 2013, 26(11): 105－106, 109. (in Chinese with English abstract)
[10]	白树斌，张维忠，袁翠梅，等. 基于RGB-D图像的深度图空洞修复[J]. 青岛大学学报：自然科学版，2015，28(1)：61－63.Bai Shubin, Zhang Weizhong, Yuan Cuimei, et al. Filling holes in depth map based on RGB-D Image[J]. Journal of Qingdao University: Natural Science Edition, 2015, 28(1): 61－63. (in Chinese with English abstract)
[11]	郭连朋，陈向宁，刘彬，等. 基于Kinect传感器多深度图像融合的物体三维重建[J]. 应用光学，2014(5)：811－816.Guo Lianpeng, Chen Xiangning, Liu Bin, et al. 3D-object reconstruction based on fusion of depth images by Kinect sensor[J]. Journal of Applied Optics, 2014(5): 811－816. (in Chinese with English abstract)
[12]	Camplan M, Salgado L. Efficient spatio-temporal hole filling strategy for kinect depth maps[C]//Three-dimensional Image Processing and Applications. Burlingame: Proceedings of SPIE, 2012, 82900E.
[13]	Hu Jinhui, Hu Ruimin, Wang Zhongyuan, et al. Color image guided locality regularized represention for kinect depth holes filling[C].//Visual Communications and Image Processing(VCIP). Kuching: IEEE, 2013: 1－6.
[14]	王勇，蒋爱民，胥立波. 基于k-means的Kinect深度图像空洞修复算法[J]. 微处理机，2015(4)：42－44，48.Wang Yong, Jiang Aimin, Xu Libo. Kinect depth hole filling algorithm based on K-means[J]. Microprocessors, 2015(4): 42－44,48. (in Chinese with English abstract)
[15]	王奎，安平，张兆杨，等. Kinect深度图像快速修复算法[J]. 上海大学学报：自然科学版，2012，18(5)：454－458.Wang Kui, An Ping, Zhang Zhaoyang, et al. Fast inpainting algorithm for kinect depth map[J]. Jornalof Shanghai University(Natural Science Edition), 2012, 18(5): 454－458. (in Chinese with English abstract)
[16]	王奎，安平，张艳，等. 基于Kinect的实时深度提取与多视绘制算法[J]. 光电子·激光，2012(10)：1949－1956.Wang Kui, An Ping, Zhang Yan, et al. Real-time depth extraction and multi-view rendering algorithm based on Kinect[J]. Journal of Optoelectronics Laser, 2012(10): 1949－1956. (in Chinese with English abstract)
[17]	王壮，胡卫东，郁文贤，等. 一种基于近邻搜索的快速k-近邻分类算法[J]. 系统工程与电子技术，2002，24(4)：100－102.Wang Zhuang, Hu Weidong, Yu Wenxian, et al. A quick k-nearest neighbour classification algorithm based on near neighbour searching[J]. Systems Engineering and Electronics, 2002, 24(4): 100－102. (in Chinese with English abstract)
[18]	黄淼，张海朝，李超. 基于八叉树空间分割的k近邻搜索算法[J]. 计算机应用，2008，28(8)：2046－2048，2051.Huang Miao, Zhang Haichao, Lichao. Algorithm for finding k-nearest neighbors based on octree segmentation in space[J]. Journal of Computer Applications, 2008, 28(8):2046－2048, 2051. (in Chinese with English abstract)
[19]	夏东垚，邹焕新，雷琳，等. 基于K近邻局部结构的截断二次谱匹配算法[J]. 系统工程与电子技术，2014(11)：2138－2142.Xia Dongyao, Zou Huanxin, LeiLin, et al. Truncated secondary spectral matching method based on K neighboring local structure[J]. Systems Engineering and Electronics, 2014(11): 2138－2142. (in Chinese with English abstract)
[20]	韩殿元，黄心渊，付慧. 基于彩色通道相似性图像分割方法的植物叶面积计算[J]. 农业工程学报，2012，28(6)：179－183.Han Dianyuan, Huang Xinyuan, Fu Hui. Measurement of plant leaf area based on image segmentation of color channel similarity[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2012, 28(6): 179－183. (in Chinese with English abstract)
[21]	Malinen M I, Mariescu-Istodor R, Franti P. K-means*: clustering by gradual data transformation[J]. Pattern Recognition, 2014, 47(10): 3376－3386.
[22]	余涛. Kinect应用开发实战：用最自然的方式与机器对话[M]. 北京：机械工业出版社，2012.
[23]	Danciu G, Banu S M, Caliman A. Shadow removal in depth images morphology-based for Kinect cameras[C]//IEEE International Conference on System Theory, Control and Computing, 2012: 1－6.
[24]	石欣，印爱民，张琦. 基于K最近邻分类的无线传感器网络定位算法[J]. 仪器仪表学报，2014，35(10)：2238－2247.Shi Xin, Yin Aimin, Zhang Qi. Localization in wireless sensor networks based on K-Nearest Neighbor[J]. Chinese Journal of Scientific Instrument, 2014, 35(10): 2238－2247. (in Chinese with English abstract)
[25]	张闯，王婷婷，孙冬娇，等. 基于欧氏距离图的图像边缘检测[J]. 中国图象图形学报，2013，18(2)：176－183.Zhang Chuang, Wang Tingting, Sun Dongjiao, et al. Image edge detection based on the Euclidean distance graph[J]. Journal of Image and Graphics, 2013, 18(2): 176－183. (in Chinese with English abstract)
[26]	徐黎明，吕继东. 基于同态滤波和K均值聚类算法的杨梅图像分割[J]. 农业工程学报，2015，31(14)：202－208.Xu Liming, Lü Jidong. Bayberry image segmentation based on homomorphic filtering and K-means clustering algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(14): 202－208. (in Chinese with English abstract)