Real-time extraction of navigation line between corn rows

Yang Yang; Zhang Boli; Zha Jiayi; Wen Xing; Chen Liqing; Zhang Tie; Dong Xiang; Yang Xuejun

doi:10.11975/j.issn.1002-6819.2020.12.020

Volume 36 Issue 12

Jun. 2020

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2020 > 36(12): 162-171. > DOI: 10.11975/j.issn.1002-6819.2020.12.020

Yang Yang, Zhang Boli, Zha Jiayi, Wen Xing, Chen Liqing, Zhang Tie, Dong Xiang, Yang Xuejun. Real-time extraction of navigation line between corn rows[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(12): 162-171. DOI: 10.11975/j.issn.1002-6819.2020.12.020

Citation:

PDF (6703 KB)

Real-time extraction of navigation line between corn rows

1.
School of Engineering, Anhui Agricultural University, Hefei 230036, China
2.
Anhui Intelligent Agricultural Machinery Equipment Engineering Laboratory, Hefei 230036, China
3.
Chinese Academy of Agricultural Mechanization Sciences, Beijing 100083, China

More Information

Received Date: November 25, 2020
Revised Date: May 13, 2020
Published Date: June 14, 2020

Graphical Abstract

Abstract

Abstract

High-clearance plant protection machine plays an important role in the large-scale chemical fertilizer spraying machinery in China. The wheels of high-clearance plant protection machine must be driven along the ridge during the working process. However, the driver's vision is blocked by corn plants, which is easy to cause corn seedlings damaged by tires. In view of the serious phenomenon of seedling injury during the operation of high-clearance plant protection machine, a method of real-time extraction of navigation line between the corn rows based on a dynamic Region of Interest (ROI) was proposed in this paper. According to the different growing periods of corn, the videos were collected when the average height of corn plant was 30, 50 and 70 cm under different light conditions. Through reducing the influence of light by pixel normalization, segmenting the image of corn and background by green image processing algorithm and the maximum class square error method, the images were enhanced and removed by morphological processing. Firstly, the collected image was divided into 10 image bands, and each image band was processed by vertical projection method to locate the center lines between crop rows in the first frame of the video. The approximate position of the static ROI was determined and marked based on the perspective principle of camera and geometric information of the corn seedling belts. It took 473.7 ms to divide the static ROI, which does not affect the real-time performance of navigation since the determined of static ROI was prior to the navigation operation. Secondly, the static ROI was applied in the second frame of video, in which the navigation line between the corn rows was obtained by sub-regional feature points clustering algorithm. According to the geometric information of navigation line and ROI, the ROI was updated and optimized. Then the updated ROI was applied to the navigation line extraction of the next video frame. 225 consecutive videos were selected to divide the ROI manually, and the dynamic ROI obtained by proposed method was compared with the artificial ROI. The pixel deviation between the artificial ROI and the ROI was about 10 pixels, which shown that the algorithm had good convergence and met the requirements of navigation accuracy. Finally, a comparative experiment of artificial navigation line extraction and the algorithm in this paper by randomly selecting 100 frames of image. The results showed that the algorithm proposed by this paper could automatically obtain the dynamic ROI, the average error of navigation line extraction between manual and algorithm in this paper was 1.157°, and the accuracy rate of corn seedling belt recognitionwas 96%, and the average processing time of a real-time image with a resolution of 1 920 pixels×1 080 pixels was 97.56 ms. July 16, 2019, the navigation experiment was carried out in corn field, and the heading angle was used as the input signal of the automatic steering control system. The results showed that the wheels of high-clearance plant protection machine walked along the corn rows, and the wheel angle floated up and down at 0°. The algorithm proposed in this paper can meet the accuracy and real-time requirements of the automatic walking for high-clearance plant protection machine in corn.
- agricultural machinery,
- machine vision,
- navigation,
- corn,
- plant protection machine,
- dynamic ROI

FullText(HTML)

References (31)

References

[1]	李志刚，傅泽田，李丽勤. 基于机器视觉的农业植保技术研究进展[J]. 农业机械学报，2005，36(8)：143-146.Li Zhigang, Fu Zetian, Li Liqin. Advance in agricultural plant protection technology based on machine vision[J]. Transactions of the Chinese Society for Agricultural Machinery, 2005, 36(8): 143-146. (in Chinese with English abstract)
[2]	陈艳，张漫，马文强，等. 基于GPS和机器视觉的组合导航定位方法[J]. 农业工程学报，2011，27(3)：126-130.Chen Yan, Zhang Man, Ma Wenqiang, et al. Positioning method of integrated navigation based on GPS and machine vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2011, 27(3): 126-130. (in Chinese with English abstract)
[3]	田光兆，顾宝兴，Irshad Ali Mari，等. 基于三目视觉的自主导航拖拉机行驶轨迹预测方法及试验[J]. 农业工程学报，2018，34(19)：40-45.Tian Guangzhao, Gu Baoxing, Irshad Ali Mari, et al. Traveling trajectory prediction method and experiment of autonomous navigation tractor based on trinocular vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(19): 40-45. (in Chinese with English abstract)
[4]	靳标，李建行，朱德宽，等. 基于自适应有限冲激响应-卡尔曼滤波算法的GPS/INS导航[J]. 农业工程学报，2019，35(3)：75-81.Jin Biao, Li Jianxing, Zhu Dekuan, et al. GPS/INS navigation based on adaptive finite impulse response-Kalman filter algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(3): 75-81. (in Chinese with English abstract)
[5]	魏爽，李世超，张漫，等. 基于GNSS的农机自动导航路径搜索及转向控制[J]. 农业工程学报，2017，33(增刊1)：70-77.Wei Shuang, Li Shichao, Zhang Man, et al. Automatic navigation path search and turning control of agricultural machinery based on GNSS[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(Supp.1): 70-77. (in Chinese with English abstract)
[6]	梁习卉子，陈兵旗，姜秋慧，等. 基于图像处理的玉米收割机导航路线检测方法[J]. 农业工程学报，2016，32(22)：43-49.Liangxi Huizi, Chen Bingqi, Jiang Qiuhui, et al. Detection method of navigation route of corn harvester based on image processing[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(22): 43-49. (in Chinese with English abstract)
[7]	彭顺正，坎杂，李景彬. 矮化密植枣园收获作业视觉导航路径提取[J]. 农业工程学报，2017，33(9)：45-52.Peng Shunzheng, Kan Za, Li Jingbin. Extraction of visual navigation directrix for harvesting operation in short-stalked and close-planting jujube orchard[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(9): 45-52. (in Chinese with English abstract)
[8]	李云伍，徐俊杰，王铭枫，等. 丘陵山区田间道路自主行驶转运车及其视觉导航系统研制[J]. 农业工程学报，2019，35(1)：52-61.Li Yunwu, Xu Junjie, Wang Mingfeng, et al. Development of autonomous driving transfer trolley on field roads and its visual navigation system for hilly areas[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(1): 52-61. (in Chinese with English abstract)
[9]	刘兆朋，张智刚，罗锡文，等. 雷沃ZP9500高地隙喷雾机的GNSS自动导航作业系统设计[J]. 农业工程学报，2018，34(1)：15-21.Liu Zhaopeng, Zhang Zhigang, Luo Xiwen, et al. Design of automatic navigation operation system for Lovol ZP9500 high clearance boom sprayer based on GNSS[J]. Transactions of the Chinese Society of Agricultural Engineering. (Transactions of the CSAE), 2018, 34(1): 15-21. (in Chinese with English abstract)
[10]	陈黎卿，许鸣，柏仁贵，等. 高地隙植保机辅助驾驶系统设计与试验[J]. 农业机械学报，2019，50(9)：25-32.Chen Liqing, Xu Ming, Bai Rengui, et al. Design and test of auxiliary driving system for high-gap plant protection machine[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(9): 25-32. (in Chinese with English abstract)
[11]	夏祥孟. 植保机器人全局路径规划与控制系统设计[D]. 济南：济南大学，2019.Xia Xiangmeng. Design of Global Path Planning and Control System for Plant Protection Robot[D]. Jinan: Jinan University, 2019. (in Chinese with English abstract)
[12]	Seong In Cho, Jae Hoon Lee. Autonomous speed sprayer using differential global positioning system, genetic algorithm and fuzzy control[J]. Journal of Agricultural Engineering Research, 2000, 76(2): 111-119.
[13]	杨为民，李天石，贾鸿社. 农业机械机器视觉导航研究[J].农业工程学报，2004，20(1)：160-165.Yang Weimin, Li Tianshi, Jia Hongshe. Simulation and experiment of machine vision guidance of agriculture vehicles[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2004, 20(1): 160-165. (in Chinese with English abstract)
[14]	张铁民，庄晓霖. 基于DM642的高地隙小车的田间路径识别导航系统[J]. 农业工程学报，2015，31(4)：160-167.Zhang Tiemin, Zhuang Xiaolin. Identification and navigation system of farmland path for high-clearance vehicle based on DM642[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(4): 160-167. (in Chinese with English abstract)
[15]	廖娟，汪鹞，尹俊楠，等. 基于分区域特征点聚类的秧苗行中心线提取[J]. 农业机械学报，2019，50(11)：34-41.Liao Juan, Wang Yao, Yin Junnan, et al. Detection of seedling row centerlines based on sub-regional feature points clustering[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(11): 34-41. (in Chinese with English abstract)
[16]	宋宇，刘永博，刘路，等. 基于机器视觉的玉米根茎导航基准线提取方法[J]. 农业机械学报，2017，48(2)：38-44.Song Yu, Liu Yongbo, Liu Lu, et al. Extraction method of navigation baseline of corn roots based on machine vision[J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(2): 38-44. (in Chinese with English abstract)
[17]	杨洋，张亚兰，苗伟，等. 基于卷积神经网络的玉米根茎精确识别与定位研究[J]. 农业机械学报，2018，49(10)：46-53.Yang Yang, Zhang Yalan, Miao Wei, et al. Accurate identification and location of corn rhizome based on faster R-CNN[J]. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49(10): 46-53. (in Chinese with English abstract)
[18]	José Miguel Guerrero,José Jaime Ruz,Gonzalo Pajares. Crop rows and weeds detection in maize fields applying a computer vision system based on geometry[J]. Computers and Electronics in Agriculture, 2017, 142：461-472.
[19]	Guoquan Jiang, Xiaojie Wang, Zhiheng Wang, et al. Wheat rows detection at the early growth stage based on Hough transform and vanishing point[J]. Computers and Electronics in Agriculture, 2016, 123：211-223.
[20]	Montalvo M, Pajares G, Guerrero J M, et al. Automatic detection of crop rows in maize fields with high weeds pressure[J]. Expert Systems With Applications, 2012, 39(15)：11889-11897.
[21]	刁智华，赵明珍，宋寅卯，等. 基于机器视觉的玉米精准施药系统作物行识别算法及系统实现[J]. 农业工程学报，2015，31(7)：47-52.Diao Zhihua, Zhao Mingzhen, Song Yinmao, et al. Crop line recognition algorithm and realization in precision pesticide system based on machine vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(7): 47-52. (in Chinese with English abstract)
[22]	司永胜，姜国权，刘刚，等. 基于最小二乘法的早期作物行中心线检测方法[J]. 农业机械学报，2010，41(7)：163-167.Si Yongsheng, Jiang Guoquan, Liu Gang. Crop rows detection based on least square method[J]. Transactions of the Chinese Society for Agricultural Machinery, 2010, 41(7): 163-167. (in Chinese with English abstract)
[23]	姜国权，杨小亚，王志衡，等. 基于图像特征点粒子群聚类算法的麦田作物行检测[J]. 农业工程学报，2017，33(11)：165-170.Jiang Guoquan, Yang Xiaoya, Wang Zhiheng, et al. Crop rows detection based on image characteristic point and particle swarm optimization-clustering algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(11): 165-170. (in Chinese with English abstract)
[24]	吕朝辉，陈晓光，吴文福，等. 用BP神经网络进行秧苗图像分割[J]. 农业工程学报，2001，17(3)：146-148.Lü Chaohui, Chen Xiaoguang, Wu Wenfu, et al. Segmentation of the Seedling Image by BP Neural Network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2001, 17(3): 146-148. (in Chinese with English abstract)
[25]	王新忠，韩旭，毛罕平，等. 基于最小二乘法的温室番茄垄间视觉导航路径检测[J]. 农业机械学报，2012，43(6)：161-166.Wang Xinzhong, Han Xu, Mao Hanping, et al. Navigation line detection of tomato ridges in greenhouse based on least square method[J]. Transactions of the Chinese Society for Agricultural Machinery, 2012, 43(6): 161-166. (in Chinese with English abstract)
[26]	冯娟，刘刚，司永胜，等. 果园视觉导航基准线生成算法[J]. 农业机械学报，2012，43(7)：185-189，184.Feng Juan, Liu Gang, Si Yongsheng, et al. Algorithm based on image processing technology to generate navigation directrix in orchard[J]. Transactions of the Chinese Society for Agricultural Machinery, 2012, 43(7): 185-189,184. (in Chinese with English abstract)
[27]	Esmael Hamuda, Martin Glavin, Edward Jones. A survey of image processing techniques for plant extraction and segmentation in the field[J]. Computers and Electronics in Agriculture, 2016, 125：184-199.
[28]	Ronghua Ji, Lijun Qi. Crop-row detection algorithm based on Random Hough Transformation[J]. Mathematical and Computer Modelling, 2011, 54(3):1016-2020.
[29]	籍颖，张漫，刘刚，等. 基于改进粒子滤波的农用车辆导航定位方法[J]. 农业工程学报，2011，27(8)：227-231.Ji Ying, Zhang Man, Liu Gang, et al. Positioning method of vehicle navigation system based on improved particle filter[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2011, 27(8): 227-231. (in Chinese with English abstract)
[30]	赵德安，贾伟宽，张云，等. 农业机器人自主导航改进自适应滤波控制器研究[J]. 农业机械学报，2015，46(5)：1-6.Zhao Dean, Jia Weikuan, Zhang Yun, et al. Design of agricultural robot autonomous navigation control based on improved self-adaptive filter[J]. Transactions of the Chinese Society for Agricultural Machinery, 2015, 46(5): 1-6.(in Chinese with English abstract)
[31]	梁喜凤，彭明，路杰，等. 基于自适应无迹卡尔曼滤波的采摘机械手视觉伺服控制方法[J]. 农业工程学报，2019，35(19)：230-237.Liang Xifeng, Peng Ming, Lu Jie, et al. Servo control method of picking manipulator based on adaptive traceless Kalman filter[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(19): 230-237. (in Chinese with English abstract)

[1]	Yang Yang, Ma Qianglong, Chen Zhizheng, Wen Xing, Zhang Gang, Zhang Tie, Dong Xiang, Chen Liqing. Real-time extraction of the navigation lines between sugarcane ridges using LiDAR[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(4): 178-185. DOI: 10.11975/j.issn.1002-6819.2022.04.021
[2]	Liang Yajie, Yang Lili, Xu Yuanyuan, Chen Zhibo, Feng Yarong, Wu Caicong. Dynamic path planning method for multiple unmanned agricultural machines in uncertain scenarios[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(21): 1-8. DOI: 10.11975/j.issn.1002-6819.2021.21.001
[3]	Li Xiangguang, Zhao Wei, Zhao Leilei. Extraction algorithm of the center line of maize row in case of plants lacking[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(18): 203-210. DOI: 10.11975/j.issn.1002-6819.2021.18.024
[4]	Qian Fengkui, Zhou Yang, Gu Hanlong. Dynamic evolution analysis of basic farmland protection research in China from 1988 to 2017[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(19): 272-282. DOI: 10.11975/j.issn.1002-6819.2020.19.031
[5]	Dai Xiang, Xu Youlin, Chen Junyang, Zheng Jiaqiang, Song Haichao, Ma Luqiang. Improved jet mixer injection nozzle enhancing pesticide mixing uniformity and dynamic concentration consistency[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(8): 65-74. DOI: 10.11975/j.issn.1002-6819.2019.08.008
[6]	Zhao Xiaoli, Ma Xinming, Wang Jucai, Ye Jing. Dynamic prediction on leaf length of maize based on metabolic model GM(1,1)[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2013, 29(10): 183-189.
[7]	Lin Jiachun, Li Wei, Xun Yi. Camera calibration technique for multi-object dynamical testing[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2005, 21(6): 18-21.
[8]	Wu Jianmin. Dynamic Study on the Economic Life Span of Farm Machines[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2000, 16(4): 28-31.
[9]	ZHENG Lian-zhu, ZHANG You-kun, LIU Ming-shu, GE Jian-min. Relationship Between the Tyre Dynamic Parameter and Torsional Vibration in Powertrain[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 1999, 15(3): 49-52.
[10]	Wu Jianmin. The Dynamic Study on Optimum Renewal Opportunity of the Small Tractors[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 1998, 14(2): 78-81.