Quantity statistics of spruce under UAV aerial videos using YOLOv3 and SORT

Chen Fengjun; Zhu Xueyan; Zhou Wenjing; Zheng Yili; Gu Mengmeng; Zhao Yandong

doi:10.11975/j.issn.1002-6819.2021.20.009

Volume 37 Issue 20

Oct. 2021

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2021 > 37(20): 81-89. > DOI: 10.11975/j.issn.1002-6819.2021.20.009

Chen Fengjun, Zhu Xueyan, Zhou Wenjing, Zheng Yili, Gu Mengmeng, Zhao Yandong. Quantity statistics of spruce under UAV aerial videos using YOLOv3 and SORT[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(20): 81-89. DOI: 10.11975/j.issn.1002-6819.2021.20.009

Citation:

PDF (40320 KB)

Quantity statistics of spruce under UAV aerial videos using YOLOv3 and SORT

1.
School of Technology, Beijing Forestry University, Beijing 100083, China
2.
Beijing Laboratory of Urban and Rural Ecological Environment, Beijing 100083, China
3.
Department of Horticultural Science, Texas A&M University, College Station TX 77843, USA
4.
Key Laboratory of State Forestry Administration for Forestry Equipment and Automation, Beijing 100083, China

More Information

Received Date: June 26, 2021
Revised Date: October 05, 2021
Published Date: October 14, 2021

Graphical Abstract

Abstract

Abstract

Abstract: A seedling quantity is a key indicator to predict the actual production, supply, and demand for the operation and management of a nursery. The manual visualization has still dominated the statistics for the number of seedlings in complete plots. However, the application needs cannot be fully met in recent years, such as high cost, low efficiency, and slow data update. Therefore, it is necessary to fast and accurately estimate the number of seedlings in the whole plots. Taking the spruce as the research object, this study aims to propose a quantity statistics approach under Unmanned Aerial Vehicle (UAV) aerial videos using YOLOv3 and SORT. The specific procedure included the data acquisition, YOLOv3 detection model, SORT tracking, and cross-line counting. Two areas were divided for the image and video acquisition, each with 6 complete test plots. In the stage of data acquisition, 558 images and 6 videos were captured by a DJI Phantom 4 (UAV). The quantity statistics dataset was then constructed with the acquired images and videos, where the training dataset contained 558 images, and the test dataset contained 6 videos. Subsequently, a YOLOv3 model was selected to detect the spruce, while a SORT model was to track the spruce, and the cross-line counting to count the number of spruce. The performance of the combined YOLOv3+SORT was also quantitatively evaluated using Mean Count Accuracy (MCA), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), and Frame Rate (FR). It was found that the MCA of 92.30%, MAE of 72, RMSE of 98.85, and FR of 11.5 frames/s for the test dataset in the quantity statistics. The experimental results showed that quick and accurate counting was achieved for the number of spruce in the complete plots. The YOLOv3+SORT was also compared with the SSD+SORT and Faster R-CNN+SORT, in order to further verify the performance of the model. The results showed that the YOLOv3+SORT performed over the SSD+SORT in all four evaluation indexes. Particularly, the YOLOv3+SORT was much faster with higher guaranteed accuracy, with 1.33 percentage points lower MCA, and 10.1 frames/s higher FR, compared with the Faster R-CNN+SORT. In summary, the quantity statistics using YOLOv3 and SORT can be widely expected to serve as an effective way to rapidly and accurately count the number of seedlings in the whole plots. This study can also offer promising potential support to the seedling quantity statistics from the perspective of UAV aerial videos.
- unmanned aerial vehicle,
- model,
- algorithm,
- spruce,
- quantity statistics,
- YOLOv3,
- SORT

FullText(HTML)

References (42)

References

[1]	Hao Z, Lin L, Liu J, et al. Automated tree-crown and height detection in a young forest plantation using mask region-based convolutional neural network (Mask R-CNN)[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 178: 112-123.
[2]	Xiong H, Cao Z, Lu H, et al. TasselNetv2: In-field counting of wheat spikes with context-augmented local regression networks[J]. Plant Methods, 2019, 15: 150.
[3]	肖德琴，张玉康，范梅红，等. 基于视觉感知的蔬菜害虫诱捕计数算法[J]. 农业机械学报，2018，49(3)：51-58.Xiao Deqin, Zhang Yukang, Fan Meihong, et al. Vegetable pest counting algorithm based on visual perception[J]. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49(3): 51-58. (in Chinese with English abstract)
[4]	梁习卉子，陈兵旗，李民赞，等. 质心跟踪视频棉花行数动态计数方法[J]. 农业工程学报，2019，35(2)：175-182.Liang Xihuizi, Chen Bingqi, Li Minzan, et al. Dynamic counting method of cotton rows in video based on centroid tracking[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(2): 175-182. (in Chinese with English abstract)
[5]	Li D, Guo H, Wang C, et al. Individual tree delineation in windbreaks using airborne-laser-scanning data and unmanned aerial vehicle stereo images[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(9): 1330-1334.
[6]	Wang J, Chen X, Cao L, et al. Individual rubber tree segmentation based on ground-based LiDAR data and Faster R-CNN of deep learning[J]. Forests, 2019, 10(9): 793.
[7]	赵静，潘方江，兰玉彬，等. 无人机可见光遥感和特征融合的小麦倒伏面积提取[J]. 农业工程学报，2021，37(3)：73-80.Zhao Jing, Pan Fangjiang, Lan Yubin, et al. Wheat lodging area extraction using UAV visible light remote sensing and feature fusion[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(3): 73-80. (in Chinese with English abstract)
[8]	Mekhalfi M, Nicolo C, Lanniello L. Vision system for automatic on-tree kiwifruit counting and yield estimation[J]. Sensors, 2020, 20(15): 4214.
[9]	Payne A, Walsh K, Subedi P. Estimation of mango crop yield using image analysis - Segmentation method[J]. Computers and Electronics in Agriculture, 2013, 91: 57-64.
[10]	Jiang H, Chen S, Li D, et al. Papaya tree detection with UAV images using a GPU-accelerated scale-space filtering method[J]. Remote Sensing, 2017, 9(7): 721.
[11]	Dorj U, Lee M, Han S. A comparative study on tangerine detection counting and yield estimation algorithm[J]. Journal of Information Security and Applications, 2013, 7(3): 405-412.
[12]	Waleed M, Um T, Khan A. Automatic detection system of olive trees using improved k-means algorithm[J]. Remote Sensing, 2020, 12(5): 760.
[13]	Hassler S, Baysal-Gurel F. Unmanned aircraft system(UAS) technology and applications in agriculture[J]. Agronomy-Basel, 2019, 9(10): 618.
[14]	Dilek K, Serdar S, Nathan A, et al. Automatic citrus tree extraction from UAV images and digital surface models using circular Hough transform[J]. Computers and Electronics in Agriculture, 2018, 150: 289-301.
[15]	Linker R, Cohen O, Naor A. Determination of the number of green apples in RGB images recorded in orchards[J]. Computers and Electronics in Agriculture, 2012, 81(1): 45-57.
[16]	Scott J, Gent D, Hay F, et al. Estimation of pyrethrum flower number using digital imagery[J]. Horttechnology, 2015, 25(5): 617-624.
[17]	李艳肖，胡雪桃，张芳，等. 基于高光谱技术的菌落图像分割与计数[J]. 农业工程学报，2020，36(20)：326-332.Li Yanxiao, Hu Xuetao, Zhang Fang, et al. Colony image segmentation and counting based on hyperspectral technology[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(20): 326-332. (in Chinese with English abstract)
[18]	Bargoti S, Underwood J. Image segmentation for fruit detection and yield estimation in apple orchards[J]. Journal of Field Robotics, 2017, 34(6): 1039-1060.
[19]	Wu B, Yu B, Wu Q, et al. Individual tree crown delineation using localized contour tree method and airborne LiDAR data in coniferous forests[J]. International Journal of Applied Earth Observation and Geoinformation, 2016, 52: 82-94.
[20]	杭艳红，苏欢，于滋洋，等. 结合无人机光谱与纹理特征和覆盖度的水稻叶面积指数估算[J]. 农业工程学报，2021，37(9)：64-71.Hang Yanhong, Su Huan, Yu Ziyang, et al. Estimation of rice leaf area index combining UAV spectrum, texture features and vegetation coverage[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(9): 64-71. (in Chinese with English abstract)
[21]	梁习卉子，陈兵旗，李民赞，等. 基于HOG特征和SVM的棉花行数动态计数方法[J]. 农业工程学报，2020，36(15)：173-181.Liang Xihuizi, Chen Bingqi, Li Minzan, et al. Method for dynamic counting of cotton rows based on HOG feature and SVM[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(15): 173-181. (in Chinese with English abstract)
[22]	刘哲，黄文准，王利平. 基于改进K-means聚类算法的大田麦穗自动计数[J]. 农业工程学报，2019，35(3)：174-181.Liu Zhe, Huang Wenzhun, Wang Liping. Field wheat ear counting automatically based on improved K-means clustering algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(3): 174-181. (in Chinese with English abstract)
[23]	李莉，王宏康，吴勇，等. 基于K-means聚类算法的草莓灌溉策略研究[J]. 农业机械学报，2020，51(1)：295-302. Li Li, Wang Hongkang, Wu Yong, et al. Investigation of strawberry irrigation strategy based on K-means clustering algorithm[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(1): 295-302. (in Chinese with English abstract)
[24]	Shang L, Guo W, Nelson S. Apple variety identification based on dielectric spectra and chemometric methods[J]. Food Analytical Methods, 2015, 8(4): 1042-1052.
[25]	徐洋，陈燚，黄磊，等. 基于多层BP神经网络和无参数微调的人群计数方法[J]. 计算机科学，2018，45(10)：235-239. Xu Yang, Chen Yan, Huang Lei, et al. Crowd counting method based on multilayer BP neural networks and Non-parameter tuning[J]. Computer Science, 2018, 45(10): 235-239. (in Chinese with English abstract)
[26]	李毅念，杜世伟，姚敏，等. 基于小麦群体图像的田间麦穗计数及产量预测方法[J]. 农业工程学报，2018，34(21)：185-194. Li Yinian, Du Shiwei, Yao Min, et al. Method for wheatear counting and yield predicting based on image of wheatear population in field[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(21): 185-194. (in Chinese with English abstract)
[27]	Djerriri K, Ghabi M, Karoui M, et al. Palm trees counting in remote sensing imagery using regression convolutional neural network[C]//IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain: IEEE, 2018.
[28]	孙壮壮，姜东，蔡剑，等. 单子叶作物叶片气孔自动识别与计数技术[J]. 农业工程学报，2019，35(23)：170-176. Sun Zhuangzhuang, Jiang Dong, Cai Jian, et al. Automatic identification and counting of leaf stomata of monocotyledonous crops[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(23): 170-176. (in Chinese with English abstract)
[29]	Xie H, Fan Z, Li W, et al. Tobacco plant recognizing and counting based on SVM[C]// 2016 International Conference on Industrial Informatics - Computing Technology, Intelligent Technology, Wuhan, China: IEEE, 2016.
[30]	姜海燕，徐灿，陈尧，等. 基于田间图像的局部遮挡小尺寸稻穗检测和计数方法[J]. 农业机械学报，2020，51(9)：152-162. Jiang Haiyan, Xu Can, Chen Yao, et al. Detecting and counting method for small-sized and occluded rice panicles based on in-field images[J]. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51(9): 152-162. (in Chinese with English abstract)
[31]	易诗，沈练，周思尧，等. 基于增强型Tiny-YOLOV3模型的野鸡识别方法[J]. 农业工程学报，2020，36(13)：141-147. Yi Shi, Shen Lian, Zhou Siyao, et al. Recognition method of pheasant using enhanced Tiny-YOLOV3 model[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(13): 141-147. (in Chinese with English abstract)
[32]	Osco L, Arruda M, Junior J, et al. A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 160: 97-106.
[33]	Fan Z, Lu J, Gong M, et al. Automatic tobacco plant detection in UAV images via deep neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(3): 876-887.
[34]	Raul M, Lodewijk V, Joris I. DiSCount: computer vision for automated quantification of striga seed germination[J]. Plant Methods, 2020, 16(1): 60.
[35]	Zhou W, Zhu X, Chen F. Spruce counting based on Lightweight Mask R-CNN with UAV Images[J]. International Journal of Circuits, Systems and Signal Processing, 2021, 15: 634-642.
[36]	Wang D, Fu Y, Yang X, et al. Combined use of FCN and harris corner detection for counting wheat ears in field conditions[J]. IEEE Access, 2019, 7: 178930-178941.
[37]	Li W, Fu H, Yu L, et al. Deep learning based oil palm tree detection and counting for high-resolution remote sensing images[J]. Remote Sensing, 2017, 9(1): 22.
[38]	陈锋军，朱学岩，周文静，等. 基于无人机航拍与改进YOLOv3模型的云杉计数[J]. 农业工程学报，2020，36(22)：22-30. Chen Fengjun, Zhu Xueyan, Zhou Wenjing, et al. Spruce counting method based on improved YOLOv3 model in UAV images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(22): 22-30. (in Chinese with English abstract)
[39]	Ibryaeva O, Shepelev V, Kuzmicheva O, et al. A study of the impact of the transport queue structure on the traffic capacity of a signalized intersection using neural networks[J]. Transportation Research Procedia, 2020, 52: 589-596.
[40]	Alver Y, Onelcin P, Cicekli A, et al. Evaluation of pedstrian critical gap and crossing speed at midblock crossing using image proccessing[J]. Accident Analysis and Prevention, 2021, 156(4): 106127.
[41]	Khazukov K, Shepelev V, Karpeta T, et al. Real-time monitoring of traffic parameters[J]. Journal of Big Data, 2020, 7: 84.
[42]	Bewley A, Ge Z, Ott L, et al. Simple online and realtime tracking[C]// 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA: IEEE, 2016: 3464-3468.

[1]	Li Jiaxing, Weng Zhonghua, Chen Xin. Influence of near-bed boundary condition on mixture model for hyper-concentrated sediment-laden flow in open-channel[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(12): 92-99. DOI: 10.11975/j.issn.1002-6819.2022.12.011
[2]	Si Qiaorui, Guo Yongsheng, Tian Ding, Ma Wensheng, Zhang Haoyang, Yuan Shouqi. Study on the inner flow characteristics of an inside unshroud impeller centrifugal pump under gas-liquid two phase condition[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(24): 30-37. DOI: 10.11975/j.issn.1002-6819.2021.24.004
[3]	Sun Zhilin, Hu Qiuyue, Tu Wenrong, Fang Shibiao, Yang Yang. Heat transfer characteristics in circulating solar heat tubes based on gas-liquid two-phase flow[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(24): 246-254. DOI: 10.11975/j.issn.1002-6819.2019.24.029
[4]	Luo Xingqi, Yan Sina, Feng Jianjun, Zhu Guojun, Sun Shuaihui, Chen Senlin. Research on force characteristics of gas-liquid two-phase centrifugal pump[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(23): 66-72. DOI: 10.11975/j.issn.1002-6819.2019.23.008
[5]	Sun Chunhua, Ning Zhi, Qiao Xinqi, Li Yuanxu, Lü Ming. Gas-liquid two-phase flow pattern affecting spray shape and droplet size distribution[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(12): 29-37. DOI: 10.11975/j.issn.1002-6819.2019.12.004
[6]	Zhang Xiaoying, Li Lin, Jin Sheng, Tan Yihai, Wu Yangfeng. Experiment studies on impact of siphon gradient on gas-liquid two-phase flow characteristic[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(14): 122-129. DOI: 10.11975/j.issn.1002-6819.2017.14.017
[7]	Sha Yi, Liu Xiangsong. Numerical calculation on gas-liquid two-phase hydrotransport and flow field measurement in volute with probes for vortex pump[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(18): 93-100. DOI: 10.3969/j.issn.1002-6819.2014.18.012
[8]	Li Hong, Xu Dehuai, Tu Qin, Cheng Jun, Li Lei. Numerical simulation on gas-liquid two-phase flow of self-priming pump during starting period[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2013, 29(3): 77-83.
[9]	Fan Lidan, Ma Wenxing, Chai Bosen, Lu Xiuquan. Numerical simulation and particle image velocimetry for gas-liquid two-phase flow in hydraulic couplings[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2011, 27(11): 66-70.
[10]	Yi Weiming, Wang Nana, Zhang Botao, Li Zhihe. Experimental study on gas-solid two-phase flow in a horizontal entrained bed[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2006, 22(1): 11-14.