Online detection of tomato defects based on YOLOv4 model pruning

Liang Xiaoting; Pang Qi; Yang Yi; Wen Chaowu; Li Youli; Huang Wenqian; Zhang Chi; Zhao Chunjiang

doi:10.11975/j.issn.1002-6819.2022.06.032

Volume 38 Issue 6

Mar. 2022

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2022 > 38(6): 283-292. > DOI: 10.11975/j.issn.1002-6819.2022.06.032

Liang Xiaoting, Pang Qi, Yang Yi, Wen Chaowu, Li Youli, Huang Wenqian, Zhang Chi, Zhao Chunjiang. Online detection of tomato defects based on YOLOv4 model pruning[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(6): 283-292. DOI: 10.11975/j.issn.1002-6819.2022.06.032

Citation:

PDF (4631 KB)

Online detection of tomato defects based on YOLOv4 model pruning

Liang Xiaoting^1,2,3,
Pang Qi^1,2,3,
Yang Yi^2,3,
Wen Chaowu⁴,
Li Youli²,
Huang Wenqian^2,3,
Zhang Chi^2,3,
Zhao Chunjiang^1,2,3

1.
College of Information Science, Shanghai Ocean University, Shanghai 201306, China
2.
Intelligent Equipment Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing 100097, China
3.
Information Technology Research Center, Beijing Academy of Agriculture and Forestry Sciences, Beijing 100097, China
4.
Hunan Agricultural Equipment Research Institute, Changsha, 410125, China

More Information

Received Date: November 29, 2021
Revised Date: February 20, 2022
Published Date: March 30, 2022

Graphical Abstract

Abstract

Abstract

Abstract: Surface defects have posed a negative impact on the quality and yield in the process of tomato growth. A post-harvest grading treatment can normally be utilized before tomato marketing. It is necessary to accurately and rapidly detect the defective tomato in the process of post-harvest grading. In this study, a real-time detection was proposed for the tomato surface defects using YOLOv4 model pruning. A diffuse light box was used to improve the acquisition system. High resolution images of tomatoes were then acquired to reduce the reflection of tomato surface under the direct exposure. Parallel computing and images combination were also selected for the high speed of image processing. The input images of the YOLOv4 network were generated to stitch the RGB images collected from three continuous detection stations. In addition, the channel pruning was selected to simplify the network parameters and structure in the YOLOv4 network model. There were a complex network structure and a large number of parameters in the original YOLOv4, leading to too large calculation and low inference speed of the model. The layer pruning was also used to further compress the depth of the network model on the basis of compressing the network width, in order to improve the detection speed for the real-time detection. A non-maximum suppression with the L1 norm was proposed to remove the redundant prediction box after fine-tuning network model, thereby accurately locating the defect location in the images. The detection performance of the improved model was evaluated using the YOLOv4 training data at the pruning rates under various target detection models. The maximum Mean Average Precision (mAP) value was taken as the pruning rate, indicating the minimum model size and inference time for the requirements of real-time performance. Therefore, the channel pruning rate of the YOLOv4 network was finally set to 80%, where the obtained model was named YOLOv4P. The YOLOv3, YOLOv4, YOLOv4-tiny, YOLOv4PC, and YOLOv4P models were compared to verify the performance of the detection model for the tomato defects. The results showed that the model pruning compression technology can effectively improve the detection speed at the lowest cost of accuracy. The real-time grading system was tested on the tomato experiment data set, including the stem, calyx, and defect types. The improved YOLOv4P network reduced the model size and reasoning time by 232.40 MB and 10.11 ms, respectively, compared with the original YOLOv4 network. In conclusion, the highest mAP and the fastest detection speed were achieved to detect the tomato surface defects using the improved model. The mAP increased from 92.45% to 94.56%, fully meeting the requirements of accurate and real-time detection. The finding can also provide an efficient online detection for the tomato real-time grading system.
- machine vision,
- models,
- tomato defect,
- YOLOv4,
- model pruning

FullText(HTML)

References (28)

References

[1]	Li J B, Luo W, Wang Z L, et al. Early detection of decay on apples using hyperspectral reflectance imaging combining both principal component analysis and improved watershed segmentation method[J]. Postharvest Biology Technology, 2019, 149(3): 235-246.
[2]	王树文，张长利，房俊龙. 基于计算机视觉的番茄损伤自动检测与分类研究[J]. 农业工程学报，2005，21(8)：98-101.Wang Shuwen, Zhang Changli, Fang Junlong. Research on automatic detection and classification of tomato damage based on computer vision[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2005, 21(8): 98-101. (in Chinese with English abstract)
[3]	Liu L, Li Z K, Lan Y F, et al. Design of a tomato classifier based on machine vision[J]. PLoS One, 2019, 14(7): e0219803.
[4]	袁亮，涂雪滢，巨刚，等. 基于机器视觉的番茄实时分级系统设计[J]. 新疆大学学报：自然科学版，2017，34(1)：11-16.Yuan Liang, Tu Xueying, Ju Gang, et al. Tomato real-time classification based on machine vision detection system design[J]. Journal of Xinjiang Universit: Natural Science Edition, 2017, 34(1): 11-16. (in Chinese with English abstract)
[5]	燕红文. 基于机器视觉的番茄损伤区域自动检测[J]. 无线互联科技，2020，17(11)：126-127.Yan Hongwen. Automatic detection of tomato damage area based on machine vision[J]. Wireless Internet Technology, 2020, 17(11): 126-127. (in Chinese with English abstract)
[6]	Muturi I D, Eisa B, Cedric O, et al. A computer vision system for defect discrimination and grading in tomatoes using machine learning and image processing[J]. Artificial Intelligence in Agriculture, 2019(2): 28-37.
[7]	Muturi I D. 基于机器学习和图像处理的番茄缺陷判别和分级系统[D]. 南京：南京农业大学，2019.Muturi I D. A Computer Vision System for Defect Discrimination and Grading in Tomatoes using Machine Learning and Image Processing[D]. Nanjing: Nanjing Agricultural University, 2019. (in Chinese with English abstract)
[8]	张廷婷. 基于机器视觉的番茄分级包装系统的研究[D]. 成都：成都大学，2021.Zhang Tingting. Research on Tomato Grad Packaging System Based on Machine Vision[D]. Chengdu: Chengdu University, 2021. (in Chinese with English abstract)
[9]	缑新科，常英. 基于机器视觉的樱桃番茄在线分级系统设计[J]. 计算机与数字工程，2020，48(7)：1798-1803.Gou Xinke, Chang Ying. Design of cherry tomato online grading system based on machine vision[J]. Computer & Digital Engineering, 2020, 48(7): 1798-1803. (in Chinese with English abstract)
[10]	LeCun Y, Bengio Y, Hinton G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[11]	Fan S X, Li J B, Zhang Y H, et al. On line detection of defective apples using computer vision system combined with deep learning methods[J]. Journal of Food Engineering, 2020, 286: 110102.
[12]	龙洁花，赵春江，林森，等. 改进Mask R-CNN的温室环境下不同成熟度番茄果实分割方法[J]. 农业工程学报，2021，37(18)：100-108.Long Jiehua, Zhao Chunjiang, Lin Sen, et al. Segmentation method of the tomato fruits with different maturities under greenhouse environment based on improved Mask R-CNN[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(18): 100-108. (in Chinese with English abstract)
[13]	Costa A Z, Figueroa H E H, Fracarolli J A. Computer vision based detection of external defects on tomatoes using deep learning[J]. Biosystems Engineering, 2020, 190: 131-144.
[14]	Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016: 779-788.
[15]	Ren S Q, He K M, Girshick R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis Machine Intelligence, 2017, 39(6): 1137-1149.
[16]	赵春江，文朝武，林森，等. 基于级联卷积神经网络的番茄花期识别检测方法[J]. 农业工程学报，2020，36(24)：143-152.Zhao Chunjiang, Wen Chaowu, Lin Sen, et al. Tomato florescence recognition and detection method based on cascaded neural network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(24): 143-152. (in Chinese with English abstract)
[17]	侯加林，房立发，吴彦强，等. 基于深度学习的生姜种芽快速识别及其朝向判定[J]. 农业工程学报，2021，37(1)：213-222.Hou Jialin, Fang Lifa, Wu Yanqiang, et al. Rapid recognition and orientation determination of ginger shoots with deep learning[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(1): 213-222. (in Chinese with English abstract)
[18]	黄丽明，王懿祥，徐琪，等. 采用YOLO算法和无人机影像的松材线虫病异常变色木识别[J]. 农业工程学报，2021，37(14)：197-203.Huang Liming, Wang Yixiang, Xu Qi, et al. Recognition of abnormally discolored trees caused by pine wilt disease using YOLO algorithm and UAV images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(14): 197-203. (in Chinese with English abstract)
[19]	燕红文，刘振宇，崔清亮，等. 基于特征金字塔注意力与深度卷积网络的多目标生猪检测[J]. 农业工程学报，2020，36(11)：193-202.Yan Hongwen, Liu Zhenyu, Cui Qingliang, et al. Multi-target detection based on feature pyramid attention and deep convolution network for pigs[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(11): 193-202. (in Chinese with English abstract)
[20]	Hu X L, Liu Y, Zhao Z X, et al. Real-time detection of uneaten feed pellets in underwater images for aquaculture using an improved YOLO-V4 network[J]. Computers and Electronics in Agriculture, 2021, 185: 106-135.
[21]	Li S W, Gu X Y, Xu X R, et al. Detection of concealed cracks from ground penetrating radar images based on deep learning algorithm[J]. Construction and Building Materials, 2021, 273: 121949.
[22]	Bochkovskiy A, Wang C Y, Liao H Y M. YOLOv4: Optimal speed and accuracy of object detection[J]. arXiv preprint arXiv: 2004. 10934, 2020.
[23]	Redmon J, Farhadi A. YOLOv3: An incremental improvement[J]. arXiv preprint arXiv: 2018, 1804: 02767.
[24]	Zhang M Y, Jiang Y, Li C Y, et al. Fully convolutional networks for blueberry bruising and calyx segmentation using hyperspectral transmittance imaging[J]. Biosystems Engineering, 2020, 192: 159-175.
[25]	Zheng Z H, Wang P, Liu W, et al. Distance-IoU loss: Faster and better learning for bounding box regression[J]. arXiv preprint arXiv, 2020, 34(7): 12993-13000.
[26]	Liu Z, Li J G, Shen Z Q, et al. Learning efficient convolutional networks through network slimming[C]//Proceedings of the IEEE international conference on computer vision, 2017: 2736-2744.
[27]	Wu D H, Lv S C, Jiang M, et al. Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments[J]. Computers and Electronics in Agriculture, 2020, 178: 105742.
[28]	周志华. 机器学习[M]. 北京：清华大学出版社，2016.

[1]	Li Jiaxing, Weng Zhonghua, Chen Xin. Influence of near-bed boundary condition on mixture model for hyper-concentrated sediment-laden flow in open-channel[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(12): 92-99. DOI: 10.11975/j.issn.1002-6819.2022.12.011
[2]	Si Qiaorui, Guo Yongsheng, Tian Ding, Ma Wensheng, Zhang Haoyang, Yuan Shouqi. Study on the inner flow characteristics of an inside unshroud impeller centrifugal pump under gas-liquid two phase condition[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(24): 30-37. DOI: 10.11975/j.issn.1002-6819.2021.24.004
[3]	Sun Zhilin, Hu Qiuyue, Tu Wenrong, Fang Shibiao, Yang Yang. Heat transfer characteristics in circulating solar heat tubes based on gas-liquid two-phase flow[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(24): 246-254. DOI: 10.11975/j.issn.1002-6819.2019.24.029
[4]	Luo Xingqi, Yan Sina, Feng Jianjun, Zhu Guojun, Sun Shuaihui, Chen Senlin. Research on force characteristics of gas-liquid two-phase centrifugal pump[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(23): 66-72. DOI: 10.11975/j.issn.1002-6819.2019.23.008
[5]	Sun Chunhua, Ning Zhi, Qiao Xinqi, Li Yuanxu, Lü Ming. Gas-liquid two-phase flow pattern affecting spray shape and droplet size distribution[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2019, 35(12): 29-37. DOI: 10.11975/j.issn.1002-6819.2019.12.004
[6]	Zhang Xiaoying, Li Lin, Jin Sheng, Tan Yihai, Wu Yangfeng. Experiment studies on impact of siphon gradient on gas-liquid two-phase flow characteristic[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(14): 122-129. DOI: 10.11975/j.issn.1002-6819.2017.14.017
[7]	Sha Yi, Liu Xiangsong. Numerical calculation on gas-liquid two-phase hydrotransport and flow field measurement in volute with probes for vortex pump[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(18): 93-100. DOI: 10.3969/j.issn.1002-6819.2014.18.012
[8]	Li Hong, Xu Dehuai, Tu Qin, Cheng Jun, Li Lei. Numerical simulation on gas-liquid two-phase flow of self-priming pump during starting period[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2013, 29(3): 77-83.
[9]	Fan Lidan, Ma Wenxing, Chai Bosen, Lu Xiuquan. Numerical simulation and particle image velocimetry for gas-liquid two-phase flow in hydraulic couplings[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2011, 27(11): 66-70.
[10]	Yi Weiming, Wang Nana, Zhang Botao, Li Zhihe. Experimental study on gas-solid two-phase flow in a horizontal entrained bed[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2006, 22(1): 11-14.