YOLOv8-ABW based method for detecting Hemerocallis citrina Baroni maturity
-
摘要:
为实现黄花成熟度的快速、高精度识别,针对其相似特征识别精确度低以及相互遮挡检测困难的问题,提出一种基于YOLOv8-ABW的黄花成熟度检测方法。该研究在特征提取网络中加入结合注意力机制的尺度特征交互机制(attention based intra-scale feature interaction, AIFI),更好地提取黄花特征信息,提高检测的精确度。在特征融合网络中,进一步采用加权的双向特征金字塔特征融合网络(bidirectional feature pyramid network, Bi FPN),实现更高层次的跨通道特征融合,有效减少通道中的特征冗余。此外使用WIoUv3作为损失函数,聚焦普通质量的锚框,提高模型的定位性能。试验结果表明:YOLOv8-ABW模型检测精确度为82.32%,召回率为83.71%,平均精度均值mAP@0.5和mAP@0.5:0.95分别为88.44%和74.84%,调和均值提升至0.86,实时检测速度为214.5帧/s。与YOLOv8相比,YOLOv8-ABW的精确度提高1.41个百分点,召回率提高0.75个百分点,mAP@0.5和mAP@0.5:0.95分别提升1.54个百分点和1.42个百分点。对比RT-DETR、YOLOv3、YOLOv5、YOLOv7模型,YOLOv8-ABW参数量最少,仅为3.65×106,且模型浮点运算量比YOLOv7少96.3 G。体现出YOLOv8-ABW 模型能够在黄花成熟度检测中平衡检测精确度和检测速度,综合性能最佳,为黄花智能化实时采摘研究提供技术支持。
Abstract:Here, rapid and high-precision detection was proposed for Hemerocallis citrina Baroni maturity using YOLO v8-ABW. Current challenges were overcome on the similar features and mutual occlusions during recognition. The precision and efficiency of detection were improved to provide crucial technical support to intelligent real-time harvesting. Feature extraction and utilization were also greatly improved in the detection process of Hemerocallis citrina Baroni maturity. Attention-based Intra-scale Feature Interaction (AIFI) was incorporated into the feature extraction network. Feature information was interacted with and combined from various scales. The detection precision was enhanced to more effectively extract the information about Hemerocallis citrina Baroni. Specifically, the AIFI was used to weigh the features using the attention mechanism. More key areas were focused on extracting the features, in order to reduce the interference of noise and redundancy. Meanwhile, the scale feature interaction was used to fully utilize the feature information of different scales, thereby enhancing the precision and robustness of feature extraction. A weighted bidirectional feature pyramid feature fusion network (BiFPN) was used in the feature fusion network. This network structure was achieved in the complementarity and enhancement of various layers of feature information using cross-channel feature fusion. Compared with the traditional Feature Pyramid Network, BiFPN was used to retain more original feature information and fuse the features of different layers in a weighted manner, thus enriching the fused features. In addition, Bi FPN had effectively reduced the feature redundancy in the channels, thereby enhancing the speed and efficiency of detection. Moreover, WIoUv3 was used in the selection of the loss function. The loss function was specifically optimized for the standard quality anchor frames, in order to focus more on the location of targets during training. The WIoUv3 loss function was introduced to successfully enhance the localization performance, leading to more accurate and reliable detection. The experiment validated that the YOLOv8-ABW model was performed better to detect the maturity of Hemerocallis citrina Baroni. The precision reached 82.32%, the recall was 83.71%, mAP@0.5 and mAP@0.5:0.95 were 88.44% and 74.84%, respectively. The harmonic mean was improved to 0.86, and the real-time detection speed even reached 214.5 frames per second (f/s). Compared with the original one, the YOLOv8-ABW model showed improvements in the precision, recall, mAP@0.5, and mAP@0.5:0.95 by 1.41, 0.75, 1.54, and 1.42 percentage points, respectively. Compared to the rest, YOLOv8-ABW shared the least number of parameters, only 3.65×106. The floating-point operations were 96.3 G less than the YOLOv7. The high precision YOLOv8-ABW also exhibited high computational efficiency for the real-time detection tasks related to Hemerocallis Citrina Baroni maturity. In summary, the YOLOv8-ABW model demonstrated excellent performance for the detection. The simplified complexity was also obtained with the detection precision and speed. The finding can offer robust technical support to the intelligent real-time harvesting of Hemerocallis citrina Baroni.
-
Keywords:
- computer vision /
- models /
- YOLOv8 /
- Hemerocallis citrina baroni /
- deep learning /
- maturity detection
-
-
图 2 真实框与预测框示意图
注:A为真实框,B为预测框。wbb、hbb、bbb、(xbb,ybb)分别表示预测框的宽、高尺寸、中心点和中心坐标;wgt、hgt、bgt、(xgt,ygt)分别表示真实框的宽、高尺寸中心点和中心坐标;Wg、Hg 分别表示最小边框宽、高尺寸。
Figure 2. Schematic representation of ground truth and bounding box
Note: A is the true box and B is the predicted box. wbb, hbb, bbb, (xbb, ybb) represent the width and height dimensions, centroid, and center coordinates, respectively. in the actual frame, wgt, hgt, bgt, (xgt, ygt) represent the width and height dimensions, centroid, and center coordinates. Additionally, Wg and Hg denote the minimum border width and height dimensions.
图 3 YOLOv8-ABW模型结构
注:CBS 为卷积+标准化+激活函数;C2f为由跨层拼接的卷积通道和若干个残差组成模块;AIFI为结合注意力机制的尺度特征交互机制模块;UpSample为上采样模块;Bi-FPN为通道信息融合机制;Conv 为参数化卷积层。Bottleneck 为残差模块, Concat 为特征连接模块, Split 为通道分割模块, BatchNorm 表示标准化处理, SiLU 为激活函数。
Figure 3. YOLOv8-ABW model structure
Note: CBS stands for Convolution + Normalization + Activation function. C2f is a module that includes a convolutional channel spliced across layers and a number of residuals. AIFI is a module that combines a scale-feature interaction mechanism with an attentional mechanism. UpSample is an up-sampling module. Bi-FPN is a channel information fusion mechanism. Conv refers to a parameterized convolutional layer.Bottleneck represents the residual module, Concat signifies the feature concatenation module, Split denotes the channel segmentation module, BatchNorm refers to normalization, and SiLU is the activation function.
表 1 黄花成熟度判别划分
Table 1 Classification of Hemerocallis citrina Baroni maturity determination
判别属性
Discriminant property黄花成熟度
Hemerocallis citrina Baronimaturity 成熟Mature 未成熟Immature 比例Percentage 颜色Colour 黄色 黄绿色 0.5 硬度Hardness 变软 较硬 0.1 形状Shape 扁平状 圆形 0.2 裂纹Crack 开裂 无裂纹 0.2 表 2 消融试验结果
Table 2 Ablation experiment results
模型
ModelAIFI Bi FPN WIoUv3 精确度
Precision/%召回率
Recall/%平均精确均值
Mean average precision
mAP@0.5/%平均精度均值
Mean average precision
mAP@0.5:0.95/%F1值
F1 value/%帧率
Frames per second/(帧s-1)YOLOv8 80.91 82.96 86.90 73.42 85 320.80 模型 1 √ 81.75 83.93 88.17 74.69 86 229.43 模型 2 √ 82.11 82.89 87.13 73.56 86 293.23 模型 3 √ 80.98 83.75 86.57 72.82 86 357.58 模型 4 √ √ 81.63 83.47 87.67 74.31 85 208.30 模型 5 √ √ 81.17 82.89 86.71 73.41 84 202.51 模型 6 √ √ 81.52 82.93 86.76 73.12 85 280.74 YOLOv8-ABW √ √ √ 82.32 83.71 88.44 74.84 86 214.50 表 3 不同模型对比试验结果
Table 3 Comparative experiments results of different models
模型
Model网络层数
Network layers参数量
Parameters/
106浮点运算量Flops
/G训练时长
Train times/h模型体积
model size/MB精确度
Precision/%召回率
Recall/%平均精度均值
Mean average precision
mAP@0.5/%平均精度均值
Mean average precision
mAP@0.5:0.95/%F1值
F1 value/%帧率
Frames per second
/(帧·s-1)YOLOv3 310 103.69 283.0 1.83 207.8 82.49 83.55 87.11 75.01 85 58.4 YOLOv5 468 46.14 107.9 3.49 92.7 81.41 82.12 86.72 72.23 84 64.1 YOLOv6 195 4.24 11.9 1.75 8.7 80.46 81.73 86.31 73.05 86 337.2 YOLOv7 415 37.20 104.8 12.04 74.8 79.82 80.07 84.44 72.87 84 119.1 YOLOv7 tiny 263 6.02 13.2 10.66 12.3 81.28 82.16 86.27 74.12 85 88.2 LSEB YOLO v7[28] 506 35.21 9.7 7.69 70.8 80.15 81.23 86.78 74.55 84 99.7 RT-DETR 421 20.18 58.6 1.91 40.5 80.29 77.19 77.83 66.74 84 107.5 YOLOv8 225 3.01 8.2 1.61 6.2 80.91 82.96 86.90 73.42 85 320.8 YOLOv8-ABW 242 3.65 8.5 1.97 7.6 82.32 83.71 88.44 74.84 86 214.5 表 4 添加不同注意力机制的模型对比试验结果
Table 4 Comparative experimental results of models with different attention mechanisms added
模型
Model成熟精
确度
Mature precision/%不成熟
精确度
Immature precision/%精确度
Precision/%召回率
Recall/%mAP@
0.5/%mAP@
0.5:0.95/%+SE 81.33 81.09 81.21 81.34 84.93 70.77 +CBAM 82.87 81.39 82.13 83.22 85.57 74.67 +ECA 81.46 80.52 80.99 82.95 83.89 73.64 +CA 81.73 80.81 81.27 81.67 85.91 73.12 +LSK 82.74 81.08 81.91 81.68 85.68 74.27 +BiFormer 81.77 81.79 81.78 82.96 85.49 73.52 YOLOv8-ABW 83.39 81.25 82.32 83.71 88.44 74.84 -
[1] 高洪香,焦炳忠,贾帅. 适宜的渗灌灌溉方式提高灌区水分利用效率及黄花菜产量[J]. 节水灌溉,2023,(9):32-37. GAO Hongxiang, JIAO Bingzhong, JIA Shuai. Suitable infiltration irrigation methods to improve water use efficiency and yield of daylily in irrigation area[J]. Water Saving Irrigation, 2023(9): 32-37. (in Chinese with English abstract)
[2] SHENG B, WU L G, ZHANG N. A maturity detection method for hemerocallis citrina baroni based on lightweight and attention mechanism[J]. Applied Sciences. 2023, 13(21): 12043.
[3] 吴利刚,吕媛媛,周倩,等. 基于改进YOLOv5的轻量级黄花成熟检测方法[J/OL]. [2024-01-16] 中国农机化学报,2023:1-11. http://kns.cnki.net/kcms/detail/32.1837.S.20231130.1316.008.html. WU Ligang, LV Yuanyuan, ZHOU Qian, et al. Lightweight method for maturity detection of Hemerocallis citrina Baroni based on improved YOLOv5 [J/OL]. [2024-01-16] Journal of Chinese Agricultural Mechanization, 2023: 1-11. http://kns.cnki.net/kcms/detail/ 32.1837.S.20231130.1316.008.html. (in Chinese with English abstract)
[4] 缪叶旻子,周斌雄,刘庭付,等. 黄花菜产量及品质成分的通径分析和预测模型研究[J]. 山东农业大学学报(自然科学版),2023,54(2):173-179. doi: 10.3969/j.issn.1000-2324.2023.02.003 MIAO Yeminzi, ZHOU Binxiong, LIU Tingfu, et al. Path analysis and prediction model of yield and quality components in daylily[J]. Ournal of Shandong Agricultural University (Natural Science Edition), 2023, 54(2): 173-179. (in Chinese with English abstract) doi: 10.3969/j.issn.1000-2324.2023.02.003
[5] GUO X , XUN Z , XU L , et al. Constructed the metabolic compound accumulation profiles of daylily at different main producing areas using multi-omics techniques [J]. Food Bioscience, 2024, 59: 104128.
[6] PENG J , LI S , MA X , et al. Spatial distribution and influencing factors of daylily cultivation in the farming–pastoral ecotone of northern china[J]. Land, 2024, 13(4): 439.
[7] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149. doi: 10.1109/TPAMI.2016.2577031
[8] LI M, ZHANG Z, LEI L, et al. Agricultural greenhouses detection in high-resolution satellite images based on convolutional neural networks: Comparison of faster R-CNN, YOLOv3 and SSD[J]. Sensors, 2020, 20: 4938.
[9] LV W, XU S, ZHAO Y, et al. Detrs beat yolos on real-time object detection[EB/OL]. [2024-04-11]. https://arxiv.org/abs/2304.08069.
[10] REDMON J, DIVVALA S. , GIRSHICK R. You only look once: Unified, real-time object detection, [C]// Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 79–88.
[11] MAO Q, LI S, HU X, et al. Foreign object recognition of belt conveyor in coal mine based on improved YOLOv7[J]. Journal of Mine Automation, 2022(48): 26-32.
[12] 贾雪莹,赵春江,周娟,等. 基于改进YOLOv7模型的柑橘表面缺陷在线检测[J]. 农业工程学报,2023,39(23):142-151. doi: 10.11975/j.issn.1002-6819.202308138 JIA Xueying, ZHAO Chunjiang, ZHOU Juan, et al. Online detection of citrus surface defects using improved YOLOv7 modeling[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(23): 142-151. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202308138
[13] 李永玉,吴继峰,王威,等. 便携式双档位柑橘多品质无损检测装置设计与试验[J]. 农业工程学报,2023,39(23):259-267. doi: 10.11975/j.issn.1002-6819.202309146 LI Yongyu, WU Jifeng, WANG Wei, et al. A portable dual-gear device for non-destructive testing on multi quality of citrus[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(23): 259-267. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202309146
[14] 周宏平,金寿祥,周磊,等. 基于迁移学习与YOLOv8n的田间油茶果分类识别[J]. 农业工程学报,2023,39(20):159-166. doi: 10.11975/j.issn.1002-6819.202307244 ZHOU Hongping, JIN Shouxiang, ZHOU Lei, et al. Classification and recognition of camellia oleifera fruit in the field based on transfer learning and YOLOv8n[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(20): 159-166. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202307244
[15] 周宏平,金寿祥,周磊,等. 基于多模态图像的自然环境下油茶果识别[J]. 农业工程学报,2023,39(10):175-182. doi: 10.11975/j.issn.1002-6819.202303054 ZHOU Hongping, JIN Shouxiang, ZHOU Lei, et al. Recognition of camellia oleifera fruits in natural environment using multi-modal images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(10): 175-182. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202303054
[16] 张钟莉莉,何婷婷,李志伟,等. 基于区域亮度矫正的番茄成熟度定量分级方法[J]. 农业工程学报,2023,39(7):195-204. doi: 10.11975/j.issn.1002-6819.202211192 ZHANG Zhonglili, HE Tingting, LI Zhiwei, et al. Quantitative grading method for tomato maturity using regional brightness correction[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(7): 195-204. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202211192
[17] 李韬,任玲,胡斌,等. 改进YOLOv5s和迁移学习对番茄穴盘苗的分级检测[J]. 农业工程学报,2023,39(23):174-184. doi: 10.11975/j.issn.1002-6819.202308078 LI Tao, REN Ling, HU Bin, et al. Grading detection of tomato hole-pan seedlings using improved YOLOv5s and transfer learning[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(23): 174-184. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202308078
[18] YU J Z, BAI Y F, YANG S Q et al. Stolon-YOLO: A detecting method for stolon of strawberry seedling in glass greenhouse[J]. Computers and Electronics in Agriculture, 2023, 215: 108447. doi: 10.1016/j.compag.2023.108447
[19] 刘莫尘,褚镇源,崔明诗,等. 基于改进YOLO v8-Pose的红熟期草莓识别和果柄检测[J]. 农业机械学报,2023,54(s2):244-251. LIU Mochen, CHU Zhenyuan, CUI Mingshi, et al. Red ripe strawberry recognition and stem detection based on improved YOLO v8-Pose[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(s2): 244-251. (in Chinese with English abstract)
[20] 余琼,张瑞,李德豪,等. 基于残差块与注意力机制的果蔬自动识别方法[J]. 农业机械学报,2023,54(s2):214-222. doi: 10.6041/j.issn.1000-1298.2023.S2.025 YU Qiong, ZHANG Rui, LI Dehao, et al. Method for fruit and vegetable automatic recognition based on residual block and attention mechanism[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(s2): 214-222. (in Chinese with English abstract) doi: 10.6041/j.issn.1000-1298.2023.S2.025
[21] 陈锋军,陈闯,朱学岩,等. 基于改进YOLOv7的油茶果实成熟度检测[J]. 农业工程学报,2024,40(5):177-186. CHEN Fengjun, CHEN Chuang, ZHU Xueyan, et al. Detection of camellia oleifera fruit maturity based on improved YOLOv7[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2024, 40(5): 177-186. (in Chinese with English abstract)
[22] 范万鹏,刘孟楠,马婕,等. 利用改进的YOLOv5s检测莲蓬成熟期[J]. 农业工程学报,2023,39(18):183-191. doi: 10.11975/j.issn.1002-6819.202305136 FAN Wanpeng, LIU Mengnan, MA Jie, et al. Detecting lotus seedpod maturation using improved YOLOv5s[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(18): 183-191. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202305136
[23] WU L, ZHANG L, CHEN L, et al. A lightweight and multisource information fusion method for real-time monitoring of lump coal on mining conveyor belts[J]. International Journal of Intelligent Systems, 2023, 11: 5327122.
[24] ZHU L Q, LI X M, SUN H M, et al. Research on CBF-YOLO detection model for common soybean pests in complex environment[J]. Computers and Electronics in Agriculture, 2023, 216: 108515.
[25] FENG J, ZHAO X, ZHU T, et al. Detection mature bud for daylily based on Faster R-CNN integrated with CBAM[J]. IEEE Access, 2023(11): 81646-81655.
[26] ZHANG L, WU L G , LIU Y. Hemerocallis citrina baroni maturity detection method integrating lightweight neural network and dual attention mechanism[J]. Electronics, 2022, 11(17): 2743.
[27] 靳红杰,马顾彧,唐梦圆,等. 复杂环境下黄花菜识别的YOLOv7-MOCA模型[J]. 农业工程学报,2023,39(15):181-188. doi: 10.11975/j.issn.1002-6819.202305100 JIN Hongjie, MA Guyu, TANG Mengyuan, et al. Identifying daylily in complex environment using YOLOv7-MOCA model[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(15): 181-188. (in Chinese with English abstract) doi: 10.11975/j.issn.1002-6819.202305100
[28] 吴利刚,陈乐,周倩,等. 基于轻量化高效层聚合网络的黄花成熟度检测方法[J]. 农业机械学报,2024,55(2):268-277. doi: 10.6041/j.issn.1000-1298.2024.02.026 WU Ligang, CHEN Le, ZHOU Qian, et al. A maturity detection method for Hemerocallis Citrina Baroni based on lightweight and efficient layer aggregation network[J]. Transactions of the Chinese Society for Agricultural Machinery, 2024, 55(2): 268-277. (in Chinese with English abstract) doi: 10.6041/j.issn.1000-1298.2024.02.026
[29] TAN M, PANG R, LE Q V. Efficientdet: Scalable and efficient object detection[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Seattle, Washington, USA, 2020:10781-10790.
[30] TONG Z, CHEN Y, XU Z, et al. Wise-IoU: Bounding Box Regression Loss with Dynamic Focusing Mechanism [EB/OL]. [2024-01-10] https://arxiv.org/abs/2301.10051.
[31] HU J, SHEN L, SUN G, et al. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City:IEEE, 2018: 7132-7141.
[32] WOO S, PARK J, LEE J, et al. CBAM: Convolutional block attention module[C]//European Conference on Computer Vision (ECCV), Munich, Germany, Springer, 2018: 3–19.
[33] HOU Q, ZHOU D, FENG, Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA, 2021:13713-13722.
[34] WANG Q, WU B, ZHU P, et al. Eca-Net: Efficient channel attention for deep convolutional neural networks[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, Washington, USA, 2020: 11531–11539.
[35] LI Y, HOU Q, ZHENG Z, et al. Large selective kernel network for remote sensing object detection[EB/OL]. [2024-01-10] https://arxiv.org/abs/ 2303. 09030.
[36] LI Y, FAN Y, XIANG X, et al. Efficient and explicit modelling of image hierarchies for image restoration [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Vancouver, Canada, 2023:18278-18289.
[37] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: Visual explanations from deep networks via gradient-based localization [J] International Journal of Computer Vision, 2020,128: 336-359.