贾伟宽, 李倩雯, 张中华, 刘国良, 侯素娟, Ji Ze, 郑元杰. 复杂环境下柿子和苹果绿色果实的优化SOLO分割算法[J]. 农业工程学报, 2021, 37(18): 121-127. DOI: 10.11975/j.issn.1002-6819.2021.18.014
    引用本文: 贾伟宽, 李倩雯, 张中华, 刘国良, 侯素娟, Ji Ze, 郑元杰. 复杂环境下柿子和苹果绿色果实的优化SOLO分割算法[J]. 农业工程学报, 2021, 37(18): 121-127. DOI: 10.11975/j.issn.1002-6819.2021.18.014
    Jia Weikuan, Li Qianwen, Zhang Zhonghua, Liu Guoliang, Hou Sujuan, Ji Ze, Zheng Yuanjie. Optimized SOLO segmentation algorithm for the green fruits of persimmons and apples in complex environments[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(18): 121-127. DOI: 10.11975/j.issn.1002-6819.2021.18.014
    Citation: Jia Weikuan, Li Qianwen, Zhang Zhonghua, Liu Guoliang, Hou Sujuan, Ji Ze, Zheng Yuanjie. Optimized SOLO segmentation algorithm for the green fruits of persimmons and apples in complex environments[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(18): 121-127. DOI: 10.11975/j.issn.1002-6819.2021.18.014

    复杂环境下柿子和苹果绿色果实的优化SOLO分割算法

    Optimized SOLO segmentation algorithm for the green fruits of persimmons and apples in complex environments

    • 摘要: 为了实现果园复杂环境下柿子和苹果绿色果实的精准分割,该研究提出了一种基于SOLO的绿色果实优化分割算法。首先,利用分离注意力网络(ResNeSt)设计SOLO算法的主干网络,用于提取绿色果实特征;其次,为更好地应对绿色果实特征的多尺度问题,引入特征金字塔网络(Feature Pyramid Networks,FPN),构造ResNeSt+FPN组合结构;最后,将SOLO算法分为类别预测和掩码生成2个分支,类别预测分支在预测语义类别的同时,掩码生成分支实现了对绿色果实的实例分割。试验结果表明,优化SOLO分割算法的平均召回率和精确率分别达到94.84%和96.16%,平均每张绿色果实图像在图形处理器(Graphics Processing Unit,GPU)上的分割时间为0.14 s。通过对比试验可知,优化SOLO分割算法的召回率分别比优化掩膜区域卷积神经网络算法(Optimized Mask Region Convolutional Neural Network,Optimized Mask R-CNN)、SOLO算法、掩膜区域卷积神经网络算法(Mask Region Convolutional Neural Network,Mask R-CNN)和全卷积实例感知语义分割算法(Fully Convolutional Instance-aware Semantic Segmentation,FCIS)提高了1.63、1.74、2.23和6.52个百分点,精确率分别提高了1.10、1.47、2.61和6.75个百分点,分割时间缩短了0.06、0.04、0.11和0.13 s。该研究算法可为其他果蔬的果实分割提供理论借鉴,扩展果园测产和机器采摘的应用范围。

       

      Abstract: Abstract: To solve the green fruit recognition problem of persimmons and apples, a green fruit segmentation algorithm based on optimized SOLO (Segmenting Objects by Locations) was proposed in this study to achieve accurate segmentation of green fruits in complex environments. The proposed algorithm was a single-stage instance segmentation method, which avoided the disadvantage that detection before segmentation in two-stage methods relied on detection performance. By introducing the concept of instance category, each pixel in the instance was assigned a category according to the location and size of the instance, therefore, the instance segmentation was transformed into a classification problem. This study takes green persimmons and green apples as the research objects. The image collection locations are Shandong Normal University (Changqing Lake Campus) Houshan and the Longwangshan Apple Production Base in Fushan District, Yantai City, Shandong Province. The acquisition device is a Canon EOS 80D SLR camera with an image resolution of 6 000×4 000 pixels. Collect under natural light during the day (7:00-17:00) and under LED light at night (19:00-22:00). A total of 568 images of green persimmons and 498 images of green apples were collected in the experiment, including nighttime, overlap, backlighting, forward light, blocked, and after rain. The collected images were annotated by LabelMe software and then were made into a dataset in COCO format. Specifically, first, split-attention networks (ResNeSt) were used to extract features of the target fruit as the backbone network of optimized SOLO, which enhanced the transfer, reuse, and fusion of features in the front and back layers. Then ResNeSt and Feature Pyramid Network (FPN) were combined to solve the multi-scale problem of green fruits. Because FPN defined allocation strategies for different scale features and assigned them to the pyramid levels optimally. Finally, the image features extracted by the ResNeSt+FPN structure were utilized for the subsequent prediction. The optimized SOLO segmentation algorithm was divided into two branches: category prediction and mask generation. While the semantic category was predicted by the category prediction branch, the object instance was segmented by the mask generation branch, in this way, the target fruit segmentation was completed. The experimental results showed that the average recall and precision of the optimized SOLO segmentation algorithm reached 94.84% and 96.16%, respectively, with an average segmentation time of 0.14 s per green target fruit image on Graphics Processing Unit (GPU). Besides, compared with four algorithms, which were the optimized Mask R-CNN fruit recognition algorithm, SOLO, Mask Region Convolutional Neural Network (Mask R-CNN), and Fully Convolutional Instance-aware Semantic Segmentation (FCIS), the recall of the optimized SOLO segmentation algorithm in this study was improved by 1.63, 1.74, 2.23, and 6.52 percentage points, the precision was improved by 1.10, 1.47, 2.61, and 6.75 percentage points, respectively, and the segmentation times were reduced by 0.06, 0.04, 0.11, and 0.13 s, respectively. The relevant results show that the green fruit optimization SOLO segmentation algorithm proposed by the study can meet the real-time performance of green fruit segmentation and improve the accuracy of green fruit segmentation. This research algorithm can provide theoretical reference for segmentation of other target fruits and vegetables to extend the application of orchard yield measurement and robot harvesting.

       

    /

    返回文章
    返回