刘长勇, 李思佳, 史慧, 查志华, 邓红涛. 基于TMU-Net网络的苹果果心分割方法[J]. 农业工程学报, 2022, 38(16): 304-312. DOI: 10.11975/j.issn.1002-6819.2022.16.033
    引用本文: 刘长勇, 李思佳, 史慧, 查志华, 邓红涛. 基于TMU-Net网络的苹果果心分割方法[J]. 农业工程学报, 2022, 38(16): 304-312. DOI: 10.11975/j.issn.1002-6819.2022.16.033
    Liu Changyong, Li Sijia, Shi Hui, Zha Zhihua, Deng Hongtao. Apple core segmentation method based on TMU-Net network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(16): 304-312. DOI: 10.11975/j.issn.1002-6819.2022.16.033
    Citation: Liu Changyong, Li Sijia, Shi Hui, Zha Zhihua, Deng Hongtao. Apple core segmentation method based on TMU-Net network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2022, 38(16): 304-312. DOI: 10.11975/j.issn.1002-6819.2022.16.033

    基于TMU-Net网络的苹果果心分割方法

    Apple core segmentation method based on TMU-Net network

    • 摘要: 针对苹果内在品质检测过程中传统测量果心大小方法效率低、准确性差等问题,该研究提出一种基于TMU-Net网络自动分割果心的方法,将Transformer编码器融入U-Net网络结构中,构建改进U型卷积网络TMU-Net模型。模型由特征提取模块、特征处理模块、解码器、特征拼接模块组成,以VGG-16前13层作为主干特征提取网络,在跳跃连接中叠加多重残差空洞卷积(Multiple Residual Dilated Convolution,MRDC)模块,增大感受野的同时增强了模型对底层特征提取能力。采用数据增强技术对果心数据集扩充后,利用迁移学习方法冻结特定的网络层,对TMU-Net模型进行训练。试验结果表明:引入迁移学习并使用最佳训练方式使模型分割精确率提高了22.48个百分点;TMU-Net网络模型在果心分割任务中实现了96.72%的精确率,与U-Net、PSPNet、DeeplabV3+网络对比,精确率分别提升了14.28、9.98、7.15个百分点。该方法能够精准、有效地实现果心分割,可为实现苹果内在品质智能检测提供参考。

       

      Abstract: Abstract: Apple quality has been ever increasingly required with the improvement of living standards in recent years. The core ratio is one of the most significant factors to determine the quality of apples. But, the manual measurement on the fruit core cannot fully meet the current detection requirements, in terms of cost and accuracy at present. In this study, an automatic segmentation was proposed for the fruit core using a TMU-Net network model. Firstly, three common types of apples were selected in the Xinjiang of China. An acquisition device was then used to capture the 311 cross-sectional images of the fruit core. Secondly, the preprocessing operations were also conducted to enhance the original images, including translation, vertical mirroring, horizontal mirroring, and adding Gaussian noise. Better training was achieved in the expanded dataset, compared with the original. Specifically, the Intersection Over Union (IOU), Precision, Recall, and F1-score of the TMU-Net network increased by 27.28, 36.62, 29.81, and 32.06 percentage points, respectively. It infers that the data enhancement improved the robustness and generalization of the model after training. The Multiple Residual Dilated Convolution (MRDC) module was also constructed with the Cavity convolution in the different void ratios and shortcut connections. Shortcut connections are skipping one layers, they simply perform identity mapping. As such, the information loss was reduced in the jump connection part of the model. There was also less semantic difference between the encoder and the decoder. The MRDC module was finally used to verify the TMU-Net jump connection. The results showed that: 1) The MRDC module was introduced to effectively improve the segmentation performance of the model, in which the IOU, Precision, and F1-score were improved by 1.59, 6.49, and 4.65 percentage points, respectively. 2) The first 13 layers of VGG-16 network were used as the backbone to capture the low-level features. The Transformer encoder was integrated into the network structure to enhance the global extraction of the network, particularly for the locality of convolution operations. The segmentation shows that the TMU-Net network was much more precise to process the sharp corner and edge details of the fruit center, indicating the feasibility of the model in the segmentation task of the fruit center. 3) The TMU-Net model was trained under a variety of transfer learning. Therefore, freezing the training of specific network layers can be expected to effectively improve the indicators of the model. The training curve of the model showed that the training was used to accelerate the convergence speed. Subsequently, the TMU-Net, DeeplabV3+, U-Net, and PSPNet models were trained to verify the test set under the same experimental parameters. The IOU, Precision, Recall, and F1-score of the TMU-Net model increased by 3.96, 7.15, 9.49, and 6.30 percentage points, respectively, compared with the DeeplabV3+ model with better effect. Therefore, this TMU-Net model can be expected to accurately and effectively realize the fruit core segmentation. The finding can also provide a strong reference for the intelligent detection of apple quality.

       

    /

    返回文章
    返回