基于改进Mask R-CNN模型的植物叶片分割方法

袁山; 汤浩; 郭亚

doi:10.11975/j.issn.1002-6819.2022.01.024

基于改进Mask R-CNN模型的植物叶片分割方法

袁山¹,
汤浩²,
郭亚¹

1.
江南大学物联网工程学院，无锡 214122
2.
绿视芯科技<无锡>有限公司，无锡 214124

基金项目: 国家自然科学基金项目（51961125102；31771680）

计量
- 文章访问数: 758
- HTML全文浏览量: 18
- PDF下载量: 3211
出版历程
- 收稿日期: 2021-08-15
- 修回日期: 2021-12-18
- 发布日期: 2022-01-14

Segmentation method for plant leaves using an improved Mask R-CNN model

1.
Department of Internet of Things, Jiangnan University, Wuxi 214122, China
2.
Lushixin Sci. & Tec. Co. Ltd., Wuxi 214124, China

摘要

摘要: 通过图像处理对植物叶片进行分割是研究植物表型性状的基础，但叶片间相互遮挡、叶片边缘特征不明显以及幼叶目标过小会对叶片分割效果造成很大的障碍。针对上述问题，该研究提出了一种基于改进Mask R-CNN模型的植物叶片分割方法，通过引入级联检测模块对模型检测分支进行改进，以提高遮挡叶片检测质量；利用注意力机制和一个2层3×3卷积模块对模型分割分支进行优化，以提高边缘特征表达能力；在模型测试过程中采用多尺度叶片分割策略，利用多个尺度上的最优目标以分割幼叶。测试结果表明，在CVPPP叶片分割挑战数据集中的对称最佳Dice得分Symmetric Best Dice（SBD）为90.3%，相比于利用域随机化数据增强策略并采用Mask R-CNN模型进行叶片实例分割的方法提高了2.3个百分点，叶片分割效果有显著提升。该研究提出的方法可以有效解决植物叶片分割效果不佳的问题，为植物表型研究提供技术支撑。
- 模型 /
- 图像处理 /
- Mask R-CNN /
- 植物叶片 /
- 注意力机制 /
- 多尺度分割
Abstract: Extracting the leaves has been one of the most fundamental researches for the phenotypic traits of plants. However, it is difficult to further improve the accuracy of leaves segmentation, due to the mutual occlusion between leaves in rosette plants. It is still lacking in the leaf edge characteristics, particularly for the small target of young leaves. In this study, an improved instance segmentation benchmark Mask R-CNN model was proposed to realize an accurate segmentation for the plant leaves. The Cascade R-CNN was also introduced to generate the selection of the proposal boxes for the region proposal network. The boxes were first sent into the Head1 convolutional network with a threshold of 0.5, and then the detection was input into the Head 2 convolutional network with a threshold of 0.6, and finally, the detection was fed into the Head 3 convolutional network with a threshold of 0.7. Three cascaded networks were used to gradually increase the threshold, where the output of the previous network was applied to the higher threshold of the next network, indicating the high quality of the detection branch. After the segmentation branch full convolutional layer (FCN), an attention mechanism (SeNet) and a two-layer 3×3 convolution module were added to weigh the features of the segmentation branch, and further to extract the leaf edge segmentation. The segmentation of multi-scale leaves was also adopted for the leaves of different sizes in an image in the test process. The original and enlarged images were input into the model at the same time for the leaves segmentation. The optimal target on multiple scales was achieved using Non-Maximum Suppression (NMS). The ResNeXt101 was selected as the backbone network to extract the characteristics of leaves. An improved model of deep learning was also utilized at the same time. The image enhancement included random mirroring, rotation, enlargement, and size adjustment. There was no longer requiring for the special function library support, indicating the simplified data enhancement. The improved model had effectively reduced the blade occlusion, edge blur, and small target in the blade segmentation. Correspondingly, the Symmetric Best Dice (SBD) was 90.3% in the CVPPP Leaf Segmentation Challenge (LSC), which was 2.3 percentage points higher than that in Mask R-CNN, particularly than 1 percentage points than before. The improved neural network model was suitable for the leaf segmentation. The finding can provide a more accurate leaf segmentation for the plant phenotype using image processing, indicating a high application value in the research field of plant leaves.
- models /
- image processing /
- Mask R-CNN /
- plant leaves /
- attention mechanism /
- multi-scale segmentation

HTML全文

参考文献(40)

[1]	Pieruschka R, Schurr U. Plant phenotyping: past, present, and future[J]. Plant Phenomics, 2019, 2019: 1-6.
[2]	Nikbakhsh N, Baleghi Y, Agahi H. A novel approach for unsupervised image segmentation fusion of plant leaves based on G-mutual information[J]. Machine Vision and Applications, 2021, 32(5): 1-12.
[3]	Fiorani F, Schurr U. Future scenarios for plant phenotyping[J]. Annual Review of Plant Biology, 2013, 64: 267-291.
[4]	Xia C, Wang L, Chung B K, et al. In situ 3D segmentation of individual plant leaves using a RGB-D camera for agricultural automation[J]. Sensors, 2015, 15(8): 20463-20479.
[5]	Grand-Brochier M, Vacavant A, Cerutti G, et al. Tree leaves extraction in natural images: Comparative study of preprocessing tools and segmentation methods[J]. IEEE Transactions on Image Processing, 2015, 24(5): 1549-1560.
[6]	Liu X, Hu C, Li P. Automatic segmentation of overlapped poplar seedling leaves combining Mask R-CNN and DBSCAN[J]. Computers and Electronics in Agriculture, 2020, 178: 105753.
[7]	Brindha G J, Gopi E S. An hierarchical approach for automatic segmentation of leaf images with similar background using kernel smoothing based Gaussian process regression[J]. Ecological Informatics, 2021, 63: 101323.
[8]	Minervini M, Fischbach A, Scharr H, et al. Finely-grained annotated datasets for image-based plant phenotyping[J]. Pattern Recognition Letters, 2015, 81: 80-89.
[9]	Scharr H, Minervini M, Fischbach A, et al. Annotated image datasets of rosette plants[C]//European Conference on Computer Vision, Zürich, Switzerland: Academia, 2014: 6-12.
[10]	Hüther P, Schandry N, Jandrasits K, et al. ARADEEPOPSIS, an automated workflow for top-view plant phenomics using semantic segmentation of leaf states[J]. The Plant Cell, 2020, 32(12): 3674-3688.
[11]	Koornneef M, Meinke D. The development of Arabidopsis as a model plant[J]. The Plant Journal : For Cell and Molecular Biology, 2010, 61(6): 909-921.
[12]	Alonso-Blanco C, Andrade J, Becker C, et al. 1,135 genomes reveal the global pattern of polymorphism in Arabidopsis thaliana[J]. Cell, 2016, 166(2): 481-491.
[13]	Dellen B, Scharr H, Torras C. Growth signatures of rosette plants from time-lapse video[J]. IEEE-ACM Transactions on Computational Biology and Bioinformatics, 2015, 12(6): 1470-1478.
[14]	Pape J M, Klukas C. Utilizing machine learning approaches to improve the prediction of leaf counts and individual leaf segmentation of rosette plant images[J]. Proceedings of the Computer Vision Problems in Plant Phenotyping (CVPPP), 2015, 3: 1-12.
[15]	Viaud G, Loudet O, Cournède P H. Leaf segmentation and tracking in Arabidopsis thaliana combined to an organ-scale plant model for genotypic differentiation[J]. Frontiers in Plant Science, 2017, 7: 2057.
[16]	Yin X, Liu X, Chen J, et al. Multi-leaf alignment from fluorescence plant images[C]//IEEE Winter Conference on Applications of Computer Vision, Colorado, USA: IEEE, 2014: 437-444.
[17]	Xu L, Li Y, Sun Y, et al. Leaf instance segmentation and counting based on deep object detection and segmentation networks[C]//2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), Toyama, Japan: IEEE, 2018: 180-185.
[18]	Ward D, Moghadam P. Scalable learning for bridging the species gap in image-based plant phenotyping[J]. Computer Vision and Image Understanding, 2020, 197: 103009.
[19]	Kuznichov D, Zvirin A, Honen Y, et al. Data augmentation for leaf segmentation and counting tasks in rosette plants[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, California, USA: IEEE, 2019: 1-10.
[20]	He K, Gkioxari G, Dollár P, et al. Mask R-CNN[C]// Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy: IEEE, 2017: 2961-2969.
[21]	Scharr H, Minervini M, French A P, et al. Leaf segmentation in plant phenotyping: A collation study[J]. Machine Vision and Applications, 2016, 27(4): 585-606.
[22]	Giusti A, Cire?an D C, Masci J, et al. Fast image scanning with deep max-pooling convolutional neural networks[C]// 2013 IEEE International Conference on Image Processing, Melbourne, Australia: IEEE, 2013: 4034-4038.
[23]	赵辉，乔艳军，王红君，等. 基于改进YOLOv3的果园复杂环境下苹果果实识别[J]. 农业工程学报，2021，37(16)：127-135.Zhao Hui, Qiao Yanjun, Wang Hongjun, et al. Apple fruit recognition in complex orchard environment based on improved YOLOv3[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(16): 127-135. (in Chinese with English abstract)
[24]	Dvornik N, Mairal J, Schmid C. On the importance of visual context for data augmentation in scene understanding[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(6): 2014-2028.
[25]	Bloice M D, Stocker C, Holzinger A. Augmentor: An image augmentation library for machine learning[J/OL]. Computer Vision and Pattern Recognition, 2017, [2017-08-11]. https://arxiv.org/abs/1708.04680.
[26]	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA: IEEE, 2016: 770-778.
[27]	Xie S, Girshick R, Dollár P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA: IEEE, 2017: 1492-1500.
[28]	Cai Z, Vasconcelos N. Cascade R-CNN: Delving into high quality object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018: 6154-6162.
[29]	燕红文，刘振宇，崔清亮，等. 基于特征金字塔注意力与深度卷积网络的多目标生猪检测[J]. 农业工程学报，2020，36(11)：192-202.Yan Hongwen, Liu Zhenyu, Cui Qingliang, et al. Multi-target detection based on feature pyramid attention and deep convolution network for pigs[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(11): 192-202. (in Chinese with English abstract)
[30]	邓颖，吴华瑞，朱华吉. 基于实例分割的柑橘花朵识别及花量统计[J]. 农业工程学报，2020，36(7)：200-207.Deng Ying, Wu Huarui, Zhu Huaji. Recognition and counting of citrus flowers based on instance segmentation[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(7): 200-207. (in Chinese with English abstract)
[31]	方鹏，郝宏运，李腾飞，等. 基于注意力机制和可变形卷积的鸡只图像实例分割提取[J]. 农业机械学报，2021，52(4)：257-265.Fang Peng, Hao Hongyun, Li Tengfei, et al. Instance segmentation of broiler image based on attention mechanism and deformable convolution[J]. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52(4): 257-265. (in Chinese with English abstract)
[32]	曹锦纲，杨国田，杨锡运. 基于注意力机制的深度学习路面裂缝检测[J]. 计算机辅助设计与图形学学报，2020，32(8)：1324-1333.Cao Jingang, Yang Guotian, Yang Xiyun. Pavement Crack Detection with Deep Learning Based on Attention Mechanism[J]. Journal of Computer-Aided Design & Computer Graphics, 2020, 32(8): 1324-1333. (in Chinese with English abstract)
[33]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA: IEEE, 2018: 7132-7141.
[34]	Li R, Wang R, Zhang J, et al. An effective data augmentation strategy for CNN-based pest localization and recognition in the field[J]. IEEE Access, 2019, 7: 160274-160283.
[35]	Chen K, Wang J, Pang J, et al. MMDetection: Open mmlab detection toolbox and benchmark[J/OL]. Computer Vision and Pattern Recognition, 2019, [2019-06-17]. https://arxiv.org/abs/1906.07155.
[36]	Paszke A, Gross S, Massa F, et al. Pytorch: An imperative style, high-performance deep learning library[J]. Advances in Neural Information Processing Systems, 2019, 32: 8026-8037.
[37]	Hanno, LSC as of CVPPP2017: evaluation of the testing data[EB/OL]. 2018, [2018-02-23]. https://competitions. codalab.org/competitions/18405.
[38]	Gomes D P S, Zheng L. Leaf segmentation and counting with deep learning: on model certainty, test-time augmentation, trade-offs[J/OL]. Computer Vision and Pattern Recognition, 2020, [2020-12-21]. https://arxiv.org/abs/2012.11486.
[39]	Ward D, Moghadam P , Hudson N . Deep leaf segmentation using synthetic data[J/OL]. Computer Vision and Pattern Recognition, 2019, [2019-03-21]. https://arxiv.org/abs/1807.10931.
[40]	王志强，钮丹丹，王佳兴，等. 大规模用水节点的灌溉物联网监控系统设计[J]. 排灌机械工程学报，2021，39(10)：1062-1067.Wang Zhiqiang, Niu Dandan, Wang Jiaxing, et al. Design of IOT irrigation management system with large-scale nodes[J]. Journal of Drainage and Irrigation Machinery Engineering, 2021, 39(10): 1062-1067. (in Chinese with English abstract)

施引文献

资源附件(0)

计量

文章访问数: 758
HTML全文浏览量: 18
PDF下载量: 3211
被引次数: 0

基于改进Mask R-CNN模型的植物叶片分割方法

计量

出版历程

Segmentation method for plant leaves using an improved Mask R-CNN model

计量

出版历程

目录