易诗, 李俊杰, 贾勇. 基于红外热成像的夜间农田实时语义分割[J]. 农业工程学报, 2020, 36(18): 174-180. DOI: 10.11975/j.issn.1002-6819.2020.18.021
    引用本文: 易诗, 李俊杰, 贾勇. 基于红外热成像的夜间农田实时语义分割[J]. 农业工程学报, 2020, 36(18): 174-180. DOI: 10.11975/j.issn.1002-6819.2020.18.021
    Yi Shi, Li Junjie, Jia Yong. Real-time semantic segmentation of farmland at night using infrared thermal imaging[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(18): 174-180. DOI: 10.11975/j.issn.1002-6819.2020.18.021
    Citation: Yi Shi, Li Junjie, Jia Yong. Real-time semantic segmentation of farmland at night using infrared thermal imaging[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(18): 174-180. DOI: 10.11975/j.issn.1002-6819.2020.18.021

    基于红外热成像的夜间农田实时语义分割

    Real-time semantic segmentation of farmland at night using infrared thermal imaging

    • 摘要: 农田环境实时语义分割是构成智能农机的视觉环境感知的重要环节,夜间农田语义分割可以使智能农机在夜间通过视觉感知农田环境进行全天候作业,而夜间无光环境下,可见光摄像头成像效果较差,将造成语义分割精度的下降。为保证夜间农田环境下红外图像语义分割的精度与实时性,该研究提出了一种适用于红外图像的红外实时双边语义分割网络(Infrared Real-time Bilateral Semantic Segmentation Network,IR-BiSeNet),根据红外图像分辨率低,细节模糊的特点该网络在实时双边语义分割网络(Bilateral Semantic Segmentation Net,BiSeNet)结构基础上进行改进,在其空间路径上,进一步融合红外图像低层特征,在该网络构架中的注意力提升模块、特征融合模块上使用全局最大池化层替换全局平均池化层以保留红外图像纹理细节信息。为验证提出方法的有效性,通过在夜间使用红外热成像采集的农田数据集上进行试验,数据集分割目标包括田地、行人、植物、障碍物、背景。经试验验证,提出方法在夜间农田红外数据集上达到了85.1%的平均交并比(Mean Intersection over Union,MIoU),同时达到40帧/s的处理速度,满足对夜间农田的实时语义分割。

       

      Abstract: Abstract: In intelligent agricultural machinery, automatic navigation, and visual perception technology have been developed rapidly in recent years, and they also play a vital role in intelligent modern agriculture. Therefore, real-time semantic segmentation of farmland environment become an important part of visual environment perception in the intelligent agricultural machinery. The visible light sensing equipment is mainly used for image collection. However, particularly in the dark environment at night, the deficient imaging effect of visible light cameras can result in a decrease in the accuracy of semantic segmentation. Infrared thermal imaging can offer an alternatively way in this case, due to this technology uses the temperature difference of the object for imaging, rather than the light source. Therefore, the infrared thermal imaging can be used to clearly capture the image in the dark night, rain, mist, smoke, and other visible light sensing equipment that is not suitable. In this study, a method for real-time semantic segmentation of infrared images of farmland environment at night was proposed using the infrared thermal imaging system. An infrared real-time bilateral semantic segmentation network (IR-BiSeNet) was also addressed suitable for infrared images, in order to ensure the accuracy and real-time performance of infrared image semantic segmentation in the farmland environment at night. According to the characteristics of low resolution and fuzzy details of infrared images, the network was improved based on the BiSeNet structure, and the low-level features of infrared images were further integrated in its spatial path. In the network, the global maximum pooling layer was used to replace the global average pooling layer in the attention enhancement and the feature fusion module, in order to preserve the texture details of infrared image. The infrared farmland data was collected by the infrared thermal imaging to create a dataset at night, thereby to train a semantic segmentation model suitable for the farmland environment in this case. The segmentation targets of dataset included the fields, pedestrians, plants, obstacles, backgrounds, using the data augmentation to produce the dataset of infrared night farmland. Five representative semantic segmentation methods were selected to verify the proposed method, including BiSeNet、DenseASPP、DeeplabV3+、DFANet, and CGNet. Experimental results showed that the proposed method can achieved the mean intersection over union of 85.1%, and the processing speed of 40 frames/s. The method proposed in this study can be used the infrared thermal imaging to perform real-time farmland environment semantic segmentation at night, which can greatly improve the visual perception of intelligent agricultural machinery at night.

       

    /

    返回文章
    返回