毛亮, 薛月菊, 朱婷婷, 魏颖慧, 何俊乐, 朱勋沐. 自然场景下的挖掘机实时监测方法[J]. 农业工程学报, 2020, 36(9): 214-220. DOI: 10.11975/j.issn.1002-6819.2020.09.024
    引用本文: 毛亮, 薛月菊, 朱婷婷, 魏颖慧, 何俊乐, 朱勋沐. 自然场景下的挖掘机实时监测方法[J]. 农业工程学报, 2020, 36(9): 214-220. DOI: 10.11975/j.issn.1002-6819.2020.09.024
    Mao Liang, Xue Yueju, Zhu Tingting, Wei Yinghui, He Junle, Zhu Xunmu. Method for the real-time monitoring of the excavator in natural scene[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(9): 214-220. DOI: 10.11975/j.issn.1002-6819.2020.09.024
    Citation: Mao Liang, Xue Yueju, Zhu Tingting, Wei Yinghui, He Junle, Zhu Xunmu. Method for the real-time monitoring of the excavator in natural scene[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(9): 214-220. DOI: 10.11975/j.issn.1002-6819.2020.09.024

    自然场景下的挖掘机实时监测方法

    Method for the real-time monitoring of the excavator in natural scene

    • 摘要: 为实时监测违法用地现象,对作业挖掘机等施工机械进行实时监测至关重要。针对自然场景下由于背景复杂、光照不均匀及遮挡等导致作业挖掘机难以准确检测出的问题,该文采用类似SSD(Single Shot Detector)方法的网络结构,提出一种自然场景下的挖掘机实时监测方法。该方法采用堆叠DDB(Depthwise Dense Block)模块组成基础网络,实现浅层特征提取,并与高层特征融合,提高网络模型的特征表达能力;在MobileNetV2网络的基础上进行改进,设计 BDM(Bottleneck Down-Sampling Module)模块构成多尺度特征提取网络,使模型参数数量和计算量减少为SSD的68.4%。构建不同视角和场景下的挖掘机目标数据集,共计18 537张,其中15 009张作为训练集,3 528张作为测试集,并在主流Jetson TX1嵌入式硬件平台进行网络模型移植和验证。试验表明,该文方法的mAP(Mean Average Precision)为90.6%,其检测精度优于SSD和MobileNetV2SSD的90.2%;模型大小为4.2 MB,分别减小为SSD和MobileNetV2SSD的1/25和1/4,每帧检测耗时145.2 ms,相比SSD和MobileNetV2SSD分别提高了122.7%和28.2%,可以较好地部署在嵌入式硬件平台上,为现场及时发现违法用地作业提供有效手段。

       

      Abstract: Abstract: In order to monitor illegal land use in real time, video surveillance technology was used to monitor the vulnerable areas of illegal land use. Excavator was one of the most important construction machinery in the engineering construction, an automatic real-time detection of excavator could provide important information for non-contact field monitoring of illegal land. But it was difficult to accurately detect the excavator due to the complex background, uneven illumination and partial occlusion in natural scene, This paper proposed a real-time excavator detection algorithm in natural scene based on the SSD-like (Single Shot Detector). Specifically, the lightweight network DDB (Depthwise Dense Block) was used as the basic network to extract shallow feature and fuse with high-level features in the excavator objection model to enhance the feature representation capability. Meanwhile, the BDM (Bottleneck Down-sampling Module) which was designed based on the lightweight network MobileNetV2 was used as the multi-scale feature extraction network to reduce the parameter quantity and computation. The data sets included 18 537 images of excavators with different shooting angles and natural scenes, 15 009 images were used as training set and 3 528 images were chosen as test set randomly. To enhance the diversity of training data, data set expansion methods such as rotation and image were adopted. Based on the Caffe deep learning framework, the proposed model in this paper was trained with end-to-end approximate joint methods and the model weight was fine-tuned by using SGD (Stochastic Gradient Descent) algorithm. The DDB module of the network was initialized with the weights pre-trained on the PASCAL VOC dataset, and the training time and resources were further reduced by transferring learning. Then the model pre-trained on the large data sets was transplanted to excavator object detection by transfer learning. The proposed method was transplanted and performed on the mainstream Jetson TX1 embedded hardware platform, and experiments on the actual data set of detecting excavator object at different angles of view and natural scenes. Experiment results showed that the parameter quantity and computational complexity of proposed model with BDM was reduced by 68.4% compared to SSD, the mAP (Mean Average Precision) of proposed method reached 90.6% on the testing set, which was 0.4% and 0.4% higher than that of SSD based on VGG16 basic net and MobileNetV2SSD based on MobileNetV2 basic net, respectively. The model size of propose method was 4.2 MB, which was about 1/25 and 1/4 of SSD and mobilenetv2ssd, respectively, and the time-consuming of each frame was 145.2 ms, which was 122.7% and 28.2% faster than SSD and MobileNetV2SSD, respectively. The proposed method not only had better generalization and robustness, but also can be better deployed on the embedded hardware platform which demonstrated the feasibility of real-time monitoring of the excavator at site of illegal land use.

       

    /

    返回文章
    返回