Research on anti-shadow tree detection method based on generative adversarial network

Ye Yang; Shen Bingyan; Shen Yuqi

doi:10.11975/j.issn.1002-6819.2021.10.014

Volume 37 Issue 10

May 2021

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2021 > 37(10): 118-126. > DOI: 10.11975/j.issn.1002-6819.2021.10.014

Ye Yang, Shen Bingyan, Shen Yuqi. Research on anti-shadow tree detection method based on generative adversarial network[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2021, 37(10): 118-126. DOI: 10.11975/j.issn.1002-6819.2021.10.014

Citation:

PDF (7553 KB)

Research on anti-shadow tree detection method based on generative adversarial network

College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou 310023, China

More Information

Received Date: March 11, 2020
Revised Date: April 12, 2021
Published Date: May 14, 2021

Graphical Abstract

Abstract

Abstract

Abstract: High-scoring remote sensing imaging has widely been applied in most management of agriculture and forestry, especially to monitor and evaluate crops and forest resources on a large scale. Nevertheless, there is a great challenge to the accuracy of single tree identification and detection during the image acquisition, due mainly to the fact that the shadow area is inevitably formed by the light. The shadow areas in the remote sensing images can be assumed as a kind of noise in the image sampling. As such, the degradation of high-resolution parameters can cause image distortion after post-processing. In this study, an anti-shadow tree detection method was proposed to detect the single tree with shadow interference using a generative adversarial network (GA-Faster RCNN). This framework consisted of a Faster RCNN network and a tree generator. The Faster RCNN network was mainly used for the tasks of feature extraction and detection. The tree generator was utilized to process the shadows in tree detection. The adversarial generation strategy was adopted by the tree generator to learn generating the minimum feature information characterizing trees. The generator was first trained separately and then put into the Faster RCNN network to finally lock its parameters. Two parts were then trained end-to-end to further improve the tree recognition ability of the network. The GA-Faster RCNN was also compared with 3 state-of-the-art methods, including region-growing, progressive cascaded convolutional neural network, and Faster RCNN on three test areas with shadows. Test area 1 presented a lot of shadows of trees, where the canopy density of trees was very high. Test area 2 showed fewer tree shadows and lower canopy closure, compared with test area 1. The shades of trees and the canopy density of trees in test area 3 were between those in test area 1 and 2. Results demonstrated that the GA-Faster RCNN achieved the highest harmonic average of precision and recall (F1) on the test area 1, 2, and 3, which were 78.4%, 91.6%, and 81.7%, respectively. The average F1 of three test areas was 84.7% for the GA-Faster RCNN, 6.2 percentage point higher than that of Faster RCNN. The user accuracy (UA) and producer accuracy (PA) of GA-Faster RCNN were also the highest among four methods, where UA was 79.8%, 95.0%, 85.3%, and PA was 77.0%, 88.5%, and 78.4% on test area 1, 2, 3, respectively. Moreover, a significance analysis, McNemar, was performed to eliminate the interference of experimental errors and other factors. It was found that there was a statistically significant difference between the three comparison methods and GA-Faster RCNN. The shadow misrecognition rate SR (proportion of the count of shadows misrecognized as trees to the count of total recognized trees) of GA-Faster RCNN was compared with that of Faster RCNN on test area 1, in order to clarify the effect of the mask on tree identification. Although the SR of GA-Faster RCNN was 13.8%, higher than that of Faster RCNN (8.6%), the UA and the number of missed tree identification were both better than those of Faster RCNN. Therefore, the GA-Faster RCNN behaved significant advantages over the other identification. In addition, the GA-Faster RCNN can still maintain the detection stability, when using different feature extraction networks, including ResNet101, ResNet50, and DenseNet. Consequently, the adversarial generative training strategy is highly suitable for learning the minimum feature information characterizing trees, while effectively reducing the interference of shadows, indicating the promising practical value for higher accuracy of tree detection.
- remote sensing,
- algorithm,
- tree,
- detection,
- image,
- shadow interference,
- deep learning

FullText(HTML)

References (34)

References

[1]	国家林业和草原局. 中国森林资源报告[M]. 北京：中国林业出版社，2019.
[2]	王维刚，史海滨，李仙岳，等. 遥感订正作物种植结构数据对提高灌区SWAT模型精度的影响[J]. 农业工程学报，2020，36(17)：158-166.Wang Weigang, Shi Haibin, Li Xianyue, et al. Effects of correcting crop planting structure data to improve simulation accuracy of SWAT model in irrigation district based on remote sensing[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(17): 158-166. (in Chinese with English abstract)
[3]	李宇宸，张军，薛宇飞，等. 基于Google Earth Engine的中老缅交界区橡胶林分布遥感提取[J]. 农业工程学报，2020，36(8)：174-181.Li Yuchen, Zhang Jun, Xue Yufei, et al. Remote sensing image extraction for rubber forest distribution in the border regions of China, Laos and Myanmar based on google earth engine platform[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(8): 174-181. (in Chinese with English abstract)
[4]	赵晋陵，金玉，叶回春，等. 基于无人机多光谱影像的槟榔黄化病遥感监测[J]. 农业工程学报，2020，36(8)：54-61.Zhao Jinling, Jin Yu, Ye Huichun, et al. Remote sensing monitoring of areca yellow leaf disease based on UAV multi-spectral images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(8): 54-61. (in Chinese with English abstract)
[5]	Hellesen T, Matikainen L. An object-based approach for mapping shrub and tree cover on grassland habitats by use of LiDAR and CIR orthoimages[J]. Remote Sensing, 2013, 5(2): 558-583.
[6]	Yang L, Wu X, Praun E, et al. Tree detection from aerial imagery[C]//Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, Seattle, USA, 2009: 131-137.
[7]	Pollock R J. The Automatic Recognition of Individual Trees in Aerial Images of Forests Based on a Synthetic Tree Crown Image Model[D]. Vancouver: The University of British Colombia, 1996: 172.
[8]	Larsen M, Rudemo M. Optimizing templates for finding trees in aerial photographs[J]. Pattern Recognition Let, 1998, 19(12): 1153-1162.
[9]	Novotny J, Hanu? J, Luke? P, et al. Individual tree crowns delineation using local maxima approach and seeded region-growing technique[C]//Proceedings of Symposium GIS Ostrava, Ostrava, Czech Republic, 2011: 27-39.
[10]	郭昱杉，刘庆生，刘高焕，等. 基于标记控制分水岭分割方法的高分辨率遥感影像单木树冠提取[J]. 地球信息科学学报，2016，18(9)：1259-1266.Guo Yushan, Liu Qingsheng, Liu Gaohuan, et al. Individual tree crown extraction of high resolution image based on marker-controlled watershed segmentation method[J]. Journal of Geo-information Science, 2016, 18(9): 1259-1266. (in Chinese with English abstract)
[11]	Malek S, Bazi Y, Alajlan N, et al. Efficient framework for palm tree detection in UAV images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(12): 4692-4703.
[12]	Windrim L, Bryson M. Detection, segmentation, and model fitting of individual tree stems from airborne laser scanning of forests using deep learning[J]. Remote Sensing, 2020, 12(9): 1469.
[13]	Dai W, Yang B, Dong Z, et al. A new method for 3D individual tree extraction using multispectral airborne LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144: 400-411.
[14]	Marinelli D, Paris C, Bruzzone L. An approach to tree detection based on the fusion of multitemporal LiDAR data[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(11): 1771-1775.
[15]	Marselis S M, Tang H, Armston J, et al. Exploring the relation between remotely sensed vertical canopy structure and tree species diversity in Gabon[J]. Environmental Research Letters, 2019, 14(9): 1748-9326.
[16]	Sainath T N, Mohamed A, Kingsbury B, et al. Deep convolutional neural networks for LVCSR[C]//IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, 2013: 8614-8618.
[17]	Le Q V, Ranzato M A, Monga R, et al. Building high-level features using large scale unsupervised learning[C]//In Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, Canada, 2013: 8595-8598.
[18]	Zhu X, Tuia D, Mou L, et al. Deep learning in remote sensing: a comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4): 8-36.
[19]	Neupane B, Horanont T, Hung N D. Deep learning based banana plant detection and counting using high-resolution red-green-blue (RGB) images collected from unmanned aerial vehicle (UAV)[J]. PLoS ONE, 2019, 14(10): e0223906.
[20]	Torres D L, Feitosa R Q, Happ P N, et al. Applying fully convolutional architectures for semantic segmentation of a single tree species in urban environment on high resolution UAV optical imagery[J]. Sensors, 2020, 20(2): 563.
[21]	Li W, Fu H, Yu L, et al. Deep learning-based oil palm tree detection and counting for high-resolution remote sensing images[J]. Remote Sensing, 2016, 9(1): 22.
[22]	Guirado E, Tabik S, Alcaraz-Segura D, et al. Deep-learning versus OBIA for scattered shrub detection with google earth imagery: ziziphus lotus as case study[J]. Remote Sensing, 2017, 9(12): 1220.
[23]	Culman M, Delalieux S, Tricht K V. Individual palm tree detection using deep learning on RGB imagery to support tree inventory[J]. Remote Sensing, 2020, 12(21): 3476.
[24]	Ren S, He K, Girshick R, et al. Faster RCNN: towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing System, Vancouver, BC, Canada, 2015: 91-99.
[25]	Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779-788.
[26]	Ding B, Long C, Zhang L, et al. ARGAN: Attentive recurrent generative adversarial network for shadow detection and removal[C]//International Conference on Computer Vision (ICCV), Venice, Italy, 2020: 10212-10221.
[27]	Zheng Q, Qiao X, Cao Y, et al. Distraction-aware shadow detection[C]//Conference on Computer Vision and Pattern Recognition (CVPR), California, USA, 2019: 5162-5171.
[28]	Wang X, Shrivastava A, Gupta A. A-Fast-RCNN: hard positive generation via adversary for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017: 2606-2615.
[29]	Gougeon F. A crown-following approach to the automatic delineation of individual tree crowns in high spatial resolution images[J]. Canadian Journal of Remote Sensing, 1995, 21(3): 274-288.
[30]	Erikson M. Segmentation of individual tree crowns in colour aerial photographs using region growing supported by fuzzy rules[J]. Canadian Journal of Forest Research, 2003, 33(8): 1557-1563.
[31]	Lamar W R, McGraw J B, Warner T A. Multi-temporal censuring of a population of eastern hemlock (Tsuga canadensis L. ) from remotely sensed imagery using an automated segmentation and reconciliation procedure[J]. Remote Sensing of Environment, 2005, 94(1): 133-143.
[32]	Larsen M, Eriksson M, Descombes X, et al. Comparison of six individual tree crown detection algorithms evaluated under varying forest conditions[J]. International Journal of Remote Sensing, 2011, 32(20): 5827-5852.
[33]	Dong T, Shen Y, Zhang J, et al. Progressive cascaded convolutional neural networks for single tree detection with google earth imagery[J]. Remote Sensing, 2019, 11(15): 1786.
[34]	Foody G M. Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy[J]. Remote Sensing, 2004, 70: 627-633.