Recognizing greenhouse instance and model transfer using deep learning

The timely and accurate extraction of greenhouse instance (GI) is of significant practical importance to estimate the vegetable cultivation areas and yield prediction. Deep learnings (driven by knowledge learned from large-scale sample data) can be expected to adaptively explore the features of image data, compared with the traditional image analysis, such as unsupervised, supervised, and object-oriented classification. The end-to-end accurate extraction of GI information can also ensure the model generalization for the less manual intervention. However, there are still two challenges to identify the GI using deep learning. One is that the multiple GI can be mistakenly assumed as a continuous distribution in the dense areas of GI, leading to the segmentation errors. Another is that the degradation of performance could occur, when transferring the GI model to the large-scale spatial context. In this study, the region-boundary instance extraction was proposed using convolutional neural networks (CNNs) and morphological post-processing. At first, a region-boundary multi-class model was constructed to generate the boundary auxiliary labels of greenhouse. The network loss function was modified to enhance the boundary information recognition, and then to facilitate the removal of greenhouse boundaries from the recognition, thereby achieving the extraction of GI. Subsequently, the high-precision GI data was obtained using morphological operations, such as the instance object dilation and the minimum bounding rectangles. The high-resolution three-band remote sensing data was collected from Shouguang, Shandong Province, in order to train the base model. Three transfer modes (namely, pure transfer, scale-adaptive, and model fine-tuning mode) were then explored in five transfer research areas, including Xinjiang, Liaoning, Yunnan, Hubei, and Zhejiang Province. The accuracy of GI extraction was evaluated using common semantic segmentation performance metrics, unit intersection over union (UIoU), and instance recall rate (IRR). The research results indicate that the UNet was better suited to construct as the "Region-Boundary" multi-class model, compared with the semantic segmentation networks, such as PSPNet, DeeplabV3+, and HRNet. The higher semantic accuracy was achieved in the UNet with the UIoU and IRR of 2.43 and 2.91 percentage points higher, respectively, compared with the overall suboptimal HRNet. Furthermore, two morphological post-processing operations (instance dilation and the minimum bounding rectangle recognition) were simultaneously introduced to increase the UIoU and IRR by 10.53 and 1.44 percentage points, respectively. The scale adaptation mode was then adopted to adjust the input image resolution. The UIoU and IRR were improved from 3.02 to 22.71 percentage points, and from 2.40 to 30.33 percentage points, respectively, in all test datasets of migration areas, compared with the simple migration mode. The fine-tuning of the base model was utilized to adjust the input image resolution in the model fine-tuning mode. The UIoU and IRR were improved ranging from 37.76 to 50.67 percentage points, and from 46.04 to 76.87 percentage points. The higher accuracy was achieved in the GI recognition, with the UIoU and IRR of 13.64 and 14.18 percentage points higher than the conventional approaches, respectively. Simultaneously, the model transfer was applied to select the different migration modes, according to the scene differences between the predicted and training regions. The automated mapping of GI can be expected to efficiently and accurately extract the GI information over the large-scale areas. The finding can provide the information support to the intelligent construction of agricultural facilities.