Classifying crops from hyperspectral images using spatial-spectral dual branches and dynamic feature selection

Hyperspectral remote sensing images can capture the continuous spectral curves of the ground surface, and then enable the delicate classification of crops, due to their diagnostic capability of high spectral data. The conventional algorithms of hyperspectral image classification are highly required to explore the spatial information and effective utilization of spectral images. It cannot be fully addressed on the "same object, different spectra; different objects, same spectra" problem, such as the Hughes phenomenon. The classification accuracy has been improved with the continuous development of deep learning-based classification on hyperspectral images. However, there are still several issues that need to be addressed: 1) Traditional convolution layers can be calculated by the equal weights for all pixels in the feature extraction, particularly without considering the spatial correlation and local similarity within the feature neighborhood. 2) Although the previous algorithms have separately captured the spatial and spectral features in the hyperspectral images, the extraction of high-dimensional features can often result in redundancy, which is lacking in effective feature selection. 3) Traditional algorithms of deep learning can often merge the temporal and spatial features using a single feature constraint method, especially for loss calculation. Comprehensive feedback is required on the classification from the spatial and spectral perspectives. The comprehensiveness of the fused features can remain to be examined during this time. In this study, a hyperspectral classification algorithm was proposed using a spatial-spectral dual-branch architecture and a dynamic feature selection strategy. The channel and spatial attention modules were introduced to extract and screen the spatial-spectral joint features. Moreover, the gated convolutional layers were used to calculate the correlation of extracted features, enabling dynamic feature selection in both the spatial and channel dimensions. A novel loss function was designed to constrain the classification, in terms of the spatial and spectral perspectives. The results indicate that: 1) The DBDS algorithm performed better in the time efficiency and accuracy, compared with the mainstream crop classification. On the JAAS dataset, the OA and Kappa of the improved algorithms were 99.35% and 99.2%, respectively, which were 4.91% and 6.12%, 6.82% and 8.53%, 2.12% and 2.63%, 2.04% and 2.54% higher than those of CDCNN, WCRN, DBDA, and DCNN, respectively. On the WHU-Hi-HanChuan dataset, the OA of 99.49% and the Kappa of 99.41% were 1.67% and 1.96%, 3.23% and 3.80%, 2.00% and 2.35%, 1.10% and 1.29% higher than those of CDCNN, WCRN, DBDA, and DCNN, respectively. On the WHU-Hi-Longkou dataset, the OA and Kappa were also improved, reaching 99.8% and 99.74%, respectively, which were 1.30% and 1.71%, 0.59% and 1.74%, 0.71% and 0.93%, 0.57% and 0.76% higher than those of CDCNN, WCRN, DBDA, and DCNN, respectively. 2) The spatial-spectral features were effectively extracted to reduce the model degradation in the dataset with the limited samples, complex and difficult-to-distinguish land cover classification. On the WHU-Hi-Hanchuan dataset, the DBDA algorithm was focused on the extraction of spectral information. The better performance was achieved in the tasks of crop classification with sufficient samples, indicating the high f1 scores for the strawberry, Cowpea, soybean, and sorghum (99.60%, 99.10%, 99. 41%, and 99.71%, respectively). However, the significant degradation of the model was observed to classify the lack-sample targets, where the f1 scores for the watermelon and bare soil were only 86.30% and 91.07%, respectively. By contrast, the DBDS algorithm improved the recognition accuracy of various crops with the f1 scores of 96.65% and 98.23% for the watermelon and bare soil, respectively, indicating the effective extraction and utilization of spatial and spectral features. Therefore, the higher accurate and efficient classification was achieved in the fine-grained crop in the regions with the imbalanced samples and the diverse types of land covering. Therefore, 2D convolution-based hyperspectral classification algorithms can be expected to obtain the effective extraction of spatial-spectral features with comparable accuracy to 3D convolution with fewer parameter computations. This finding can also provide important implications and strong references for the target recognition tasks using hyperspectral data.