Flower bud detection model for hydroponic Chinese kale based on the fusion of attention mechanism and multi-scale feature

An accurate detection of flower bud features can greatly contribute to classifying the maturity for the timely harvesting of the hydroponic Chinese kale. The Faster Region-based-convolutional neural network (R-CNN) can be widely expected to serve as a compelling accuracy of detection without high real-time performance. However, the shape and size of flower buds vary greatly in the different varieties of hydroponic Chinese kale. The flower buds, stems, and leaves are also similar in color features. In this study, an improved Faster R-CNN model was proposed to accurately detect the flower buds of hydroponic Chinese kale in natural environment using the fusion of attention mechanism and multi-scale feature. The first 37 layers of InceptionV3 network were first selected as the basic network of feature extraction for the rich features without overfitting. The Squeeze-and-Excitation Network (SENet) was embedded with the ReductionA, InceptionA, and InceptionB modules to enhance the weight of the channels containing valid feature information, but to reduce the interference from the irrelevant background. The extraction features from the second to the fourth convolution group were output to the Feature Pyramid Network (FPN) layer, where a multi-scale FPN layer was obtained for the Region Proposal Network (RPN) during fusion operation. Different anchor sizes were also designed for each FPN feature map, according to the target frame size of flower buds. The improved model was verified using the dataset of Lubao Chinese kale (1 255 images), and Hongkong Chinese kale (1 319 images), as well as the mixed dataset of two varieties. The precision rate, recall rate, average precision, and mean average precision were also selected to evaluate the performance of the improved model in the experiments. The results showed that: 1) The average accuracy of the model increased, while, the comprehensive loss declined gradually, with an increase of the iteration. The peak value of average precision appeared stable after 10 iterations, indicating the strong fitting and generalization ability of the model. The mean average precisions of the model were 96.5%, 95.9%, and 96.1% for the Lubao Chinese kale, the Hongkong Chinese kale, and the mixed dataset, respectively, indicating a high detection accuracy of the model for different varieties. 2) An ablation experiment was carried out on the mixed dataset. The mean average precision of the basic feature extraction network was 90.7%. The mean average precisions were 94.3% and 94.8% for the basic feature extraction networks combining with the SENet and FPN module, respectively. 3) In the immature, mature, and over-mature flower buds, the average precisions were 92.3%, 98.2%, and 97.9%, respectively, where the mean average precision was 96.1%. As such, either the SENet or FPN module contributed greatly to improving the detection accuracy for the hydroponic Chinese kale with different maturity. 4) The mean average precisions of the improved model were improved by 10.8, 8.3, 6.9, and 12.7 percentage points, respectively, compared with the VGG16, ResNet50, ResNet101, and InceptionV3 networks. Furthermore, the mean average precision of hydroponic Chinese kale with different maturity was all above 90%, when the recall rate was 80%, indicating the high robustness of the model. The finding can provide a strong reference to determine the harvest period of hydroponic Chinese kale.