Image classification method for tomato leaf deficient nutrient elements based on attention mechanism and multi-scale feature fusion

This study aims to realize the accurate identification of deficient nutrients elements in tomato leaves. An experiment was conducted on the lack of nutrients in the climate chamber in the laboratory of Big Data Intelligence Department of the Beijing Academy of Agriculture and Forestry Sciences, China. An artificial climate chamber was also selected to regulate the growth environment factors of tomato plants for the specific lack of nutrients. Three types of nutrient deficiency groups were set, namely nitrogen deficiency, phosphorus deficiency, and potassium deficiency, as well as a normal control group. The experiment was started with the appearance of nutrient-deficient traits in seedlings, and the images of nutrient-deficient leaves were then collected according to the growth stages. The experimental results show that there were diversity and differences in the traits of tomato nutrient deficiency. Specifically, there were relatively small changes of leaves in the early stage of tomato nutrient deficiency. Furthermore, it was difficult to capture the details and textures, due to the smaller area of traits. For example, the manifestation of phosphorus deficiency was that the leaves gradually turn purple along the veins. The trait details were hardly identified in the early stage of phosphorus deficiency, due mainly to the mostly small vein structure. Particularly, tomato leaves under different conditions of nutrient deficiency presented similar color and texture characteristics at a certain stage. For example, the leaves were both slightly yellow in the early stage of nitrogen deficiency and the early stage of potassium deficiency. The only slight difference was the characteristic display of morphology in the size of characteristic areas. There were obvious differences in color and texture at different stages under the same nutrient deficiency. The images were collected from the climate chamber to serve as the experimental data. An attempt was made on the inconsistency of feature area size, and the difficulty of feature extraction, resulting from the different types of nutrient deficiency, the insignificant early traits of nutrient deficiency, and the large differences in the characteristics of each growth period. Therefore, an image classification was proposed for the nutrient deficiency of tomato leaves using an attention mechanism and multi-scale feature fusion convolutional neural network (MSFF & AM-CNNs). First of all, a multi-scale feature fusion (MSFF) module was set for nutrient deficiency traits, due to the low efficiency of a fixed-scale convolution kernel for different sizes. The MSFF input image was carried out with multi-channel feature stitching after the MSFF convolution kernel of multiple scales, where the shallow image was multiplied while expanding the number of channels. As such, the fusion of scale features was adopted in this structure. Secondly, an MSFF&AM module was used to improve the large-scale convolutional layer for the extraction of shallow features using the attention mechanism (CBAM). A multi-scale fusion of Bottleneck was also utilized to improve the Dense Block for the extraction of deep features. Deep-MSFF Block aimed to combine the attention mechanism and the MSFF module, where the multiple feature channels were selectively emphasized the global multi-scale information feature function. The recalibration of features in nitrogen deficiency was improved on the tomato leaves the classification accuracy. Finally, a Focal Loss function was introduced as the loss function to reduce the weight of easy-to-differentiate samples. Correspondingly, the image recognition model of tomato elements lacking was widely expected to focus on difficult-to-classify samples during training, particularly for the overall performance of the model. The experiments demonstrated that the MSFF & AM-CNNs can meet the high-precision classification requirements of nutrient-deficient images in tomato leaves, particularly with high recognition accuracy and wide applicability (an average recognition accuracy rate of 95.92%). The model can also be expected for the identification of plant leaf nutrient deficiency.