Identifying fine-grained vineyard pest using attribute knowledge graph

Pest infestation is one of the main reasons for the decrease in crop yield and quality in vineyards. Deep learning can be expected to identify the pest species for scientific prevention and control strategies, in order to improve the production level of vineyards. In this study, an identification model was proposed, termed as ACKGViT (attribute characteristics knowledge graph enhanced vision transformer), for the fine-grained vineyard pest using constructed attribute characteristics knowledge graph (KG). Particularly, the KG was fully utilized to treat the variety, complex morphology, and low correlation of vineyard pests. A specific description was also given for the entity attribute characteristics and inter-entity association of pests. The improved approach was then applied in various agricultural settings. Pest recognition challenges were clarified across diverse domains. The inherent structure of KGs incorporated valuable contextual complements with the visual features extracted by the visual encoder. The graph convolutional network facilitated the efficient learning of pest attributes and relationships from the KG. This information was then seamlessly integrated into the classification. ViT (vision transformer) was used as the backbone network to extract the high-level representation of images. The fine-grained attribute was combined with the associated features provided by KG for pest identification in vineyards. The KG recognition model of vineyard pests was also trained and optimized using traditional features of images. Traditional image features (such as color, texture, and shape) were extracted using various techniques, including histogram-based methods, wavelet transform, and local binary patterns. The essential information was obtained about the pest species and their characteristics, enabling the KG to accurately capture the associations between different pests. Two datasets were evaluated: one was the GP21 dataset, which was composed of 21 vine-related pest categories from the IP102 public large-scale pest dataset, and another was the GP8 dataset, including eight vine-related pest categories that were collected from agricultural production sites and hand-labelled. The experimental results show that the performance of ACKGViT was significantly better than the general pre-training networks. The accuracies of GP21 and GP8 datasets reached 91.21% and 95.03% respectively, which increased by 1.64 and 1.17 percentage points, compared with the original ViT. The attribute characteristic KG was introduced to assist the ViT with more pest information on the vineyard. In conclusion, accurate and rapid pest identification was offered for the fine-grained vineyard pest recognition model with attribute feature KGs. The improved model can substantially contribute to the targeted and effective pest prevention and control strategies, and further improve the grape yield, quality, and overall productivity in vineyards. Future work can expand the KG into additional attributes and relationships. Other sources of information can also be introduced, such as temporal data or expert knowledge. Furthermore, the proposed approach can be extended into other agricultural domains, indicating the versatility of pest recognition under various settings. Additionally, better performance can be expected to achieve in the high efficiency of the graph convolutional network and the full integration of KG information during classification.