Abstract: The extensive range of olive cultivars available in the market exhibit minor differences in their phenotypic traits. Nonetheless, their quality attributes, particularly the oil content and fatty acid composition, significantly differ among distinct cultivars, resulting in the emergence of use iinferior products as superior products in the market. The accurate and quick identification of olive cultivars holds significant importance in enhancing the production and quality of olives. As such, delving into the study of olive cultivar identification is crucial for the advancement of the olive industry. This study presents a novel approach to address the challenge of identifying olive cultivars in natural conditions. Specifically, a bilinear attentional EfficientNet model is proposed, which incorporates the bilinear network design concept and attention mechanism. The model is trained and evaluated using four commonly planted olive cultivars (i.e., Frantoio, Leccino, Picholine, and Ezhi 8) in Longnan, Gansu. The experimental results demonstrate the effectiveness of the proposed model in accurately and quickly identifying different olive cultivars. A bilinear network has been suggested to comprehensively extract feature information from olive images, for the limited phenotypic differences across different olive cultivars. In light of this, the selection of a feature extraction network has been made with consideration for both speed and accuracy, leading to the selection of the EfficientNet-B0 network. To tackle the challenge of identifying olive cultivars under natural conditions which are prone to intricate background interferences, a novel approach combining convolutional block attention module (CBAM) with bilinear network has been proposed. This approach facilitates the model in selectively focusing on the salient features responsible for cultivar identification during the feature extraction process of olive images. Upon conducting experiments, the bilinear attention EfficientNet model presented in this study has exhibited an overall accuracy of 90.28% and an inference time of 9.15 ms in identifying four distinct olive cultivars. These experiment results demonstrate that the proposed model better achieved better rapaid and accurate identification of olive cultivars under natural conditions. The present study proposes a bilinear attention EfficientNet model for identifying olive cultivars and utilizes gradient-weighted class activation mapping (Grad-CAM) to analyse its performance. The results demonstrate that the proposed model exhibits a greater attention towards fruit regions, as well as some leaf regions, within olive images. These findings are in agreement with the expert knowledge and experience of human practitioners. The analysis of heat maps generated from misidentified olive images revealed that inadequate focus on the fruit and leaf regions, which are pivotal for successful identification of cultivars, was the primary contributing factor to misidentification. The outcomes of the ablation experiments indicated that the bilinear network and the CBAM exhibited a positive impact on the precision of olive variety recognition. To ascertain the efficacy of the method elucidated in this manuscript, a set of comparative experiments has been formulated. The primary objective of these experiments is to juxtapose the proposed bilinear attention EfficientNet model against the conventional cultivar identification models, including bilinear ResNet34, EfficientNet-SE attention, bilinear ResNet18, bilinear VGG16, and bilinear GoogLeNet. The experimental results obtained from the comparison analysis provide that the proposed bilinear attention EfficientNet model exhibits superior performance in terms of overall accuracy for the identification of olive cultivars. The bilinear attention EfficientNet model's accuracy for the identification of olive cultivars exceeded that of bilinear ResNet34, EfficientNet-SE attention, bilinear ResNet18, bilinear VGG16, and bilinear GoogLeNet models by 12.78, 11.53, 11.11, 10.70, and 5.00 percentage points, respectively. The present study establishes a foundation for resolving the challenge of accurately identifying olive cultivars in natural conditions. Moreover, it offers a valuable point of reference for the identification of cultivars of diverse crops.