Detecting citrus fruit maturity using improved YOLOv8s

Accurate and automatic identification of citrus fruit ripeness can greatly contribute to the fruit flavor and quality during intelligent harvesting. However, citrus fruits in the natural environment can not ripen at the same time, leading to the varying degree of ripeness on the same tree. It is also difficult to identify the maturity of citrus fruits, due to the occlusion by branches and leaves, as well as the overlapping occlusion among fruits. In this study, the national standard was developed for the pickable maturity of citrus fruits, with reference to the national standard of the Ministry of Agriculture, NY/T 716-2003, and the operating procedure of wide-skinned citrus for post-harvest storage and logistics, GH/T 1336-2021, according to the phenotypic characteristics of citrus fruits during the ripening period. The citrus fruits to be picked were classified into two categories: 1) Citrus fruits beyond the harvestable maturity of UM (unpickable maturity, UM); and 2) Citrus fruits reaching pickable maturity, PM (pickable maturity, PM). An improved YOLOv8s model was also proposed to identify the ripeness degree of citrus fruits. Firstly, a hybrid attention transformer (HAT) module was added into the network backbone. The low-quality citrus fruit images were reconstructed into the high-resolution images. More information was utilized to capture the ripeness features of the fruits, thus improving the accuracy of the detection; Secondly, the detection head was replaced with a four-head adaptive spatial feature fusion (FASFF) detection head. Adaptive spatial fusion weights were learned at different spatial locations. Thus, different levels of citrus fruit features were effectively fused together to improve the scale invariance of features. The improved YOLOv8s model was finally tested by 209 citrus fruit test images. The precision P of improved YOLOv8s model on the test set was 94.8%, the recall R was 88.6%, the mAP_0.5 and mAP_0.5~0.95 were 95.6% and 88.9%, respectively. Comparing with the Faster-RCNN, YOLOv3-spp, YOLOv7, and original YOLOv8s models, the precision of the improved YOLOv8s model was improved by 26.0, 5.8, 4.8, and 5.5 percentage points, respectively, while the mean average precision mAP_0.5 was improved by 0.5, 1.7, 1.6, and 1.6 percentage points, respectively, and the mAP_0.5~0.95 was improved by 26.4, 1.9, 4.5, and 3.0 percentage points. In terms of the number of parameters of the model, the improved YOLOv8s model was 21.51 M, although it was 10.38 M higher than that of the original YOLOv8s. The precision P, mAP_0.5 and mAP_0.5~0.95, were higher than those of the original YOLOv8s by 5.5, 1.6, and 3.0 percentage points, respectively. The number of transmitted frames per second of the model was 52.9, which was fully met the real-time detection requirements of the mobile edge device. The improved YOLOv8s model shared the high detection accuracy and real-time detection speed. Therefore, the improved YOLOv8s model can be expected to detect the ripeness of citrus fruits in the natural environment. The finding can provide the effective support to the selective harvesting of automatic picking robots.