Detecting kiwi flowers in natural environments using an improved YOLOv5s

Abstract: Artificial pollination can be essential to improve the fruit quality in kiwifruit production. An efficient detection of kiwifruit flowers is one of the key technologies in the automatic pollination machinery. In this study, an improved YOLOv5s model (YOLOv5s_S_N_CB_CA) was proposed to rapidly and accurately detect the kiwifruit flowers. The C3HB module and criss-cross attention (CCA) were added into the YOLOv5s. The sample slicing was combined to add the negative sample processing, in order to enhance the feature extraction of the model for the kiwifruit flowers, particularly for the detection accuracy and detection speed of the model. A total of 1032 images of kiwifruit flowers were collected from a trellised kiwifruit orchard grown in a natural environment, including 779 images on sunny days and 253 images on cloudy days. Two periods of light conditions under sunny days were considered, including 726 images of kiwifruit flowers under the 9:00-11:00 hours and 378 images of kiwifruit flowers under the 15:00-17:00 hours. Two occlusion cases were selected, with 726 images of kiwi flowers with occlusion and 306 images of kiwi flowers without occlusion. The captured images of kiwifruit flowers were classified into three categories, including kiwifruit buds, kiwifruit flowers, and pollinated kiwifruit flowers. Three targets were labelled separately, and then sent to the improved YOLOv5s model for training. A total of 300 iterations of training were implemented for the improved model. The results showed that the improved model shared the detection accuracy of 85.21%, the recall of 90%, the mean average precision (mAP) of 92.45% at an intersection over union (IoU) ratio of 0.5, a model size of 14.6 MB, and a detection speed of 35.47 fps. Compared with the four improved YOLOv5s models with only sample scaling or two resolutions, sample slicing, and adding negative samples, the C3HB-CCA module and focal loss function, the mAP0.5 were improved 31.91, 38.32, 2.55, and 1.08 percentage points, respectively, while the mean average accuracy at IoU of 0.5-0.95 ( mAP0.5-0.95) by 34.38, 42.93, 1.92, and 1.37 percentage points, respectively. The improved model increased the recall by 2, 7, and 12 percentage points, compared with the original, YOLOv4, and SSD model, respectively, while the mAP0.5 was improved by 2.55, 9.95, 13.64 percentage points, and 34.15%, 144.62%, and 20.03% improvement in the detection speed, respectively. The original and improved models were then used to detect the kiwifruit flowers under different weather light intensities, or under different light intensities at the different times of the day on sunny days. The results showed that the improved model had 85.17% and 83.88% accuracy, 90% and 89% recall, and 91.96% and 91.15% mAP0.5 for the detection of the kiwifruit flowers under sunny and cloudy skies, respectively. The improved model shared 84.47% and 84.79% accuracy, 89% and 89% recall, and 92.11% and 92.10% mAP0.5 for the detection of kiwifruit flowers in the hours of 9:00-11:00 and 15:00-17:00 on sunny days, respectively. The better performance was achieved in the improved model, compared with the original. Therefore, the improved YOLOv5s-based detection model was achieved in the rapid and accurate detection of kiwifruit flowers with the high robustness while maintaining lightweight. The finding can also provide the technical support to develop the automated pollination equipment for kiwifruit.