Abstract: Leaf diseases have seriously threatened the quality and yield of apple fruits. Efficient and accurate identification of apple leaf diseases is of great significance for refined orchard management. However, it is difficult to detect apple leaf disease at the early stage, due to the small target disease lesion and complex scenarios. Furthermore, the parameters of the detection model are too large to deploy on mobile terminals or embedded devices. In this study, a lightweight detection was proposed to detect the small apple leaf disease using improved You Only Look Once Version 5s (YOLOv5s). Firstly, the ShuffleNet v2 lightweight network was employed as the backbone network of YOLOv5s, in order to reduce the parameters and float-point operations per second (FLOPs) of the new network. Secondly, the Convolutional Block Attention Module (CBAM) was adopted to focus on the features of the small target disease of apple leaves. High performance was improved in the network information transmission and the sensitivity of the model to features, and then combined the spatial and channels to enhance the cross-channel interaction of the model. Thirdly, an improved Receptive Field Block-s (RFB-s) branch was added to obtain the multi-scale features for the high feature extraction of the model and the detection accuracy of apple leaf disease. Finally, SCYLLA-Intersection over Union (SIoU) was selected as the loss function of the bounding box regression. The performance of disease spot localization was enhanced for the high accuracy and training speed of the model. The experimental results demonstrated that the mAP_0.5 and FPS of the improved YOLOv5s reached 90.6% and 175 frames/s, respectively. The mAP_0.5 of the improved model increased by 0.8%, while the number of parameters was reduced by 6.17 MB, and the calculation amount was reduced by 13.8 G FLOPs, compared with the baseline model YOLOv5s. Particularly, the average precision of small disease targets was up to 38.2%, indicating the high efficiency and robustness of the model. Moreover, the mAP_0.5 were 2.0, 1.4, 2.0 and 9.4 percentage points higher in the improved model YOLOv5s, respectively, and the average accuracy rate of small disease targets increased by 1.5, 2.0, 1.8 and 2.1 percentage points, respectively, compared with Squeeze-and-Excitation (SE), Efficient Channel Attention (ECA), CoordAttention (CA), and Non-local neural network in the CBAM attention module. The SIoU bounding box loss function presented the highest detection accuracy, compared with the DIoU, CIoU, and GIoU. In addition, the improved YOLOv5 object detection model displayed the minimum number of parameters and calculations. Specifically, the detection accuracy of frog eye leaf spot and rust diseases increased by 1.4, 4.1, 0.5, 5.7, 3.5, 3.9 and 1.5, 4.3, 1.2, 2.1, 4.0, 2.6 percentage points, respectively, compared with similar object detection models, such as Faster R-CNN, SSD, YOLOv5m, YOLOv7, YOLOv8, and YOLOv5s. The improved model can be expected to detect the small target diseases in complex environments of the actual field, particularly for a variety of diseases in apple leaves at the same time. The findings can provide a strong reference for the lightweight detection of apple leaf diseases in real natural environments, especially small-target diseases.