Real-time classification and detection of citrus based on improved single short multibox detecter

Abstract: Manually classifying citrus based on its surface defects is tedious and time-consuming and a new real-time method is proposed in this paper based on the improved SSD deep learning model. In the testing bench of the waxing machine, 2 500 images of a variety of citrus species were taken, of which 2 000 were randomly selected as training set and 500 as testing set. Among them, the method classified 19 507 as normal, 9 097 skin defects and 4 327 mechanically damaged. Considering that traditional methods using near-infrared spectra, support vector machines, HSV and RGB color space model are inefficient to detect surface defects of citrus and can only identify one, we proposed an improved method to calculate the image using the one-stage detection model - SSD-ResNet18. The method gets the feature maps through backbone first, and then predicts the number of boundary boxes from the feature maps before determining the location and category of citrus using confidence and non-maximum suppression. This can detect a batch of citrus. In the proposed method, we used the mAP (mean average precision) as the precision index and the mean detection time as the speed index. Optimization in the proposed method was solved using the SGD (stochastic gradient descent) algorithm. The learning scheduler was based on cosine decay, enabling the learning rate to drop to 0 at the end of the training period. This ensures the lost value during the training period to continuously decline. As the model was stable at the end of the training period, it can be saved at the end of the training for further use. While the VGG16 was used as the original SSD backbone, it needs a multitude of parameters and is hence computationally inefficient. We replaced it with the ResNet18, which is approximately 100 times more efficient than the VGG16. An improved feature map was obtained from the analysis of the effective sensory field of different feature maps and the size of citrus in the map, the anchor in which was obtained using the K-means clustering algorithm from the manual label box. The suitable image resolution for the proposed model was obtained by comparing images taken at five resolutions: 512×512 pixels, 640×640 pixels, 768×768 pixels, 896×896 pixels and 1024×1024 pixels. The results showed that the accuracy of the mAP of SSD-ResNet18 was 87.89%, improving 0.34 percentage points higher than the original SSD. The average detecting time of the SSD-ResNet18 was 20.72 ms, reduced by 436.90% compared to the original SSD's 108.83 ms. The accuracy of the AP of SSD-ResNet18 was 94.72%, 85.79% and 83.17%, respectively, for detecting normal, skin lesion and mechanical damage. We compared MobileNetV3, ESPNetV2, VoVNet39 and ResNet18 as backbones and did not find significant difference between their accuracy, but ResNet18 was 10.52 ms, 16.78 ms and 36.76 ms less than MobileNetV3, ESPNetV2 and VoVNet39 in detection time, respectively. The method proposed in the paper meets the requirement on detecting speed in real-time citrus production line and can effectively classify and detect a multitude of citrus simultaneously.