Navigation path recognition between crop ridges based on semantic segmentation

Abstract: A navigation path has been widely considered as one of the most important sub-tasks of intelligent agricultural equipment in field operations. However, there are still some challenges remaining on the recognition of current navigation paths between crop ridges, including the accuracy, real-time performance, generalization, and difficulty in the interpretation of deep learning models. In this research, a new Fast-Unet model was proposed to accurately and rapidly recognize the navigation path between crop ridges using semantic segmentation. The jump connection of the Unet model was also retained to generate the navigation line and yaw angle using the least square regression. Specifically, a cotton dataset of inter-ridge navigation path consisted of 800 images, 640 of which was set as the training set, 160 of that as the validation set. Subsequently, two datasets of 100 images each were constructed for the navigation paths of sugarcane and cotton ridges, which were divided into 50 images in the training set, and 50 images in the verification set. The training strategy was selected as the data augmentation and learning rate adjustment. The training order was ranked as the corn first, and then the sugarcane dataset. The Mean Intersection over Union (MIoU) was utilized as the accuracy indicator of the Fast-Unet model, which was 0.791 for cotton, 0.881 for maize, and 0.940 for sugarcane. Furthermore, the least-squares regression was selected to calculate the navigation path of maize and sugarcane with good linearity between the ridges. Additionally, the navigation line was selected to further calculate the yaw angle. The mean difference between the predicted yaw angle of maize and sugarcane navigation path and the labeled were 0.999° and 0.376° under the Fast-Unet model, respectively. In terms of real-time performance, the inference speed of the Fast-Unet model was 6.48 times higher than that of Unet. The inference speed was 64.67 frames per second to process the RGB image data on a single-core CPU, while the number of parameters of the Fast-Unet model was 6.24% of that of Unet model. Correspondingly, the computing devices were deployed with weak computing power, thereby performing real-time calculations. A gradient weighted class activation mapping(Grad-CAM) was also used to visually represent the final feature extraction of model recognition and transfer learning. More importantly, the special features were highlighted on the navigation path between crop ridges in the optimized Fast-Unet structure, concurrently to remove a large number of redundant feature maps, while retaining only the most crucial feature extractors. The transfer learning also presented a larger activation area than the direct training, where the activated area matched the main road to be identified. In summary, the improved model can be fully realized the real-time recognition of maize navigation path. The finding can also provide technical and theoretical support to the development of navigation equipment for intelligent agricultural machinery in the field.