Navigation path detection method for a banana orchard inspection robot based on binocular vision

Abstract: Banana is one of the most important fruits. A non-motorized vehicle can often be driven to inspect an orchard during the traditional planting management. However, the labor-intensive and time-consuming mode cannot fully meet the large-scale production in modern agriculture. Alternatively, it is very necessary to develop inspection robots with automatic navigation for banana orchards, particularly on the detection of navigation paths in a complex field. In this study, a navigation path detection was proposed to realize the automatic inspection in a banana orchard using binocular vision of mobile robots. The inspection robot also consisted of a binocular camera, a main control unit, and a robot chassis. The specific procedure was as follows. 1) The binocular images were acquired to reconstruct in three dimensions using the ZED camera mounted on the mobile robot. Then, the point cloud was preprocessed, including the coordinate system conversion and space limit. After that, the point cloud was segmented by double thresholds. Specifically, the height of the point cloud was obtained for the banana trees, where the heights of ground points were mainly distributed below 0.5 m, and those of leaves were above 1.5 m. Therefore, the height range of 0.5~1.5 m was selected as the region of interest (ROI) of the point cloud in the navigation path detection. 2) The ROI region was projected onto a two-dimensional plane. The projection of the point cloud was also gridded to reduce the amount of calculation. A traditional K-means clustering was improved, according to the grid’s distribution of banana trees. The initial centers of the cluster were also determined to improve the clustering effect using the vertical and horizontal projection of the grid map, together with the first and the second order Gaussian fitting. Specifically, the grid map was first projected vertically, thereby to be fitted using the second-order Gaussian curve. Among them, the X-axis for the two maximum points of the fitted curve was taken as the X-axis for the two initial centers of the cluster. The grid map was then divided into two areas on the left and right by the center line of two X-axes. After that, a horizontal projection was performed on the two regions, further to be fitted by the first-order Gaussian curve. The maximum points of two curves were selected as the coordinates of the initial cluster centers on the left and right areas. A calculated position was utilized as the initial clustering center. The grids of banana trees on both sides of the road were also divided into two clusters using the K-means clustering. Correspondingly, the clusters were surrounded by the smallest rectangular boxes. The middle line of the two boxes was detected as the navigation path. 3) The path detection was tested in the banana orchards. The robot was manually controlled to perform the inspection operations on the motorized and non-motorized lanes in the banana garden, while the point cloud images in the front were acquired simultaneously. The point cloud images were used to test the path detection. The results showed that the clustering success rate of the improved K-means clustering was 93%, compared with the traditional one of 81%. The positions on the edge of banana trees near the road were also labeled manually. The least square method was then utilized to fit the positions for the road boundary line. The center line of the left and right boundary lines was taken as the desired navigation path. The 93 clustered images were tested successfully by the improved K-means. Consequently, the path extraction demonstrated that the average distance deviation of the navigation path within 2 m was 14.27 cm, and the average angle deviation was 4.83°, compared with the manually labeled path. The findings can provide strong support to the automatic navigation of inspection robots in a banana orchard.