Acquisition and experiment on location information of picking point of tomato fruit clusters

Abstract: Automatic harvesting of fruits and vegetables is a key process to realize the automation and intellectualization of agricultural production, which can reduce production costs and improve the production efficiency. Tomato is one of the most fruits and vegetables planted in China. The researches on tomato harvesting robots are focused on the single fruit harvesting. The end-effector of the harvesting robot touches the tomato fruit directly and picks one tomato once during harvesting, so the tomato fruits are easy to be damaged and the harvesting efficiency is very low. To overcome the above shortages of the single fruit harvesting, the method of the tomato cluster picking was presented, and the determination method of picking point was developed based on the corner detection on the tomato stem skeleton for the binocular vision system of the tomato cluster picking robot in the paper. The test platform of the vision system was set up and the experiments on the determination of the picking points have been conducted. In the paper, the color images of the tomato clusters were collected in greenhouse in the agricultural production base. The target tomato cluster was segmented from background by OTSU method according to the color features of the tomato clusters and denoised by the morphologic method and threshold method. The centroid of tomato fruit cluster was obtained by enclosing rectangle method. Furthermore, the rectangle region of interest of the target stem was recognized according to constraint relationship between the fruit cluster and the fruit stem, which included the whole stem of the target tomato cluster. Converting the RGB (red, green, blue) color channel map to the HSV (hue, saturation, value) channel map, there are different grayscales between the stem and the background. The target stem of the tomato cluster was separated from background by using the gray difference of H channel and denoised by morphologic method, median filtering method and extracting the maximum connected region. To avoid the effects of the irregular stem on the picking point determination, we abstracted the stem skeletons by Zhang's thinning algorithm to simplify the stem original shape and extract the kernel information of the stem, and then, the Harris algorithm was applied to extract the corner points of the skeleton, and the corner points were sorted from large to small according to the vertical axis coordinates. The picking feature points on the target stem were obtained by averaging pixel coordinates of the 5 corner points. Sixty groups of experiments on the determination of picking points of the tomato cluster were carried out on the binocular vision test platform. The results show that the success rate of position information obtaining of the picking point on the stem is 90%. Comparing X and Y coordinates data of the picking point and centroid of the tomato fruit clusters, the X coordinates of the picking feature points vary in small range, while Y coordinates vary within wide limits, which results from the main stem growing in longitudinal direction. The mean absolute value of the pixel deviation of the Y coordinates is 21. The research provides the basis for locating the picking point on the stem for the tomato fruit cluster picking robots.