Abstract:
Abstract: Undamaged picking is one of difficulties in automatic harvesting robots. Since grape is a cluster growing fruit and its pericarp and sarcocarp are weak, so grape is easy to be collided and damaged by the manipulator and end-effector when they approach to pick a candidate grape. To plan a collision-free path, a calculation and localization method for bounding volume of grape based on binocular stereo vision was presented. The vision system was consisted of two MV-VD120SC color cameras and the baseline distance of two cameras was 50 mm. Firstly, the binocular stereo vision system was calibrated by using a calibration plate ordered from MVTec Software GmbH (Germany), and subsequently the images captured by two cameras were rectified. Secondly, the grape cluster region was acquired by segmenting the rectified left image using an adaptive threshold method based on H-I color component. The exterior rectangle and barycenter of the region were extracted. The region of interest of peduncle was determined according to those extracted geometric information, and subsequently the picking point on peduncle was calculated out by combining the Hough line detection and the minimum distance restraint between barycenter and the detected line. The center and radius of grape berries were acquired using circle regression within the exterior rectangle. To accelerate circle regression and enhance the accuracy of berries recognition, an adaptive predication model of the berry radius in images captured at various distances was built through power multiplication method, and two rules were built to eliminate these redundant circles that were produced by circle regression. Thirdly, the disparity of the picking point and the center of berries between left and right images were calculated by stereo matching method based on similarity function of normalized correlation coefficient, and subsequently three-dimensional coordinates of picking point and center of berries were extracted by using the triangulation principle. Three-dimensional virtual grape cluster was rendered by using OpenGL functions. Fourthly, the coordinate system XYZ of grape cluster was built, which defined the picking point as origin and took vertical downward Y axis as grape of center axis. The distance L between origin and each detected berry under XZ plane was calculated, the Y-L coordinate system that was used to solve the maximum section of grape cluster was built, and the maximum contour section of grape cluster was obtained through seeking convex polygon, the bounding volume of grape acquired by rotating the section 360 degree around the center axis. Finally, to validate the performance of the proposed method, thirty pairs of images were captured at various distances within 600-1 100 mm using binocular stereo vision system in the Tianjin Chadian Grapes Science Park. The localization and size accuracy of grape bounding volume and the elapsed time of algorithm were tested. The test results indicated that the localization error of grape bounding volume was less than 5 mm, the relative error of height and maximum diameter was less than 4.95% and 5.64%, respectively, and the elapsed time of algorithm was less than 0.69s when the depth was less than 1 000 mm, which showed that the proposed method can prevent grape from damaging during grape picking. However, the localization accuracy was not satisfactory when multiple grapes were overlapping each other or the occlusion was serious, so further studies should focus on the localization of multiple overlapping grapes and occluded grapes.