Motion planning method and experiments of tomato bunch harvesting manipulator

Tomato picking environment is characterized by an unstructured space covering most obstacles, such as branches and vines. It is difficult to accurately express in a regular way, particularly for the relatively large volume of tomato bunches. Therefore, the motion planning of the robotic arm in a harvesting manipulator needs to consider how to pick tomato bunches, while avoiding obstacles after cutting the tomato bunches, and finally extracting them from a complex actual environment. Most previous motion planning of tomato picking focused mainly on the obstacle-free moving to the position of picking stalk. But only a few studies reported the specific fruit extraction with an increase in volume after the end-effector of robotic arm gripping the tomato bunch. Taking the tomato bunch picking cultivated in the greenhouse as the research object, real-time motion planning with collision-free Optimal Picking Space (OPS) was proposed here using space segmentation. A reasonable and effective space was also selected for the robotic arm to implement the picking task, in advance to avoid the failure caused by fruit collision or beyond the working range of the manipulator. The specific procedure was as follows. 1) Thousands of color pictures with tomato bunches were first collected. The YOLO-V3 model was used for training to obtain a better recognition network. An RGB-D camera was then used to capture the color and depth information of the environment. The trained YOLO-V3 model was to identify and locate the pixel position of the picking point for the tomato strings in the color map. Next, the internal and external parameters of the camera were cooperated to determine the three-dimensional position of the picking point for a tomato string. An improved density clustering was utilized to focus on the picking area near the picking point of the tomato bunch, while separate the multiple obstacles in the environment. A polynomial function was selected to fit the space curve of branch obstacles falling from top to bottom during the tomato cluster picking in an actual situation. The picking space was divided into multiple sub-spaces, according to the relative positions of branch obstacles and picking points. These sub-spaces, therefore, served as the basis to select the optimal picking space. 2) The volume of each sub-space was calculated to accommodate tomato bunches, while filter out the invalid narrow subspace. Correspondingly, a feasible configuration of the robot arm was achieved in a set of effective subspace, including the unfiltered subspaces, where the invalid subspace outside the working range was filtered out. An evaluation function was also formulated to comprehensively consider the path length and operational space of the robot arm in the joint space. Therefore, an optimal picking subspace was selected from the remaining effective subspaces via the evaluation function. 3) The optimal picking subspace was used as the guidance space of the path plan for the robot arm. Sensing and execution points were then set for the robot arm. Real-time obstacle-free factors were added into the motion planning of the robot arm. As such, the robot arm rapidly switched the modes between obstacle avoidance and attitude adjustment. Therefore, OPS was selected to guide the robotic arm for the real-time, obstacle-free, and non-destructive harvesting of tomato bunches. The optimal picking space greatly contributed to the highly efficient action of a manipulator. The average picking time of single bunch tomatoes using OPS simulation was 12.51s, reduced by 31.23% compared with the current mainstream RRT*-connect, while reduced by 53.23% compared with the Lazy-PRM*, and 19.29% reduction compared with the manual picking. The number of nodes in the OPS path was also reduced, compared with the random expansion node RRT*-connect and Lazy-PRM*. The success rate of motion planning was close to 100%, indicating a high-precision, real-time and rapid intelligent picking.