Pesticide spraying route planning algorithm for grapery based on Kinect video technique

Abstract: This paper performs a precise spraying algorithm and builds an experimental platform including hardware of spraying equipment and software of spraying route planning algorithm. With the color images and depth images captured by Microsoft Kinect, it proposes leaf wall area (LWA) average distance calculation method based on spline regions and spraying route planning method based on route deviation for precise spraying of grapery. The purpose of this algorithm is to optimize the spraying route for keeping the suitable distance between LWA and sprinkling nozzle, and reduce the waste or pollution of the pesticides. The algorithm consists of 2 modules including LWA segmentation and distance estimation module, and optimal spraying route planning module. Firstly, to detect the distance between the platform and LWA, the morphological method is adopted to segment the color video frames captured by Kinect for dividing the LWA region accurately. The binary images were created by green layer minus red layer in color images. Then, closing and opening operation were performed for filling and de-noising. And to get continuous LWA, 300 pixels filling algorithm was used to fill the hollows or gaps in LWA. Secondly, a method of calculating LWA average distance based on spline regions is proposed combined with the information of depth image, which is used to accurately estimate the distance between spraying equipment and LWA precisely. The average distance is more reliable than single point distance because there are usually some hollows and gaps in LWA, which will lead to big errors when fetching the distance data from depth image. Finally, a route deviation and correction planning method is presented to guide and keep the spraying system in the optimal route. The errors that result from the spraying system drifting off the route center were computed by the LWA average distance to adjust the spraying arm into optimal spraying distance. And the corrective navigation lines were calculated by these errors and drawn in the video to plan the best spraying route. In the hardware of the precise spray experimental platform, the self-propelled spraying equipment with adjustable sprayer arms was designed and implemented. The self-propelled spraying platform included an air assisted sprayer and 2 moving sprayer arms with 4 nozzles, a Kinect at the head of platform, a laptop PC (personal computer) for planning the spraying route, and a power supply. To test the effectiveness of this algorithm, the experimental data were collected at 30 positions randomly within a big grapery in the North China. The experimental results showed that the differences and variance of the spray distance and route deviation between calculated values and measured values were small. The mean square errors of spraying distance were 3.7 and 1.9 cm respectively at the left and right of spraying system. And the variance of route deviation was 2.29 cm, which is also small. According to the adjusting range of sprayer arms, it will not affect the results of estimating best spraying distance and route. Therefore, it is proved that the results of this algorithm are accurate, and this spraying system can accurately estimate the spraying distance and plan the optimal route to spray precisely in grapery. Moreover, the processing time of the algorithm was also tested, which is 0.574 9 s per frame without program optimization. Hence improving the speed and realizing the real-time system is one of the important parts in our further work