Application of image registration technology in apple harvest robot

Abstract: To reduce the effect of natural light, this paper provides a novel apple harvest robot vision system, which integrates a new technique that combines a color-camera system with a PMD-camera. A registration method of color-camera and the PMD-camera is presented to find precise corresponding color pixel information with range distance data from the PMD-camera. The registration algorithm used in the article has the following steps: feature extraction, feature matching, coordinates transformation and interpolation, and feature extraction and feature matching are the key technologies of all. Firstly, as large portions of corners in multi-source images have high correlation, Harris corner detection based on differential operation and autocorrelation matrix was chosen as the method of extracting image features. Secondly, the Normalized Correlation Coefficient (NCC) algorithm was used to realize many-many matching relationships between color images and PMD images, which is a pre-alignment stage. NCC relies on gray information around the corner and has good ability to noise. Thirdly, combined with the information of corners around corresponding points, the refinement stage is completed by way of calculating the support strength of its neighbor points. Finally, the article gets the final registration image after affine transformation and bilinear interpolation. Fifty groups of apple tree pictures that were taken in a natural scene are used to verify the algorithm, including 28 groups of pictures in front lighting and 22 groups of pictures in backlighting. What's more, statistical results of contrast test with different algorithm used in previous articles is obtained, in which each dataset shows minimum, mean, and standard deviation values of matching rate. The experimental result shows that, the algorithm used in this paper is obvious better than previous articles. The matching rate reaches85.75% in front lighting condition and 79.57% in backlighting condition, which can meet the requirements for accurate image registration. As the matching rate doesn't perform as well as expected, error sources is briefly analyzed. A region of interest (ROI) with a PVC pipe frame is suggested to simplify image registration. The registration algorithm lays a foundation for the later work of multi-source image fusion. Moreover, visual perception or feature extraction can be improved by the combination of two images from different cameras of a scene.