Plant image mosaic based on depth and color dual information feature source from Kinect

Abstract: In agricultural production, image mosaic plays an important role for a panorama view when images are collected by on-board real-time sensor. Most of the agricultural operation is outside, and traditional method of image mosaic based on SIFT (scale-invariant feature transform) algorithm is affected by uneven illumination or wind and some other factors, so image after mosaic presents dislocation or absence of information and other mistakes. It is difficult to meet the requirements of the reliability of agricultural vehicle applications by using image feature element method for image mosaic. Aiming at this problem, the paper proposes a new method of image mosaic based on the color and depth dual information feature source from Kinect. Firstly, depth information and color images are collected by Kinect sensor from 3 different angels. Kinect sensor is fixed on the slider, and when the motion slider moves at a constant speed of 0.5 m/s, the Kinect moves at the same speed. The Kinect is invoked to obtain color image and depth information and save them every 0.6 s through MATLAB. Three consecutive time points are selected in order, and the color images of 3 angles are taken as the mosaic images. Secondly, SIFT algorithm is used to get feature points from color images. SIFT algorithm can extract the feature points which have invariance for illumination, affine and projection transformation. It is helpful to reduce the number of feature points matching and improve the speed and accuracy of feature point matching. Thirdly, feature points matches are gotten by similarity measure, and the Euclidean distance with high efficiency is used as the similarity measure in the 2 pictures in this paper. A key point in an image is gotten, and the closest 2 key points in another image are found. If the ratio of the nearest distance to the next closest distance is less than a certain threshold, then the pair of matching points are received. But some wrong matches exist with this method. Too many mismatches may result in mosaic errors. Therefore, a solution is needed to remove mismatches to improve the accuracy of the matches. From the nature of Kinect, if Kinect moves horizontally, the depth data of a fixed point is the same. Based on this characteristic, some mismatches would be removed. If the depth data of these 2 feature points are the same, the match is retained, and otherwise they would be removed. As a result, the accuracy of matches is improved. Next, by RANSAC (random sample consensus) algorithm, projection transformation matrix can be found. The RANSAC algorithm uses the least possible points to estimate the model and then as far as possible to expand scope of the influence of the model. The projection transformation matrix is more accurate than traditional image mosaic method on account of the removing of mismatches. At last, through the best suture line algorithm, image fusion is relatively smooth. From indoor and outdoor test, the mosaic method based on color and depth dual information feature source has obvious advantages, and it can effectively overcome the influence of light, wind and other environmental factors and avoid mosaic errors such as the loss of image and the difference of brightness. In the indoor test, the mosaic method of this article takes 9.70 s, the accuracy of matches is 92.9%, while traditional method based on SIFT algorithm takes 13.04 s, the accuracy of matches is 88.1%. In the outdoor test, the mosaic method of this article takes 71.15 s, the accuracy of matches is 99.1%, while traditional method based on SIFT algorithm takes 77.67 s, the accuracy of matches is 92.1%. So the mosaic method in this article takes less time than the traditional method based on SIFT algorithm. The data of mosaic accuracy show that the average matching accuracy of the method in this article is 96.0%, and the average accuracy is 5.9% higher than the traditional image mosaic method based on SIFT. So, this method can be further applied in other occasions of image mosaic. It can realize precise spraying of drug fertilizers and the control of pests and diseases based on information collected by Kinect.