Image mosaics reconstruction of canopy organ morphology of apple trees

Abstract: Annual growth cycle of fruit trees is the whole process of life activities with specific laws, including flower thing period and mature period. For a long time, how to construct the three-dimensional shape of an apple tree canopy with color information in different growth stages of fruit tree, has been always a research priority. At present, three methods are usually used to reconstruct 3D shape including stereo vision technology, laser scanner, and three-dimensional digitizer. The stereo vision technology is vulnerable to unstructured outdoor light. The use of a laser scanner can overcome the disadvantages above, but with slow speed to access information. The three-dimensional digitizer requires strict conditions of the external environment, and cannot obtain color information of objects. Photonic Mixer Device (PMD) is a three-dimensional imaging device based on time of flight technology, through which the distance information of objects could be obtained at a speed of 40fps. Although the resolution of the PMD is relatively low, it can be made up by color images. Therefore, the combination of the PMD camera and color camera might be a reliable tool to reconstruct the 3D shape of an apple tree canopy. Two or more inter-public areas of the images can be built into a larger view by image mosaics technology, which has been widely used in many fields, such as computer vision, medicine, and remote sensing, but has not been applied in the canopy organ image mosaics of apple trees in different growth stages. The image mosaics of the canopy are a key to the three-dimensional reconstruction of an apple tree. In order to reconstruct the three-dimensional shape of apple tree canopies in annual growth cycle, the apple tree canopies in the dormant period, the flower thinning period, and the mature period were set for the study, and the color and intensity images were captured by a color camera and a PMD camera based on photonic mixer detector technology, respectively. The images were investigated by mosaics technology following the two steps. First, a scale invariant feature transform (SIFT) algorithm combined with random sample consensus (RANSAC) algorithm was used to establish an image space mapping model which avoided the influences caused by non-structured light and image scale transformation. Secondly, on the basis of what was studied above, the canopy image mosaics were realized through a Laplace pyramid decomposition and reconstruction algorithm, as well as different fusion rules for different frequency bands of pyramid decomposition, which overcome the disadvantages of obvious mosaic trace and bad capacity of reflecting details for fusion images. In order to analyze the quality of the images fused by the algorithm above in the paper, entropy, mutual information, root mean square error, as well as running time were used to evaluate the fusion quality. The test in different orchard environments showed that the algorithm proposed in the paper was suitable for canopy image mosaics in the annual growth cycle of apple trees. The algorithm was robust, real-time, and accurate. The results here had significance for improving information level of orchard management, such as pruning, thinning, yield, and picking.