Three-dimensional reconstruction and its precision evaluation of plant architecture based on multiple view stereo method

Abstract:Plant architecture is an important determinant of the canopy light interception and photosynthesis. Therefore, effective and nondestructive methods for obtaining plant architecture can help us understand the relationships between plant physiological processes and morphogenesis. Digital camera technology has become relatively ubiquitous and inexpensive, leading to a recent surge in utilizing plant imaging to capture data. Therefore, a three-dimensional (3-D) reconstruction of plant architecture based on these photographed image sequences can be realized. However, the accuracy evaluation of reconstruction is always determined from visual effect and or from 1-D or 2-D measured data. In this study, image sequences were obtained around experimental plants (e.g. egg plant, sweet pepper and cucumber) by slightly moving a commercial camera for image generation so that each neighboring image pair shared short baseline. Structure from motion (SFM) method was applied to produce a set of sparse point cloud based on plant image sequences. As the sparse point cloud was inadequate for the reconstruction of complicated plant architecture, multiple-view stereo (MVS) method was further used to produce dense and accurate point cloud based on the output of SFM. Software Bundler and CMVS were applied to implement the SFM and MVS methods, respectively. Bundler takes a set of images as input, and produces a 3D reconstruction of sparse scene geometry and camera parameters as output. CMVS takes a set of images and camera parameters (the output of Bundler) as input, and outputs a set of dense points with geometry details. Once original point cloud has been obtained, point cloud processing procedures were conducted to refine point cloud, including deleting noise points and scaling to the actual size of experimental plants. In order to obtain point cloud of individual blade, segmentation of point cloud of plants was conducted based on region growing segmentation algorithm. Once point cloud of individual blade was obtained, Poisson surface reconstruction algorithm was used to reconstruct each leaf. Before accuracy evaluation, point cloud of individual leaf blade and corresponding laser scanning point cloud were aligned to the same 3-D coordinate system using the Iterative Closest Point algorithm. Then, comparison on 3-D scale based on Hausdorff distance was made between point cloud data obtained from plant image sequences and referenced point cloud data with laser scanning on individual leaf blade level. Furthermore, phenotypic attributes, such as leaf blade width, leaf blade length and blade area were extracted based on data from image sequences and from laser scanning methods. Besides, these attributes of each blade were manually measured. Finally, comparisons were made for blade area, blade length and blade maximum width between data from the image sequence-based model, laser scanning based model and manually measured data. The results showed that high accuracy of 3-D reconstruction was obtained based on plant image sequence method. Hausdorff distances of experimental plants were ranged from 0 to 10 mm, and most of the values were less than 4.0 mm. There was a good agreement between measured and calculated blade area, blade length and maximum width with R2 > 0.95 for blade area, RMSE < 4.5 mm for blade width, and RMSE < 5.6 mm for blade length. There was no significant difference for each attribute between measured and calculated data (ANOVA, P > 0.05). A key advance of the current 3-D reconstruction of plant architecture is the capability to non-destructively capture plant traits with high accuracy. This advance permits time-series measurements that are necessary to follow the progression of growth and stress on individual plants, and will play an important role in related research fields, such as plant phenotyping, genetic breeding, interactions between plant phenotype and environment.