Point cloud skeleton extraction of maize leaves based on adaptive weighting operator and principal curve

Abstract: Three-dimensional (3D) skeleton is one of the most important representations of plant leaf morphology. Accurate extraction of leaf skeleton from 3D point clouds has been an essential way to obtain 3D plant phenotypes in recent years. In this study, a novel adaptive weighting operator was presented to calculate the point set of skeleton constraints from the 3D point clouds of maize leaf, particularly considering the leaf shape and data completeness. A principal curve was also selected to fit the skeleton constraint point set for a better skeleton. As such, it was likely to accurately extract the skeleton features from the point clouds of maize leaf, especially for the missing point cloud data. The extraction was composed of four steps. 1) An input point cloud of maize leaf was generally divided into 3 or 4 parts by the classical K-means clustering for the normal of the input point cloud, where the shape of each part changed slowly. 2) Each part of the point cloud was then reordered and segmented into many cross sections. A distance field was estimated to realize the segmentation, where the distance field was calculated by the distance from each point to the orthogonal plane perpendicular to the direction of leaf elongation. Such a distance field was also more efficient and accurate than the Euclidean distance. 3) A novel adaptive weighting operator was applied to the related point set of each cross section, in order to extract the skeleton constraint points. The adaptive weighting operator included the weight of spatial distance, normal difference, and point cloud completeness. Specifically, the weight of spatial distance was used to quantitatively describe the sparsity of the point cloud. The weight of normal difference was used to rearrange the spatial positions of constraint points, according to the difference between the normal of the point and the orientation of the leaf. The weight of point cloud completeness was to measure the missing level of leaf points. 4) A skeleton of the input point cloud was obtained to compute the principal curve of the skeleton constraint point set using a K-segment. It was found that a smooth curve was passing through the middle of the skeleton constraint point set. The final skeleton was generated to extend at both ends of the curve, where the input point cloud was used to eliminate the shrinkage during principle curve fitting. In addition, 30 point clouds of maize leaf with the typical shape characteristics were selected to evaluate the performance of the extraction. Experiment results show that the extracted skeletons fully represented 3D structural features of maize leaves, indicating better performance than that of the commonly-used Laplace-based skeleton extraction. Correspondingly, the extracted skeletons were also used to conduct the phenotype measurements, such as the estimation of the leaf length of all the leaves. Three statistical errors were selected to evaluate the leaf length between the calculated and the measured, including the mean absolute percent error (MAPE), root mean square error (RMSE), and normalized root mean square error (NRMSE). It was found that the MAPE was 2.10%, RMSE was 2.21 cm, and NRMSE was 2.89% in the new extraction, whereas, the MAPE was 6.16%, RMSE was 6.31 cm, and NRMSE was 8.26% in the previous Laplace-based approach. More importantly, the weight of point cloud completeness was dominated to extract the skeletons from incomplete point clouds. By contrast, there was an outstanding deviation in the extracted skeletons from the incomplete leaves without using the completeness weight. The new extraction has also been a fully automatic processing on the leaf point clouds without any human interactions. Thus, the new extraction can be widely expected to apply for the big data of point clouds in plant phenomics. The finding can provide promising technical support to the high-throughput and fully automatic phenotyping evaluations in precision agriculture.