Ear segmentation and phenotypic trait extraction of maize based on three-dimensional point cloud skeleton

Abstract: Accurate and high-throughput maize plant phenotyping is vital for crop breeding and cultivation research. Ear related phenotypic parameters are important agronomic traits. However, fully automatic and fine ear organ segmentation of maize shoots from three-dimensional (3D) point clouds is still challenging. To address this issue, a skeleton-based maize plant organ segmentation process was used to segment the ear organs of the plant and extract phenotypic parameters. Firstly, the Laplace based skeleton extraction algorithm was utilized to generate plant skeleton. In this study, breadth first search method was used to obtain all the connected branches of the plant skeleton. The connected branches with the largest number of vertices were retained as the plant skeleton, while the small skeletons formed by the other connected branches were deleted to ensure that the plant skeleton is a connected undirected graph. Secondly, the plant skeleton was decomposed into several organ sub skeletons using Dijkstra algorithm, and then the organ sub skeletons were divided into stem sub skeletons and non-stem sub skeletons according to the angle features of sub skeletons and point cloud cylinder features. After that, the organ sub skeletons were used to obtain the seed points of each organ, and then the unsegmented points were classified in the order from the top to the bottom of the plant in turn, to get the final organ segmentation results. Four constraints (organ height constraint, sub-skeleton length constraint, cylindrical feature constraint, and the point cloud number constraint) were used to identify ear organs from all organ instances. Four constraints (organ height constraint, sub-skeleton length constraint, cylindrical feature constraint, and the point cloud number constraint) were used to identify ear organs from all organ instances. Four phenotypic traits, ear height, ear length, ear diameter and the ratio of plant height to ear height, were extracted using the ear organ instance. The segmentation method was tested on 15 maize plants. This study took about 24 seconds to process the maize plant with 10 000 point clouds. The result showed that the proposed method had a strong ability of ear recognition. The ear recognition accuracy was 91.3%. The average F1 score, average precision, and average recall of the all the ear organs were 0.73, 0.82, and 0.70 respectively. Furthermore, to compare with the phenotypic parameters obtained by the proposed method in this paper and those obtained by manual measurement, the regression analysis was done and the results showed that the determination coefficients of ear height, ear length, ear diameter and the ratio of plant height to ear height, were 0.97, 0.78, 0.85, and 0.96, respectively, the root mean square error were 3.23, 4.98, 0.73 cm, and 0.07, respectively. There were also some problems in this method. First of all, if the distance between the ear tip point cloud and the other organ point cloud was too close, the ear skeleton might fail to be extracted, resulting in the ear could not be segmented, which often occurred in the second ear with a smaller volume. Secondly, the ability of the segmentation method to identify the boundary between the ear and other organs needs to be improved, which would lead to false segmentation of the ear point cloud, and more probability of under segmentation. The proposed algorithm cloud extract the point cloud and phenotypic parameters of ear organs. As far as we know, this was the first method to obtain this effect. The proposed approach might play an important role in further maize research and applications, such as genotype-to-phenotype study, geometric reconstruction, and dynamic growth monitoring.