Abstract:
Abstract: Accurate and high-throughput maize plant phenotyping is vital for crop breeding and cultivation research. However, fully automatic and fine stem-leaf segmentation of maize shoots from three-dimensional point clouds is still challenging, especially for the newly emerging leaves that are close to each other during the seedling stage. To address this issue, this study proposed an automatic segmentation method consisting of six main steps, including skeleton extraction, skeleton decomposition, point cloud coordinate system transformation, coarse segmentation, fine segmentation, and phenotypic parameter extraction. The Laplacian-based skeleton extraction algorithm was used to extract the maize plant skeleton in the skeleton extraction step. In the process of skeleton decomposition, the plant skeleton was decomposed into a stem sub-skeleton and some leaf sub-skeletons based on the morphological characteristics of leaves. A plant spatial coordinate system was constructed. The Z-axis of this coordinate system coincided with the stem axis, which could be better to represent the morphology of maize plants. The plant point cloud and skeleton vertices were transformed from the original coordinate system to the plant spatial coordinate system, and the subsequent point cloud segmentation steps were carried out in the plant spatial coordinate system. In the coarse segmentation step, using the decomposed organ sub-skeletons, the plant point cloud was roughly segmented into stem and leaf instances. However, the stem instance contained some wrongly segmented points, and these points should belong to the leaf instances. In the fine segmentation step, firstly, the wrong points were identified and removed from the stem instance, and then the stem-leaf classification algorithm based on the optimal transportation distance was used to segment these points into the correct organ instance. The classification algorithm used top-down order to classify points based on optimal transportation distances, which was equivalent to adopting a segmentation strategy from the leaf tip to the stem. The top-down order was critical as it facilitated the complete utilization of the segmented point cloud information in the new leaf while classifying the points. At the same time, it also ensured that the points belonging to the new leaves were determined first. Compared with the Euclidean distance, the optimal transportation distances described the local geometric characteristics of the point cloud more accurately, which helped to deal with the case of new leaves wrapping each other. Based on the segmented organ instances, six phenotypic parameters were accurately and automatically measured, including plant height, crown diameter, stem height and diameter, leaf width, and length. The segmentation method was tested on 30 maize plants and compared with manually obtained ground truth. Average precision, average recall rates, average micro F1-scores, and average overall accuracy of the stem-leaf segmentation algorithm were 0.967, 0.961, 0.964, and 0.967, respectively. To compare the phenotypic parameters obtained by the proposed method in this study and those obtained by manual measurement, the regression analysis was done with the root mean square error. Results showed that the determination coefficients of plant height, crown diameter, stem height, stem diameter, leaf width and leaf length were 0.99, 0.99, 0.96,0.97, 0.93, and 0.96, respectively; the root mean square errors were 1.71, 3.44, 7.07, 0.41, 0.85, and 5.28 cm, respectively. The results indicated that the proposed algorithm could automatically and precisely segment not only the fully expanded leaves but also the new leaves wrapped together and closed together. The proposed approach might play an important role in further maize research and applications, such as genotype-to-phenotype study and geometric reconstruction.