基于三维点云的群株生菜分割模型与鲜质量预测方法

    Segmenting collective lettuce to predict fresh weight using three-dimensional collective

    • 摘要: 密集场景下群株生菜的有效分割与参数获取是植物工厂生长监测中的关键环节。针对群株生菜中个体生菜鲜质量提取问题,该研究提出一种利用实例分割模型提取个体生菜点云,再以深度学习点云算法预测个体鲜质量的方法。该方法以群株生菜为研究对象,利用深度相机采集群株生菜俯视点云,将预处理后的点云数据输入实例分割模型Mask3D中训练,实现背景与生菜个体的实例分割。之后使用鲜质量预测网络预测个体生菜鲜质量。试验结果表明,该模型实现了个体生菜点云的分割提取,无多检和漏检的情况。当交并比(intersection over union,IoU)阈值为0.75时,群株生菜点云实例分割的精确度为92.4%,高于其他实例分割模型;鲜质量预测网络实现了直接通过深度学习处理点云数据,预测个体生菜鲜质量的目的,预测结果的决定系数R2值为0.90,均方根误差值为12.42 g,优于从点云中提取特征量,再回归预测鲜质量的传统方法。表明该研究预测生菜鲜质量的精度较高,研究结果为利用俯视单面点云提取群株生菜中个体生菜表型参数提供了一种新思路。

       

      Abstract: Acquiring the individual parameters of collective lettuce under dense scenarios can greatly contribute to environmental regulation, yield prediction, and harvest timing determination in the growth monitoring center of the plant factory. Traditional monitoring can often involve the manual measurement of geometric parameters and root removal for fresh weight determination, leading to the less comprehensive and efficient. Fortunately, non-destructive monitoring can be expected to extract the crop phenotypic parameters using machine vision and machine learning at present. However, most existing machine learning exhibited certain limitations in the application of lettuce point clouds. For instance, the majority of application scenarios have been focused on the organ segmentation of individual crops. It is still lacking in the individual segmentation of collective plants. Additionally, the extraction of crop phenotypes from point clouds can often rely mainly on manually predefined feature quantities. There is a high demand to fully explore the effective phenotypic information within the lettuce point clouds. In this study, instance segmentation was proposed to process the point cloud data of collective crops. Subsequently, deep learning of point clouds was employed to predict the fresh weight of individual crops. The collective lettuce was also taken as the research object. A depth camera was also utilized to collect the single-plane point clouds of the collective lettuce. After point cloud preprocessing, the data was then input into the instance segmentation model (Mask3D) for training. A feature backbone network was employed to extract the features from the point clouds of the collective lettuce. A Transformer decoder was utilized to process the instance queries. Point cloud features were integrated with the instance queries through the mask module. A mask was then generated for each instance. The background and lettuce point clouds were segmented to distinguish the individual lettuce. Finally, the fresh weight prediction (FWP) network was employed to predict the fresh weight of individually segmented lettuce. The feature extraction network (PointNet) was utilized to extract the features from the segmented point clouds of individual lettuce. A multilayer perceptron was also employed to regressively predict the fresh weight of the lettuce. Experimental results indicated that the segmentation and extraction of individual lettuce point clouds were successfully achieved without over- or under-detection on the point cloud dataset. When the Intersection over Union (IoU) threshold was set to 0.75, the average precision (Ap) of instance segmentation was 92.4% for collective lettuce point clouds, superior to the instance segmentation models, such as Jsnet. Furthermore, the direct prediction of lettuce fresh weight reduced the errors associated with the manual feature extraction during processing using deep learning point cloud. The coefficient of determination (R²) and the root mean squared error (RMSE) were 0.90 and 12.42 g, respectively, indicating the superior accuracy of the fresh weight prediction network. Traditional and manual feature extraction from point cloud parameters with machine learning was achieved in the maximum R² of 0.83 and the minimum RMSE of 15.06 g. Therefore, the deep learning instance segmentation and point cloud regression can be expected to estimate the fresh weight of collective lettuce, indicating exceptional better performance. The finding can also provide significant importance for the growth monitoring, yield estimation, and harvest timing determination of facility vegetables.

       

    /

    返回文章
    返回