Field growth parameter estimation system of winter wheat using RGB digital images and deep learning

Abstract: Leaf Area Index (LAI) and Above Ground Biomass (AGB) are the key traits to fully reflect the growth of winter wheat at early stages. After winter wheat enters the flowering stage, the development of wheat ears has been basically completed. The number of ears is a vital agronomic parameter to characterize the growth of winter wheat at this stage, and also a critical factor to evaluate the germplasm and yield of winter wheat. At present, the traditional methods for measuring LAI, AGB and ear number are destructive and time-consuming, and the accuracy of estimation methods based on RGB images and shallow machine learning technology needs to be further improved. In order to accurately and quickly obtain the winter wheat growth information, and further improve the accuracy of winter wheat growth parameter estimation, this study developed a winter wheat growth parameter estimation system based on RGB images and deep learning. The system mainly included growth parameter estimation module and wheat ear counting module. The data of winter wheat during the 2017-2018 and 2018-2019 growing seasons were collected consecutively. Combined with the characteristics of RGB images, the deep learning models were explored, which were applicable to obtain the growth parameters at the early stage of winter wheat and to count the number of wheat ears. Therefore, for the estimation module of growth parameter at early stages, the residual network ResNet18 was used as the basic network to establish the growth parameter estimation model, with the 2017-2018 growth season data. Based on this estimation model, the LAI and AGB at early stages of winter wheat were obtained. Moreover, the generalization ability of the ResNet18-based model was tested using transfer learning, with 2018-2019 growth season data. For wheat ear counting module, a wheat ear counting model was built, which was based on the Faster R-CNN and Non Maximum Suppression (NMS), and achieved accurate counting of wheat ear at flowering stage. Moreover, the Faster R-CNN+NMS wheat ear counting model was compared with the Faster R-CNN ear counting model without NMS and the classification counting model based on Convolutional Neural Networks (CNNs). The results showed that, for the estimation module of growth parameter at early stages, the determination coefficients of the ResNet18-based model for LAI estimation respectively were 0.83 and 0.80 on the dataset of the two growing seasons of 2017-2018 and 2018-2019. And the determination coefficients of the ResNet18-based model for AGB estimation both were 0.84. The model was superior to the model based on VGG16 and GoogLeNet and the published CNN-based estimation model. And the results of generalization ability test showed that the ResNet18-based model was robust to the seasonal differences of data. For wheat ear counting module, given the ear counting model based on Faster R-CNN, the determination coefficient increased by 25.8% from 0.66 to 0.83, after the NMS optimization. And the NRMSE decreased by 73.7% from 0.19 to 0.05. Compared with the classification counting model based on CNN, the wheat ear counting model based on Faster R-CNN+NMS had better performance, with a determination coefficient of 0.83, improved by 33.9%, and a single ear identified time of 1.009 s, improved by 20.7%. In conclusion, this system can meet the demand of field growth parameter estimation of winter wheat and provide support for fine field management of winter wheat.