Automatic rumen filling scoring method for dairy cows based on SOLOv2 and cavity feature of point cloud

Abstract: Rumen Filling Score (RFS) has been an intuitive indicator to reflect the individual and energy intake status of dairy cows. However, the manual scoring can be confined to subjectivity and repeatability, leading to a core bottleneck for the popularization and application of this technology in feed intake monitoring. There is no report on the automatic rumen scoring system so far. Therefore, this study aims to realize the automatic scoring of rumen filling degree for the dairy cows, further improving the monitoring accuracy of individual feed intake and the management level of pasture feeding. A three-dimensional structural feature map was constructed using the imaging of rumen point cloud, according to the instances segmentation network and the gradual characteristics of rumen shape over time. A classification model was then established to implement the automation and objectivity of scoring operation. Firstly, the background difference method was used to preprocess the acquired images of cow back depth, thereby extracting the target cow. Secondly, the SOLOv2 instance segmentation network was used to segment the rumen area of the cow image after the rotation correction, according to the inclination of the spine point fitting line, where the rumen point cloud was obtained, according to the color difference of the ruminal region before and after segmentation. Then, the rumen point cloud was projected to the X-Z plane, in order to obtain the cavity feature map for the rumen filling degree of dairy cows. Specifically, the two-dimensional convex hull was utilized to define the cavity range using some operations, such as binarization, dilation corrosion, and the maximum connected domain solution. The nearest neighbor interpolation was also used to unify the image size of the rumen point cloud for better cavity comparability. Thirdly, a two-level classification model was constructed using the tilt angle detection and LeNet-5 network. The feature images were then classified to balance the sample distribution and the accuracy of the model. Finally, the model was trained and tested using 1784 depth images of 75 cows, further to compare the backbone networks, segmentation models, and scoring recognition. The results showed that the average precision of the SOLOv2 model for the image segmentation of the test set increased by 1.7%, and 86.29% with the backbone network of ResNet-101-FPN, compared with ResNet-50-FPN. The proportions of images with the rumen filling score error within 0 and 1 were 85.77% and 99.90%, respectively, and the average F1 values were 86.85% and 99.9%, respectively, where the average recognition rate was 5.5 FPS in the two-level classification. Compared with the combination of BlendMask, Yolact++, MaskRcnn segmentation models and two-level classification model, the identification index values of the two-level models are all optimal. And compared with the above three segmentation networks, the AP value of SOLOv2 network segmentation was increased by 1.3, 3.52 and 2.01 percentage points respectively. Therefore, the recognition accuracy of two-stage scoring with the SOLOv2 and classification model was improved by 16.95 percentage points, where the precision, recall, and F1 values all reached more than 84.37%, indicating much higher than the index value of the single-step method. The average accuracy of RFS recognition under 9 BCS values reached 89.7%, and the correlation coefficient between BCS and RFS was only 0.0034, indicating the strong robustness to the changes of cow body conditions and individual differences. Therefore, the non-contact evaluation can be realized for the ruminal filling degree of dairy cattle in large-scale farming, indicating the high precision, strong applicability, and low cost. The automatic approach can also be widely expected to improve the accuracy of individual feed intake monitoring of dairy cattle.