Abstract:
Cow lameness represents a significant challenge affecting the economic viability of dairy operations.It not only leads to reduced milk production and overall performance but also increases the risk of health issues for affected cows. Consequently, addressing lameness is crucial for maintaining both the welfare of the herd and the profitability of dairy farms.To address this issue, this paper proposes a deep learning-based algorithm for the automated detection of lameness in cows. Lame cows typically show observable indicators during walking, such as a lowered head position, pronounced head movement, and an arched back, whereas healthy cows demonstrate minimal head movement, maintain a straight back, and exhibit normal gait and body equilibrium. Based on these obvious movement characteristics, this investigation aimed to detect cow lameness by tracking the movement patterns of six key anatomical points: the head, neck, shoulder, center of the back, loin, and tail.Firstly, two mobile devices were positioned adjacent to the passage leading to the milking area, and video data of 160 walking sequences from 83 cows were collected. In order to reduce the effects of light variations in the channel, background barbed wire fence boundaries, and foreground fence occlusion,the YOLOv8n-seg instance segmentation algorithm, recognized for balancing computational efficiency and accuracy, was employed to accurately identify cows in the images and extract their coordinates and pixel regions.Secondly,based on the results of instance segmentation six types of keypoint detection datasets were constructed, including RGB images, binary segmentation images, segmentation result images along with their corresponding versions cropped based on the target detection frame. Four backbone networks, MobileNet-V2, ResNet-50, ResNet-101, and ResNet-152, are used to train and test these datasets. After comparative analysis of the keypoint detection effect. Segmented images cropped by the detection frame were selected as the optimal input format, with ResNet-152 chosen as the best-performing backbone network. Then, the DeepLabCut algorithm was used to automatically extract the coordinates of six key points from the video sequences: the head, neck, shoulder, center of the back, waist, and tail, resulting in the creation of a lameness detection dataset.Lastly, a comparative analysis was performed to evaluate the performance of Temporal Convolutional Network (TCN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Bidirectional LSTM, BiLSTM, and FN-BiLSTM models in claudication detection. (Bidirectional LSTM, BiLSTM), and FN-BiLSTM models in lameness detection. In order to gain a deeper understanding of the effect of the models, ablation experiments were conducted on the FN-BiLSTM model, focusing on verifying the effects of the Filter and Noise layers on lameness detection in cows.The results demonstrated that the FN-BiLSTM model achieved optimal performance with 97.16% accuracy, 95.71% precision, and 99.04% recall for lameness recognition on a test set of 16 videos from 16 cows. Moreover, the instance segmentation model exhibited high efficacy in capturing the image sequences of cows and their whole-body semantic information, even under variable illumination conditions and different bovine-to-camera distances. The precision, recall, and mAP of the test set reached 99.97%, 100%, and 99.5%, respectively. During the keypoint detection phase, optimal performance was achieved when utilizing the cropped segmentation maps as input, with ResNet-152 as the backbone network, resulting in root mean square errors of 2.04 pixels and 4.28 pixels for the training and test sets, respectively.These findings suggest that this investigation offers a valuable technical approach for the automated detection of cow lameness in the livestock industry. This has the potential to enhance the efficiency and animal welfare of dairy operations, thereby promoting the sustainable development of the livestock sector.