张智荣,褚燕华,王月明,等. 基于头颈背部关键点的奶牛跛行检测[J]. 农业工程学报,2024,40(21):1-8. DOI: 10.11975/j.issn.1002-6819.202406078
    引用本文: 张智荣,褚燕华,王月明,等. 基于头颈背部关键点的奶牛跛行检测[J]. 农业工程学报,2024,40(21):1-8. DOI: 10.11975/j.issn.1002-6819.202406078
    ZHANG Zhirong, CHU Yanhua, WANG Yueming, et al. Cow lameness detection based on head, neck, and back key points[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2024, 40(21): 1-8. DOI: 10.11975/j.issn.1002-6819.202406078
    Citation: ZHANG Zhirong, CHU Yanhua, WANG Yueming, et al. Cow lameness detection based on head, neck, and back key points[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2024, 40(21): 1-8. DOI: 10.11975/j.issn.1002-6819.202406078

    基于头颈背部关键点的奶牛跛行检测

    Cow lameness detection based on head, neck, and back key points

    • 摘要: 为实现奶牛跛行的自动检测,该研究鉴于跛行奶牛行走时,头部起伏较大且背部弓起的特点,提出了一种基于深度学习的奶牛头、颈、背部6个关键点的跛行检测算法。首先,在通道旁固定摄像头,采集奶牛行走的视频数据,利用YOLOv8实例分割算法将奶牛从图像中识别出来;其次,使用DeepLabCut算法提取视频帧序列中奶牛的头、颈、肩、背部中心、腰和尾部6个关键点坐标,对比分析实例分割得到的6种格式输入图像在MobileNet-V2和ResNet系列主干网络上的训练效果后,最终选择按目标检测框裁剪后的分割结果图和ResNet-152作为DeepLabCut算法的最佳输入和最优主干网络;最后,对比分析4种时间序列模型和FN-BiLSTM模型在奶牛跛行检测中的表现。试验结果表明,FN-BiLSTM算法的性能最优,在包含16头奶牛16段视频的测试集上跛行识别的准确率达到了97.16%。研究表明,该算法可为养殖场奶牛跛行检测提供技术支持。

       

      Abstract: Cow lameness represents a significant challenge affecting the economic viability of dairy operations.It not only leads to reduced milk production and overall performance but also increases the risk of health issues for affected cows. Consequently, addressing lameness is crucial for maintaining both the welfare of the herd and the profitability of dairy farms.To address this issue, this paper proposes a deep learning-based algorithm for the automated detection of lameness in cows. Lame cows typically show observable indicators during walking, such as a lowered head position, pronounced head movement, and an arched back, whereas healthy cows demonstrate minimal head movement, maintain a straight back, and exhibit normal gait and body equilibrium. Based on these obvious movement characteristics, this investigation aimed to detect cow lameness by tracking the movement patterns of six key anatomical points: the head, neck, shoulder, center of the back, loin, and tail.Firstly, two mobile devices were positioned adjacent to the passage leading to the milking area, and video data of 160 walking sequences from 83 cows were collected. In order to reduce the effects of light variations in the channel, background barbed wire fence boundaries, and foreground fence occlusion,the YOLOv8n-seg instance segmentation algorithm, recognized for balancing computational efficiency and accuracy, was employed to accurately identify cows in the images and extract their coordinates and pixel regions.Secondly,based on the results of instance segmentation six types of keypoint detection datasets were constructed, including RGB images, binary segmentation images, segmentation result images along with their corresponding versions cropped based on the target detection frame. Four backbone networks, MobileNet-V2, ResNet-50, ResNet-101, and ResNet-152, are used to train and test these datasets. After comparative analysis of the keypoint detection effect. Segmented images cropped by the detection frame were selected as the optimal input format, with ResNet-152 chosen as the best-performing backbone network. Then, the DeepLabCut algorithm was used to automatically extract the coordinates of six key points from the video sequences: the head, neck, shoulder, center of the back, waist, and tail, resulting in the creation of a lameness detection dataset.Lastly, a comparative analysis was performed to evaluate the performance of Temporal Convolutional Network (TCN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Bidirectional LSTM, BiLSTM, and FN-BiLSTM models in claudication detection. (Bidirectional LSTM, BiLSTM), and FN-BiLSTM models in lameness detection. In order to gain a deeper understanding of the effect of the models, ablation experiments were conducted on the FN-BiLSTM model, focusing on verifying the effects of the Filter and Noise layers on lameness detection in cows.The results demonstrated that the FN-BiLSTM model achieved optimal performance with 97.16% accuracy, 95.71% precision, and 99.04% recall for lameness recognition on a test set of 16 videos from 16 cows. Moreover, the instance segmentation model exhibited high efficacy in capturing the image sequences of cows and their whole-body semantic information, even under variable illumination conditions and different bovine-to-camera distances. The precision, recall, and mAP of the test set reached 99.97%, 100%, and 99.5%, respectively. During the keypoint detection phase, optimal performance was achieved when utilizing the cropped segmentation maps as input, with ResNet-152 as the backbone network, resulting in root mean square errors of 2.04 pixels and 4.28 pixels for the training and test sets, respectively.These findings suggest that this investigation offers a valuable technical approach for the automated detection of cow lameness in the livestock industry. This has the potential to enhance the efficiency and animal welfare of dairy operations, thereby promoting the sustainable development of the livestock sector.

       

    /

    返回文章
    返回