Multi-target tracking of group-housed pigs based on PigsTrack tracker

An accurate and rapid identification of pig behavior can play a crucial role in the evaluation of health status and welfare. Individual detection and tracking of pigs can greatly enhance the recognition and statistical analysis of their behaviors in large-scale farming environments. This shift from group to precision management can enable better treatment and care, ultimately leading to the goal of welfare-friendly farming. Nevertheless, the occlusion and poses challenges often remain in the actual scenario of pig farming during object tracking, where the group-raised pigs often overlap and move irregularly with each other. Among them, the current mainstream optimization tracking algorithms (such as ByteTrack) can be used to effectively reduce the missed detection from the occlusion using low-score box matching. But it may increase the false detection. The occlusion recovery matching scheme of the OC-SORT algorithm can be used to significantly reduce the identity changes from the occlusion. However, both algorithms still lack the appearance feature matching, which can easily lead to the target loss in the low frame rate or stuttering videos. The appearance feature matching can be expected to solve occlusion and irregular motion. Unfortunately, the existing convolution-based identity recognition networks often struggle to obtain the appearance features with better discriminatory properties, due to the highly similar appearance of pigs. In this study, a PigsTrack tracker was proposed to further optimize the multi-object tracking for the group-raised pigs in actual scenarios. Firstly, the high-performance detector (YOLOX) was used in the tracker to reduce the false and missed detection rates of pigs in the occluded scenarios. Then, an identity recognition (Re-ID) network was constructed using the Transformer model to extract the appearance features with better discriminative properties. Secondly, the matching strategies of appearance feature, IoU, and occlusion recovery were integrated with the tracker for the accurate tracking of group-raised pigs using the idea of OC-SORT. 20 videos were selected (a total of 36 000 frames) from the original PBVD (Pigs Behaviours Video) dataset. The obtained dataset was labeled using the DarkLabel tool. Furthermore, the main evaluation metrics included HOTA (higher order tracking accuracy), MOTA (multiple object tracking accuracy), and IDF1 (identification F1 score). The comparison showed that the PigsTrack tracker outperformed the rest in all indicators of tracking performance with the HOTA of 85.66%, MOTA of 98.59%, and IDF1 of 99.57 %, which were higher than those of DeepSORT, OC-SORT, ByteTrack and StrongSORT trackers, respectively. Additionally, the three-step matching also performed better stable tracking to reduce the number of false and missed detections, as well as the identity switches, with a total of only 13 identity switches. Overall, the PigsTrack can be suitable for the multi-object tracking of group-reared pigs in the breeding environment, particularly for the pig interactions, occlusions, and similarities in appearance. The visualization of the attention map confirmed that the precise tracking of pigs in long videos was achieved in the feature extractor using the Transformer model. Furthermore, more distinctive appearance features were extracted for more detailed information. The tracking results were compared with and without occlusion recovery matching. As such, the occlusion recovery matching greatly contributed to the tracking pigs in dense environments. Finally, the pig behavior in videos was analyzed using the PigsTrack with the Slowfast network. Experiments demonstrated that superior accuracy, effectiveness, and continuity were achieved to recognize and statistically analyze the daily behaviors of group-reared pigs. In addition, the ABVD dataset was utilized to create Dataset 2 for the supplementary experiments, and then to validate the generalization performance of the model in multi-target tracking tasks of group-raised pigs. The performance of the PigsTrack tracker on Dataset 2 (with HOTA, MOTA, and IDF1 scores of 69.14%, 94.82%, and 90.11%, respectively) outperformed the rest, indicating only a slight gap, compared with the PBVD dataset. Therefore, it is also necessary to further optimize the experimental environment and tracker for the engaged pigs in aggressive behavior. Overall, the PigsTrack can provide an important foundation for precision farming.