Automatic detection of multi-target ruminate cow mouths based on Horn-Schunck optical flow algorithm

Abstract: It is of great importance to realize the intelligent monitoring of dairy cow's rumination for improving the modern cultivation and supervision of cows' health. Cow's mouth area is the key to the intelligent monitoring of dairy cows' rumination. A multi-target cow rumination detection method based on Horn-Schunck optical flow method was proposed. The Horn-Schunck optical flow model was carried out to get optical flow field of the time series images for the ruminating video. In each frame of optical flow diagram, vector was used to represent the movement trend of the pixel block, the length of the vector represented the speed of the movement, and the direction of the vector represented the direction of the movement. The most intense part of the movement when cow is ruminating is related to the mouth area, and the region with the densest vectors in each frame of optical flow diagram is the cow's mouth area too. By superimposing each frame of optical flow diagram, the complete optical flow of cow mouth area could be obtained. The data of large optical flow in each frame were superimposed to a new optical flow graph, and the new optical flow graph was divided into binary image by using appropriate threshold segmentation. Disk type structure element was adopted in the mathematical morphological operation to the binary image. Because the most intense part of the movement when cow is ruminating is related to the mouth area, it could be determined that the largest area of the pixel block was the cow's mouth region, and the candidate cows' mouth regions were filtered out. A mathematical model was applied to detect the ruminant cows' mouth area. The main principle of this mathematical model was to determine the size and position of the test box. Setting test box as a square, the side length of the square is equal to the width of cow's mouth. To get the position of the test box, we need to get the coordinate of center point for the test box. Setting the x coordinate of the center point be equal to the rightmost x coordinate of the cow's mouth area minus 50% of cow's mouth width, the y coordinate is equal to the mean value of the y coordinate of each point for cow's mouth area minus 50% of cow's mouth width. In order to verify the effectiveness of the algorithm, 12 video data obtained from different environments including sunny day, cloudy day and strong windy day were verified. The length of each video is 10 seconds, and the number of video frames for each video is between 250 and 280 frames. The results showed that, for the multi-target cows, the proposed method could not detect the area of ruminant cow's mouth in strong wind environment in 2 videos, and could be used for the detection of ruminant cow's mouth in the remaining 10 videos. For 8 videos of the remaining 10 videos the region of ruminant cow's mouth could be detected successfully. According to the percentage of the actual mouth area by the detection box, the true filling rate was defined. According to the percentage of the optical flow algorithm's mouth area by the detection box, the detected filling rate was defined. The highest true filling rate of the 8 videos was 96.76%, the lowest true filling rate of the 8 videos was 25.36%, and the average true filling rate of the 8 videos for which the region of ruminant cow's mouth can be detected was 63.91%. The highest detected filling rate of the 8 videos was 98.51%, the lowest detected filling rate of the 8 videos was 43.80%, and the average detected filling rate of the 8 videos was 70.06%. The above results indicate that it is feasible to apply the Horn-Schunck optical flow method to detect the area of the multi-target cow's mouth automatically. This study provides the reference for intelligent monitoring of the ruminating behavior of cows.