Lactating sow image segmentation based on fully convolutional networks

Abstract: The behaviors of a lactating sow reflect welfare and health that affect piglet survival and growth during lactation. Computer vision has been widely used to perceive the behavior of animals for precision husbandry, which is useful to increase the productivity and reduce the disease rate. Effective and accurate segmentation of individual lactating sow is a vital step to record and analyze the lactating sow behavior automatically. However, under real pigsty conditions, it is a challenge to segment lactating sow from the background due to occlusion, uneven color on sow body surface, variations of sow size and pose, varying illumination and complex floor status. In this paper, we proposed an algorithm for lactating sow image segmentation based on fully convolutional networks (FCN). To design FCN for accurate segmentation, VGG16 was chosen as a basic network where the fully connected lays were converted to convolutional layers, and the FCN-8s skip structure was designed by combining semantic information from a deep, coarse layer with appearance information from a shallow, fine layer. We called this network FCN-8s-VGG16. The steps of our work were as follows: First, top view images were taken from 28 pens of pigs under a real pigsty circumstance and a total of 4 334 images were obtained, of which 3811 training images were selected from images of 7 pens and 523 test images were selected from images of the other 21 pens. And, all the images in training set and test set were manually labeled. Second, adaptive histogram equalization was used to improve contrast in training images. Then, the pre-processed training set was fed into FCN-8s-VGG16 to develop an optimum FCN model by the fine-tuning of the network parameters using Caffe deep learning framework on an NVIDIA GTX 980 GPU (graphics processing unit). After that, test set was put into the trained model to obtain the segmentation results. Then, to fill holes within objects and remove small objects, a post-processing was performed by using a disk structure of mathematical morphology and calculating the areas of connected regions. Finally, we compared our FCN-8s-VGG16 network architecture with different network architectures including a different skip architecture (FCN-16s based) and 2 different basic networks (CaffeNet based and AlexNet based). Besides, comparisons with other methods were also conducted, including the previous state-of-the-art simultaneous detection and segmentation (SDS), Graph-based and Level-set algorithm. The results on the test set showed that the algorithm achieved a complete segmentation of lactating sow by minimizing the effects of uneven color, light variations, occlusions, adhesion between sow and piglets and complex floor status, with an average accuracy of segmentation of 99.3% and a mean regional coincidence degree of 95.2% at an average speed of 0.22 second per image. However, it is hard to completely segment the sow's head when sow's head is downwards to floor, or close to the wall or adheres to piglets. The comparison with different network architectures showed that the mean regional coincidence degree of our proposed network architecture was higher than that of the others, and on GPU, the segmentation speeds of our FCN-8s-VGG16, FCN-16s based, CaffeNet based and AlexNet based were 0.22, 0.21, 0.09, and 0.09 second per image, respectively, which had good real-time performance. The comparison with other methods showed that our FCN-8s-VGG16 model outperformed others, which improved the mean regional coincidence degree of SDS, Graph-based and Level-set by 9.99, 31.96 and 26.44 percentage point, respectively. All of the experimental results suggest that the proposed method demonstrates a higher generalization and robustness, and provides an effective access to accurate and fast segmentation of lactating sow image under a pigsty circumstance.