Automatic registration of IR and optical pig images based on contour match of radial line feature points

Abstract: In the research of pigs health monitoring, pig contour segmentation and feature extraction using optical images are difficult because of pig manure and illumination in the rough environment of pig house. To improve the effect of pig contour segmentation and feature extraction, fusion of infrared thermal image and optical image with an IR thermal imager is suggested. Moreover, it may provide the helpful data source about pigs for the research of the relationship between abnormalities and the temperature of body surface. Evidently, automatic registration of IR and optical images is a crucial step towards constructing fusion. As to this kind of non-rigid multi-sensor images registration, apart from some similarity in overall structure, there is almost no commonality in some popular feature spaces between the pair of images, and it shows that feature-based approaches, such as scale invariant feature transformation (SIFT) may be unsuitable for this type of image registration. In this paper, an auto registration method of IR and optical pig images is proposed based on contour match of radial line feature points. For reducing the computational complexity, a FLIR T250 infrared thermal imager is used，which can acquire 320×240 IR thermal image and 2 048×1 536 optical image at the same time and make the centers of the two images coincided. Comparing with acquiring IR thermal image and optical image by using IR thermal imager and digital camera separately, this device can not only avoid the inherent error caused by asynchrony when acquiring moving image, but also reduce the computation of translation and rotation in registration process. It only needs to compute suitable scale factor. As the difference between pigs and environment is often evident in IR image and the contour of pig is relatively complete, a series of pre-processing need to be taken to output the binary image of pig, such as optimum global thresholding using Otsu method, hole filling, etc. As to pre-processing of the optical image, the canny operator is used to acquire the edge image.Several processes are taken to achieve the registration. Firstly, the feature points set of IR image, which is used in matching, is constructed by extracting the cross points of the contour line and the auxiliary lines which are a group of radial lines starting from the area centroid of the binary image with equal interval angle. Secondly, the mapping point of the area centroid is found by transforming it into optical edge image using an initial scale factor. A feature points set is also be constructed by extracting the cross points of the edge in optical edge image and auxiliary lines, these lines are also a group of radial lines starting from the mapping point with the same interval angle. Thirdly, weighted partial Hausdorff distance (WPHD) between the two feature points sets serves as the similarity measure. In the matching process, the scale factor is changed repeatedly and the feature points set of the optical image has been alternated relatively, the total of WPHD between the two sets is computed. Finally, by iterating the scale factor, the minimum value of WPHD is corresponding to the optimum scale factor. The resilient propagation (RPROP) algorithm is also used to accelerate the iteration procedure. In this paper, 50 pairs of IR and optical images were tested by using the proposed algorithm. The success rate of registration is 94%. The average error of registration is less than 1 pixel, which is equal to or better than manual registration result. Experimental results demonstrate the effectiveness of the proposed algorithm. The research can provide a basis for the further researches such as fusion of IR and optical pig images, multi-senor image feature extraction for pig health monitoring.