Segmentation method for cotton canopy image based on improved fully convolutional network model

Abstract: Using computer vision technology to monitor cotton growing state is the development trend of cotton production informatization, and is also the widely used automation technology. Cotton field canopy image segmentation under natural scenes is an important part for monitoring growth status of cotton. Since the existing methods are not good in the image segmentation of cotton fields in the environment of light and background, and the adaptive ability is not strong, an image segmentation method based on the fully convolution network and conditional random field was proposed. Firstly, fully convolutional network was used to extract the image features, and the fully convolutional network was trained by back propagation algorithm and stochastic gradient descent(SGD)to obtain the initial segmentation results; This network model is an improvement on the traditional network model convolutional neural network (CNN), which was widely used in image classification. The full connect layer is transformed into a convolution layer, so that the fully convolutional network's output image as input image size, so it can be applied in the field of image segmentation. For achieve the FCN segmentation model structure, VGG16 was chosen as a basic network structure. And for improve the accuracy of segmentation results, we design the skip structure which can combine pool3 pool4 pool5 feature pictures to combine the FCN-8s. Secondly, the initial segmentation results were input into the conditional random field, used the classification vector of image pixels and the relative relationship between the pixels to construct the energy function and train the conditional random field, obtained the best performance conditional random field; conditional random field is an undirected graph model, this model builds a graph on the relationship between image pixels and image pixels. Each pixel has different label, conditional random field improve the quality of the segmentation results and optimal segmentation results by minimizing the graph energy function. Thirdly, we through the verification process, the structure of the full convolutional network and the conditional random field model is optimized. Finally, used the trained fully convolutional network and conditional random field make inferences about input image to get the final segmentation result.In order to verify the effect of the method proposed in this paper, 625 test cotton canopy images were captured from the cotton producing areas in Xinjiang, China from April to July 2017. These images were collected by the Canon EOS6D digital camera with 1360×800, and zoomed into 850×500 pixels. The 375 images in these images are used as the training set image, and 125 images are used as the validation set images, and the 125 images are used as the test set. This method programming development environment is Python2.7, the operating system is Ubuntu 14.04 LTS, the deep learning development framework is Chainer 1.24 on an NVIDIA Quadro k5000 GPU(graphics processing unit). The experimental result show that proposed method's mean pixel accuracy is 83.24%, mean IOU is 71.02%, Compared with the traditional fully convolutional network, the performance of the proposed method is improved by 16.22 and 12.1 percentages respectively, proposed method has obvious improvement effect. Compared with the Zoom-out and conditional random field as recurrent neural networks(CRFasRNN) segmentation methods, the average pixel accuracy of fully convolutional network is improved by 4.56 and 1.69 percentages, and the average crossing ratio is improved by 7.23 and 0.83 percentages respectively. Compared with the logistic regression method and SVM method, the average pixel accuracy of fully convolutional network is improved by 3.29 and 4.01 percentages, and the average crossing ratio is improved by 2.69 and 3.55 percentages respectively. In order to verify the robustness of the method proposed in this paper, the segmentation of cotton canopy images under complex environment(plastic film), complex canopy area(leaf cover) and complex illumination condition(poor contrast in the canopy area) were carried out by different methods. Because of the GPU parallel computation technology, the proposed segmentation method is very efficient, the average processing time is 0.33s. The segmentation result show that the proposal method is less affected by environment and has better robustness. This study can provide reference for automatic monitoring of cotton growth status.