Recognition of weeds at seedling stage in paddy fields using multi-feature fusion and deep belief networks

Abstract: Weed identification was the key to the site-specific weed management in the field. The machine vision method was adopted to realize automatic and rapid detection of weeds. This paper selected 6 weed species in paddy fields, including Alternanthera philoxeroides, Eclipta prostrata, Ludwigia adscendens, Sagittaria trifolia, Echinochloa crus-galli, and Leptochloa chinensis, which were captured in early growth stages with natural background and variable illumination. A total of 928 images were taken. The Alternanthera philoxeroides, Eclipta prostrata, and Ludwigia adscendens were dicotyledonous weeds which had large heart-shaped opposite leaves, and the other 3 weed species were monocotyledonous weeds which had narrow leaves. The image was 640×480 pixels and only a single seedling of weed was in the scene, and the acquisition format was color images of RGB (red, green, blue). The component with 1.1G-R was applied to gray level transformation of original RGB images. The OTSU adaptive segmentation method was adopted to realize the image segmentation of grayscale image. The morphological operation was used to fill vacancies in weed images. The noises and small target were eliminated based on area-reconstruction operator. The background was removed by masking algorithm between binary image and original RGB images. The 101-dimensional features were extracted from the foreground image of weed, including color, shape and texture feature. The color feature was composed of the first, second and third moments, the shape feature was composed of geometric features and improved moment invariant features, and the texture feature was composed of gray level co-occurrence matrix and local binary patterns (LBP) feature. The weighting matrix of color, shape and texture feature would be the input parameter after unitary processing. A three-step method for model updating consisting of model structure tuning, model parameter updating and model validation was presented in this article. Firstly, the deep belief networks (DBNs) of double hidden layers and single hidden layer were established. Secondly, the influence of the 3 types of constant, rising and descending nodes of double hidden layers in DBN was analyzed. The experimental result showed that the descending nodes of double hidden layers in DBN could learn the distributed characteristics of the original characteristic data better than the other node types of double hidden layers. Finally, the testing optimization parameters of double hidden layers and single hidden layer were obtained by experiment. The recognition rate of double hidden layers of DBN was 83.55% when the number of nodes stood at 101-210-55-6, and the recognition rate of single hidden layer of DBN was 91.13% when the number of nodes stood at 101-200-6. The DBN structure of single hidden layer was better able to excavate the distribution rule of weed features than DBN with double hidden layer. The single color, shape, texture and fusion feature were used to construct 3 types of weed classification models, which were support vector machine (SVM), BP (back propagation) neural network and DBN. In the experiment, the recogniton rate of DBN model with single color and shape feature was lower than that of the SVM and BP neural network model. The dimensions of color and shape feature were relatively small, which could not reflect the advantage of characteristic representation with DBN. On the other hand, the recognition of DBN model with single texture and fusion feature was more accurate than that of the SVM and BP neural network model, and the recognition rate of DBN model reached 86.58% and 91.13% with single texture and fusion feature, respectively. The results demonstrate that the method put forward in the paper can improve the classification accuracy of weeds with the complex background and variable illumination in paddy fields.