Method for measurement of maize stem diameters based on RGB-D camera

Abstract: Stem diameters of maize are important phenotype parameters and can characterize the crop growth and lodging resistance, drawing more attentions from breeders. Traditional measurement about stem diameters is usually manual measurement, which is timeconsuming, laborious, and subject to human error. In order to rapidly measure stem diameters of maize in field, a method based on RGB-D (red, green, blue - depth) camera was proposed in this paper to extract stem diameters of maize. The color images and depth images of the maize plants at the small bell stage were captured by a RGB-D camera in field. First, maize stem was extracted by processing the color image. It was hard to recognize maize just according to the color differences in red, green and blue component between maize and background due to the illumination variations. To solve the problem, the component that represented the difference between green signals and illumination brightness was calculated and applied to segment maize with Otsu algorithm, and the binary image of maize was generated. And then erosion operation was conducted within region of interest to cut off the connection between little leaves and maize stem, and small regions were eliminated to remove weed and little leaves. The largest region of maize was saved after dilation operation. After that, skeletonization was conducted for main stem. There were crossing points at the points of contact between leaves and stem, and ending points at the points of contact between ground and stem, and the potential measurement region of stem could be identified by searching crossing points and ending points. The color coordinates of the potential measurement region were saved and corresponding point cloud data were generated based on the mapping relationship between color coordinate, depth coordinate and camera coordinate. Second, stem diameters were calculated by processing point cloud data. Noise points affected measurement accuracy of stem diameters, and K-nearest method was applied to remove scattered points from point cloud data. Then the filtered point cloud data of potential measurement region were clustered. There were some point cloud data on the edge of stem due to the measurement of time of flight (ToF), which were background noises. K-means method was used to divide the filtered point cloud data into 2 groups, and only the group whose central point was nearer to the camera was saved to represent maize stem. The saved point cloud data were one side of stem, and ellipse fitting based on least square method was carried out for the point cloud data. Long axis parameter and short axis parameter of ellipse were calculated respectively to indicate the stem diameters of maize. 20 samples were tested to verify aforementioned method, and the experimental results showed that the method proposed in this paper had a good performance in segmenting and identifying maize stem, though ellipse fitting method needed to be improved. The mean errors, standard deviation and mean relative errors of measuring stem diameters were 3.31 mm, 3.01 mm, 10.27% for long axis and 3.33 mm, 2.39 mm, 12.71% for short axis, respectively, indicating that the proposed method could be applicable for plant phenotyping.