Green ripe tomato detection method based on machine vision in greenhouse

Abstract: During the detecting and locating process of green tomatoes based on obtained visible images, problems such as shadow caused by uneven illumination source, occlusion of stems and leaves, and occlusion between fruits, need to be solved. In this study, a machine vision algorithm was put forward, which aimed to determine fruit location and size of green tomatoes. Normalized cross-correlation function (NCC) is a feature detection method using template matching. The algorithm firstly detected the potential location of green tomatoes through fast normalized cross-correlation function (FNCC). Then a gray histogram based classifier was used to classify if the location corresponded to green fruit. The histogram based classifier was built based on fruit areas and non-fruit areas of a certain size (30×30 pixels) extracted from the obtained image. Seven texture features, including mean, standard deviation, smoothness, third moment, uniformity, entropy, and gray level range were calculated for those fruit areas and non-fruit areas. Three classifiers including k-nearest neighbor (KNN), SVM (support vector machine), and Naive Bayes, were used to classify fruit and non-fruit areas using those 7 texture features as input vectors. SVM was chosen based on its performance. The non-fruit location was filtered out, and the number of fruit locations was estimated. Meanwhile, the image was segmented based on color analysis. Red and Blue component from RGB (red, green, blue) image, and Hue component from HSV (hue, saturation, value) image transformed from RGB images, were used as the basis for color analysis. Using the fruit potential location number estimated using FNCC as an input parameter of circular Hough transform (CHT), CHT was then applied to the edge image of the segmented result. The center coordinates and radius value of each circle were calculated. Finally, the detection results were merged based on the analysis of the distances of 2 centers of fruit circles detected using CHT. If the distance between 2 circles is smaller than the minimum fruit radius, the circle with a larger radius will be kept, while the other circle will be flagged as repeatedly detected one. Thus, the recognition and positioning of the green tomato were realized. When green fruits and red fruits appear on the same image, a red fruit detection algorithm based on the local maximum value method and random circle round transform detection, which was proposed by the author in another paper, would be carried out on the obtained image. Then the red fruit detection result was combined with the green fruit detection result. The proposed method combined the texture, color, and shape information of the image, and eliminated the disturbance of the color similarity between the green tomatoes and green leaves and stems. A total of 70 images were used in this study, in which 35 images were used as training images, and the other 35 images were used as validation images. The correct detection ratio for 72 fruits in the training dataset was 89.2%, and the correct detection ratio for 105 fruits in the validation dataset was 86.7%. The proposed method has provided a reference for the development of tomato harvesting robots for both red and green mature tomato fruits.