Abstract: Image recognition with fruit color character in natural scene plays an important role for efficient and real-time recognition. Eighty images were recognized with an R/B segment threshold of 1.375. The rates of detection, successful recognition, false recognition and missing recognition were 98.3%, 86.1%, 12.4% and 13.9% respectively. Furthermore, the above rates were 90.3%、84.9%、6.0% and 15.1% with an additional V segment threshold of 0.45. The comparison results showed that although the rate of successful recognition decreases slightly, the rate of false recognition fell remarkably. A series of test were executed under direct sunlight with strong, medium and weak illumination, and backlighting with strong, medium and weak illumination. The results showed that the rate of detection and successful recognition were the highest of 91.7% and 87.7%, and the rate of miss recognition was the lowest of 12.3% under direct sunlight and medium illumination. The rates of successful recognition in various conditions were different, but not significant.