Abstract: For harvesting robots, it is difficult to recognize and locate ripe litchi fruits and their main fruit bearing branch from clusters of litchi in complicated background. Hence, analyzing and recognizing color feature of the images of litchi fruits, litchi main peduncles and leaves become research focuses. Firstly, according to the specialty and uncertainty of litchi fruits, litchi main fruit bearing branch, the method of combining the exploratory analysis with litchi image recognition was put forward in this research by discovering the uncertain element spaces of targets in images, sorting the litchi main fruit bearing branch into three colors of partial green, partial red and partial brown, sorting the litchi fruits-influencing illumination into highlight, normal light and backlighting, thus classifying and analyzing the images of all the parts of litchi, and then providing the flow chart of the exploratory analysis of litchi image data. Secondly, the exploratory analysis about color feature of litchi fruits, litchi main fruit bearing branch and leaves was illustrated in this research, and sorted box-plot on color component for all parts of litchi based on the RGB(red, green, blue), HSI(hue, saturation，intensity ), Lab (L stands for lightness, and a and b stand for the color-opponent dimensions based on nonlinearly compressed CIE XYZ color space coordinates), YCbCr (luminance is denoted by Y, Cb and Cr are the blue- difference and red-difference), normalized rgb and I1I2I3 were designed. With data analysis about the graphics of box-plot, a vision model of recognition of different parts of litchis was given based on Cr gray-scale image of YCbCr color space. When the threshold value of Cr was between 0.50 and 0.54, the leaves and lateral branches in complex background can be removed and thus litchi fruits and their main fruit bearing branch from the cluster of litchi can be segregate. Finally, taking 60 groups (all together 180) of differently illuminated litchi images collected in natural circumstance as test objects, all ripe litchi clusters and litchi fruits of testing images with the threshold segmentation method were effectively recognized based on the vision model of Cr color feature, and their recognition ratio was 91.67% and 95.00%, respectively. After that, the main fruit bearing branch was successfully extracted from the recognized litchi cluster by operation on the recognized images (with recognition ratio 86.67%), and with the calculated picking-point, the visual location simulation was carried out. The results of the test experiment and visual simulation attest that vision model based on Cr gray-scale image of YCbCr color space combined with the corresponding segmentation method can effectively recognize different parts of litchi.