Abstract: Abstract: When the harvesting robot picks fruits, the separation of fruit and fruit branch leads to the other fruits oscillating on the fruit tree, using the cut way or twist way. In addition, the wind also leads to the fruit oscillating. In the past, the harvesting robot using the static recognition method recognized them accurately when the oscillating fruits would stop and either wait or frequently recognize them in the process of the fruits' oscillation, which spent more picking time and influenced the total picking efficiency of the harvesting robot. Because the static recognition method cannot obtain the dynamic characteristics of the oscillating fruit, the harvesting robot cannot accurately pick them during the fruits' oscillation. It is obvious that the static recognition method cannot meet the fast picking requirement of the oscillating fruits. In order to resolve the problem that the fruit oscillation influences on its recognition, orientation, and then fast picking, the dynamic recognition method under the fruit oscillation condition for the harvesting robot was researched. In the first place, the dynamic recognition process of oscillating fruit was introduced. The picking target fruit was determined which would be taken as the template of the following recognition. In the next place, the Fast Inverse Square Root algorithm and Fast Hartley Transform algorithm were applied to increase the recognition speed of the Mean-residual Normalized Product Correlation matching algorithm. Then, the fast Mean-residual Normalized Product Correlation matching algorithm was improved to be the property of resistance to rotation again, which was used to recognize the dynamic images. At last, the matching recognition tests were done, and the test results showed that the average recognition time of the improved fast algorithm was 0.33 s and the improved rotation invariant algorithm had the rotation invariance in the wide range of ?55°, 60° and could recognize the oscillatory fruit accurately. In addition, the timely update of the template was also used, which could meet the recognition requirement.