Abstract: To realize apple automatic grading completely, the method for classifying apple surface defect by digital image processing was researched. The feature parameters of blemish image region were extracted. According to features of apple surface defect and considering projective distortion of defect shape, a method for classifying apple surface defect was presented. The method divides a multi-pattern classification into the multilevel and simple two-pattern classification by binary tree. The method classifies defect into bruising, skin puncture, crack, maggoty fruit and insect bites by combining the neural networks with threshold judgement. Test shows that the method can be used to classify apple surface defect effectively.