Dynamic nondestructive sensing and grading manipulator system for apple quality

Abstract: Flexible and efficient detection and classification were here proposed for the multiple quality index of the apple. In this study, a manipulator system was also developed with nondestructive sensing and grading for the internal and external quality of apple using machine vision and visible and Near-Infrared (Vis/NIR) spectroscopy. A six-Degree of Freedom (DOF) mechanical arm was used to equip a self-designed end effector in the system. Specifically, the end effector was loaded with the optical sensor and grasping structure, in order to capture the Vis/NIR spectrum of the apple. A manipulator was obtained to combine the end effector with the mechanical arm. The apple was first grabbed on the assembly line, and then the spectrum of the apple was collected at the same time for sugar content detection. The spectra of apple samples were collected in the static and dynamic states. Some spectral preprocessing was implemented for the modeling and analysis using the Partial Least Squares (PLS). A CMOS camera was selected to collect the images for the dynamic positioning and external quality detection of apples. A target detection model of PP-YOLO deep learning was trained on the apple images to calculate the coordinate position of the apple for the fruit diameter and coloration. The experimental results show that the Normalized Spectral Ratio (NSR) preprocessing performed the best in the static and dynamic states. The best performance was achieved in the dynamic spectral model of the manipulator using the NSR and Coherent Anti-Stokes Raman Scattering (CARS). The correlation coefficient, Rv, was 0.958 9 in the dynamic spectral model, where the Root Mean Square Error (RMSE) was 0.462 7%. There was less influence on the prediction model. The overall manipulator system was verified in the field test. The manipulator was used to flexibly grab the apples without damage during work. Three detection indicators were also given for the fruit diameter, coloring degree, and sugar content. An automatic grading was then implemented, according to the indicators. As such, the apples were finally placed into the corresponding level box in terms of the grade information. A comparison was also made between the measured and predicted values of the three indexes. The predicted correlation coefficient of apple diameter, coloring degree, and sugar content were 0.977 2, 0.967 4, and 0.964 3, respectively, with the RMSE of 1.631 5 mm, 5.973 4%, and 0.504 8%, respectively. There was a strong linear relationship between the prediction and actual value, indicating a lower prediction error than before. The maximum classification accuracy was up to 95% in the manipulator system. The grading system of the mechanical arm was taken about 5.2 s to realize the positioning, grasping, detection, classification, and placement of an apple, indicating better reliability.