Abstract: Corn grain damage has been one of the most serious challenges during harvest, even to restrict the popularization and application of direct harvest technology in China. It is necessary to rapidly and accurately obtain the grain damage in the intelligent process of corn harvest. In this study, an improved detection was proposed for the corn kernel damage using deep learning. Two parts included: the detection device and the algorithm of corn kernel monolayer. The single-layer detection device aimed to change the chaotic grain flow into a stable state, particularly for the high-quality corn grain images that fully met the detection requirements. The feeding speed was controlled to ensure the normal operation of the detection device in the process of image acquisition. The angle between the device and the horizontal plane was optimized to solve the phenomenon of image dragging. A two-stage model of deep learning segmentation and classification was used to detect the damaged corn grains. Specifically, the deep learning classical instance segmentation model (Mask R-CNN) was used to complete the segmentation of corn kernel monomer in the region at the image segmentation stage. The image classification was realized by a new network model (BCK-CNN) using the residual module. The experiments show that the Mask R-CNN model shared the better performance on the segmentation of corn grains, in order to fully support the subsequent whole and damaged corn kernel classification task. The effectiveness of the BCK-CNN classification model was verified to compare it with the GoogLeNet, VGG16, ResNet classical classification network, and Mask R-CNN model. The visual technology was used to evaluate the classification performance of different models for corn grains. The results showed that the BCK-CNN model achieved the best comprehensive classification performance for corn grains, with the classification accuracy of whole and damaged corn grains reaching 96.5% and 94.2%, respectively, indicating the highest detection efficiency, compared with GoogleNet, VGG16, ResNet and Mask R-CNN models. The average processing time of 60 single corn grain images was only 19.9 ms. The performance of the damaged corn kernel detection was verified (Mask R-CNN+BCK-CNN), where the average relative error was selected as the evaluation index using the manual calculation of the damaged kernel rate. The average relative error of the improved model was only 4.02%, compared with the manual Mask R-CNN and (used alone), Mask R-CNN+GoogLeNet, Mask R-CNN+VGG16, and Mask R-CNN+ResNet. The detection time was controlled within 1.2s for the single-cycle corn kernel set image when deployed on the mobile industrial computer, which basically met the real-time detection requirements. The finding can provide a strong reference for the efficient and accurate detection of damaged grains in the process of corn harvesting.