Tractor driver fatigue detection based on convolution neural network and facial image recognition

Abstract: Tractor is a popular tool for agricultural farming in the world. But many factors such as high labor intensity, absence of sleeping and monotonous environment make driver fatigued easily in farming season. Aiming at the phenomenon of fatigue driving which is the major reason for tractor traffic accidents, a tractor fatigue detection method based on facial feature recognition using convolution neural network was proposed. Firstly, 1200 driving images were sampled during different daytimes by 60 drivers in which the male accounted for 77%. Then the face images of tractor drivers were pretreated by gamma intensity correction (GIC) method, aimed to solve the problem of the uneven brightness of images with fast processing speed because of its simple algorithm, and hence reduce the influence of illumination of faces. Due to the complex working environment of the tractor and the special surrounding window design, wavelet denoising method was applied in image denoising because it is powerful for the removal of impulse noise and Gaussian noise which are the main noise in the images. In addition, principal component analysis - skin color model (PCA-SCM) method was used to detect and locate faces and then skin color model was applied to rectify and extract the facial area. For improving the face recognition precision, dual-face recognition and checking algorithm was proposed. Firstly, core feature face database was established and human face recognition classifier was generated. Secondly the core feature memory space of human face was formed to locate human face in image. Thirdly, to establish a Gaussian skin color model in YCbCr color space and perform the binary converting, bloat and corrosion methods were also applied to remove fake areas for improving accuracy. In the end, the obtained face position was compared with the previous one to evaluate the effectiveness of this image. Finally, the drivers' facial images were input to the convolution neural network (CNN) for training, and the driver fatigue detection model was established to identify the fatigue of tractor driver based on the face image. The neural network was mainly composed of an input layer, 2 convolution layers, 2 pooling layers, 2 fully connected layers and an output layer. The batch size was 30, and the convolution layer sampled 16 channels of input layer. After reducing the dimension of the feature map by the pooling layer, the full connection layer classified the extracted features and then the correctness rate and the loss rate were obtained. The AdamOptimizer was selected as the optimizer to improve the gradient descent by using the momentum (moving average of the parameters). The hyperparameters were dynamically adjusted to minimize the loss function of the network to achieve better discrimination. Additionally, the changes of weight parameter and bias parameter in convolutional layer and all-connected layer were analyzed, as well as the change trend of correctness rate and loss rate. The T-SNE dimensionality reduction iterative analysis was applied in CNN training. Compared with other methods of fatigue driving detection, such as dynamic template matching and BP (back propagation) neural network, CNN has obvious advantages in detecting correctness and detecting speed. Due to the unique convolutional kernel structure of CNN, more and more useful information in the image can be extracted more efficiently and quickly. CNN still has good detection efficiency compared with the fuzzy inference detection method. Experiments showed that the proposed detection model's accuracy rate is 98.9%, and the recognition time for each frame of image is 38 ms (Using Inter i7-4510U dual-core processor), which demonstrate that the proposed image processing method using CNN can realize fatigue detection of tractor driver in real time.