Test of binocular vision-based guidance for tractor based on virtual reality

Abstract: Machine vision-based guidance of agricultural machinery operates flexibly in complex field. The classical test methods for agricultural guidance systems are based on real field test. There are many problems for the classical test methods, such as high test cost, strong dependence on crop growing period, long test period, and being easy to damage crops. To solve those problems, a novel test method based on virtual reality for binocular vision based guidance system was presented. A virtual system was built with this method. The virtual test system is composed of the modules of test scene, physics engine of tractor, and control of path tracking. The test scene module consists of crop rows, road and four-wheel tractor, which provides image data for pathway detection and road roughness for the tractor. Models of the test scene were created with 3ds Max and Multigen-Creator as modeling tools and with Vege Prime as visual simulation tool. The physics engine of tractor was used to simulate the dynamics of tractor accurately and quickly according to the real tractor parameters and the information of the test scene. The position and attitude of the tractor were solved and rendered in Vega Prime. A simplified model was used to solve the dynamics of the tractor, including the vehicle model, tire model, and road solution model. To reduce the computational cost, the vehicle model was simplified to a model of 11 degrees of freedom, which are 6 degrees of freedom for the attitude of tractor body, 4 degrees of freedom for wheel rolling, and 1 degree of freedom for front wheel steering. The tire model was built based on the model of Dugoff-I to obtain the parameters of tire easily. The road model was built based on the modules of vpGroundClamp and tripod for collision detection, which solves the road roughness of each wheel. The control of path tracking consists of pathway determination, computation of turning angle of front wheel, and control of turning angle of front wheel. A reported and validated crop row detection method based on binocular vision was used to detect centerlines of crop rows. The initial alignment of tractor is located in the middle of the crop rows. Thus the centerline of that middle crop row would be the pathway during the path tracking. A computational model of the front wheel angle was built based on the pure pursuit method. The control of front wheel angle was designed based on the classical increment proportion-integral-derivative (PID) algorithm. Parameters of the PID controller were optimized with the genetic algorithm. Results of tracking a sinusoidal signal with the time of 5 s and 5° amplitude show that the control system responses quickly and overshoot is small. The software of the virtual test system was developed based on the C++ language in Visual Studio 2008. A tractor with the systems of front steering, rear driving and rear braking was used as the operation machine, the cotton at seedling stage was used as the target crop, and the crop row field was taken as the test scene. Virtual tests of tracking the curved crop rows at the tractor speed of 0.5, 1, 1.5, 2, 2.5, and 3 m/s were conducted. Results show that, the virtual test system simulates the crop field and tractor well in the virtual reality environment and can conduct the tests of tractor guidance based on binocular vision. The proposed method could provide theoretical basis and experimental data for the experiment and improvement of the guidance system. Results of path tracking are satisfying for the tractor speed within 2 m/s, and the amplitude, absolute average value and standard deviation of the position deviation are less than 0.347, 0.072, and 0.141 m, respectively; the amplitude, absolute average value and standard deviation of the direction deviation are less than 11.570°, 2.622°, and 4.462°, respectively.