Abstract: Some agricultural tasks consist of applying chemical fertilizer to crops, but the products are often applied throughout the field in most cases, which cause pollution of water and possible chemical residues. In order to apply the products selectively and reduce the quantity of application, an autonomous vehicle can be used. Generally, this kind of vehicle follows the crop rows autonomously in the field where plants are arranged in rows, so its pose relative to crop row is important for tracking algorithm to work. With the machine vision, a novel method to calculate this pose was demonstrated, which could adapt to the complex characteristics of field environment excellently. First, some shortcomings involved in the conventional measuring method were analyzed carefully, such as processing time being long, pixel weight in the digital image being ignored and so on. With the local linear model of the tracked crop row then the algorithm was deduced at full length. Finally, based on the prototype of autonomous agricultural vehicle, the experiment was carried out, and it was shown that compared with the manual measurement the standard deviation of offset was 3 cm and of heading angle 0.62 deg while without any fixed displacement.