Abstract: Convolutional neural network (CNN), a hierarchical neural network, can extract powerful feature representations and make accurate classification at the same time. CNN has already made remarkable achievements in various fields such as image classification and object recognition. The ability of feature extraction of CNN has been used to retrieve images in lots of works, however, the powerful classification ability of CNN is ignored by most researchers. To improve the agricultural image retrieval performance, this paper proposes a reranking method that uses the classification ability of CNN. Firstly, the fine-tuned cnn model is used to extract the retrieval features of the query image and estimate the weight of each category of the query image. Second, the retrieved images are sorted according to the image similarity of the CNN features between the query image and each retrieved image, and then the initial retrieval results are obtained. Third, the initial retrieval results are used to calculate the weighted class average precision (CAP) of each image class. Finally, the order of image classes is obtained through sorting the classes according to the weighted CAP, and the retrieved images are re-ranked by the order of image classes. The images in the same class are retained their order in the initial result. Hence, the final retrieval result is obtained. Experiments of two publicly available datasets of remote sensing, PatternNet and UCM_LandUse, are carried to verify the validation of the proposed method. The experimental results are concluded as follows: 1) The reranking method can improve the initial results and get more relevant images in a contrast experiment. 2) Per class mean average precision (mAP) values of three features (FC6 and FC7 of VGG16, pool5 of ResNet50) are evaluated on UCM_LandUse dataset, and the reranking retrieval results have increased by approximately 30% than the initial results. 3) To determine the optimal parameter values, an experiment of the different training data volume on PatternNet is conducted to evaluate the influence of different number of training images on the retrieval performance. It can be seen that the mAP and ANMRR(Average normalized modified retrieval rank) improves with the increases of the number of training image. For example, the mAP of ft_pool5_rerank feature increases from 75.89% to 97.56% as the number of the training image per class grows from 5 to 90. 4) The average resort retrieval time increases by no more than 1% over the initial retrieval time. 5) The mAP of the proposed method on UCMD is 93.67%, and the ANMRR is 0.049 2, which is 0.235 8 lower than that of the state-of-the-art methods.The proposed method can realize higher retrieval performance of agricultural remote sensing image retrieval, it will be helpful to improve the level of information and intellectualization in the agricultural information field.