Abstract: This study aims to better define the features of non-grain cultivated land in hilly and mountainous areas, in order to clarify the driving mechanisms and causes of its occurrence. An empirical study was carried out in Xiushui County in Jiangxi Province, China. The specific procedures were as follows. Firstly, the rice planting pattern was obtained using the multi-temporal Sentinel-1A backward dispersion coefficient analysis in conjunction with field research, and high-resolution Google Earth images for classification and supervision. Secondly, this non-grain cultivated land was generated to remove the rice planting patterns from permanent basic agriculture, after which the permanent basic farmland pattern was superimposed. The geographical distribution of non-grain cultivated land in the case sites was analyzed using the standard deviation ellipse, kernel density analysis, and spatial autocorrelation. Lastly, the geographic detector analysis was conducted to evaluate the impact of various factors in three dimensions of cultivated land site, traffic location, and policy control factors on non-grain cultivated land. The interaction relationship between the factors was evaluated as well. The results demonstrated that: 1) The area and percentage of non-grain cultivated land were 17899.59 hm² and 57.47%, respectively, in the study area. The geographical distribution was found in the two distinct groups in the southeast and northwest. The spatial distribution patterns also shared with the low-low, high-high, and low-high agglomeration with a substantial positive correlation. 2) The majority of non-grain cultivated land was also characterized by the distribution patterns in the vicinity of the industrial center towns. The distribution density was the highest in the west, while the lowest in the east, with a progressive decline towards the northeast. Furthermore, the cash crop cultivation resulted in a much greater amount of non-grain usage, compared with the locations. 3) Some variables shared an impact on the non-grain cultivated land, including the distance to rural roads, slope, elevation, decision-making on the main grain-producing area, soil erosion intensity, and the distance to town centers, urban highways, and water resources. The types of driver interactions were all nonlinear enhancement. Particularly, three driving factors interacted strongly with the rest: distance to town centers, distance to rural highways, and decision-making on the main grain-producing area. The measurement index of non-grain cultivated land was then investigated using multi-source data analysis and multi-dimensional influencing factors, in terms of the definition of non-grain crops and long-term time series images. More comparable measurement was required to determine the hot spots at the regional level of non-grain farmland. The findings can provide a sound foundation for the development of differentiated governance and control measures. The protection and use of cultivated land resources can be better realized for the scientific diagnosis of regional farmland non-grain production.