Abstract: Wheat stripe rust and wheat yellow dwarf have posed a great threat to the yield and quality of wheat. An accurate identification has important implications for the prevention and control of wheat diseases. However, the phenotypic symptoms are similar to the infected leaves of wheat stripe rust and wheat yellow dwarf. Particularly, drought, nutrient deficiency, and bacterial disease can lead to the chlorosis and yellowing of plant leaves. In addition, the infected leaves are also similar to the healthy ones, due to the indistinct phenotypic symptoms in the early stage of diseases. It is difficult to quickly and accurately distinguish them by the existing identification. In this study, an improved Faster Regions with CNN Features (Faster-RCNN) was proposed for disease identification. There were two improvement strategies. Firstly, three 3×3 grouping convolution and down-sampling delays were employed to optimize the Deep Residual Neural Network (ResNet-50), which was designed as the backbone feature extraction network, in order to enhance the fine feature extraction of the entire network. Secondly, the region of interest (ROI) alignment was employed to reduce the feature error caused by double quantization, instead of ROI pooling. As such, the subtle differences were recognized after alignment. Transfer learning was selected to improve the training speed of the model. The data augmentation was then utilized to reduce the over-fitting, in order to further enhance the recognition performance and generalization ability. The image data set of disease leaf was collected from more than 200 wheat varieties with different resistance and susceptibility to the diseases, covering various symptoms at different disease stages. A series of experiments were carried out to evaluate the improved strategy. The performance indicators were selected to verify the model, such as loss function convergence curve and mean average precision (mAP). The experimental results showed that the mAP of the improved Faster-RCNN reached 98.74% for the wheat stripe rust and wheat yellow dwarf. Moreover, the early identification of disease infection was strengthened to predict the diseases as early as possible. The dataset contained 683 and 630 mild symptom images of these two diseases, respectively. The mAP reached 91.06% for the mild and severe symptom identification of two diseases. A comparison was made on the mainstream deep learning models, such as the SSD, YOLO, and RCNN series, under the same experimental conditions. Specifically, there were 9.26, 7.64, and 16.57 percentage points higher than the SSD, YOLO, and RCNN, respectively. Meanwhile, the loss function decreased significantly, while the model performed better than before. Finally, the intelligent recognition system was developed for wheat disease. Consequently, the average return delay was 5.024s under the maximum concurrent access of 100, and the success rate of recognition reached 97.85%. Anyway, the improved system can rapidly and accurately recognize wheat diseases via a WeChat applet. The finding can also greatly contribute to the control of wheat diseases.