Abstract: Abstract: Apple diseases have pose a serious risk on the income of orchards in recent years. An accurate and rapid identification of apple diseases can be great benefit to better prevent and control diseases. Most effort has been made in the laboratory to train the identification model, due mainly to the limited condition for the deliberately infect apples in the real orchard. However, most models cannot fully meet the requirement of the disease detection in the large-scale production. In this study, a deep learning model (called CaTNet) was proposed to extract both the global and local information from the diseases of apple leaf. The image data of disease was collected from the apple orchards in the Jilin Province of China. A total of 16,464 images were obtained from the several publicly available datasets with the laboratory and natural environmental data collected from the field. Firstly, a model structure was constructed with both Transformer and convolutional neural network (CNN). Global and local information was extracted from the original images using the two branches. The strong generalization ability of the model was improved to learn a wider variety of features. Meanwhile, the global features were acquired to improve the resistance of the model to interference. Secondly, the Transformer block in the Transformer branch was optimized to make the structure simpler. In addition, a channel compression and expansion module was designed in the Transformer branch, in order to reduce the training cost of CaTNet for the less channel dimension of the input features. Afterwards, the multiple multilayer perceptrons were replaced by the grouped convolutional layers to further improve the computational speed of the model. Thirdly, the lightweight CNN branch was constructed with an inverse residual structure to fuse the point convolution of the expanded channels with the 3×3 convolution of the extracted information. The CNN branch was utilized to extract the local features of the image. As such, the model was more sensitive to the fine-grained features. Finally, the concat operation was implemented to fuse the different output of features from the two branches. After that, the CNN branch was selected to extract the local features from the global ones, whereas, the Transformer branch was extracted the global from the local. The multiple features to be cycled were also improved the generalization of the model. A comparison was made to clarify the effect of different down-sampling on the two-branch network. Specifically, an accuracy rate of 79.35%, 74.06% and 67.95% were obtained using pooling, 3×3 size convolution kernel, and 1×1 size convolution kernel for the down-sampling, respectively. The CaTNet model with two branches showed a computational speed of 0.108 2 s/Frame), which was faster than the various deep learning models, such as the EfficientNetV2 s (0.383 2 s/Frame) and PVT t (0.177 8 s/Frame). Consequently, the two-branch structure can be expected to accommodate more computation for the much higher computational speed. This finding can provide a design approach to build the deep learning models with the high generalization capability, particularly on the training with the high accuracy under only easily accessible data.