Abstract: Fish living in a three-dimensional water environment exhibits various swimming gestures in aquaculture, thereby resulting in the irregular shapes of fish in the images, such as straight or curved bodies, even curved fishtails and bodies. The adhesion between different fish is also complex and varied significantly. In this study, a new depthwise separable convolution network was proposed to improve the accuracy and universality of recognition for the adhesive fish. Image processing was performed on the collected fish images, and then the images of fish-connected areas were segmented to construct an adhesive fish recognition dataset, with a total of 27 836 images. Firstly, some preprocessing operations were performed on the fish images to enhance image contrast, such as color space conversion, color component extraction, and median filtering. Value (V) component of Hue-Saturation-Value (HSV) color space was extracted to serve as the initial image for subsequent processing. Next, the fish images were segmented using the background subtraction. Finally, the open, close, and small-area noise removal operations were implemented to remove the isolated small dots and burrs. After that, the hole filling operation was then conducted to fill the holes on the surface of fish, further to obtain the fish-connected area image. MobileNet was selected as the classification network to build an adhesive fish recognition model using depthwise separable convolution, while the transfer learning was adopted to train the model. Two training mechanisms of transfer learning were designed and then optimized, where the better one was selected according to the experimental data. Specifically, two training were as follows: 1) To freeze all convolutional layers, and only perform the rough training on the fully connected layer; 2) After the first step, subsequently to unfreeze the convolutional layer, and perform fine-tuning training on all layers. As such, the adhesive fish was recognized using the well-trained model. The collected dataset of adhesive fish recognition was randomly divided into a training set and a test set at a ratio of 9:1. Additionally, both images were contained for the model training and validation at a ratio of 9:1 in the training set. Correspondingly, the proposed model was trained and then verified in two transfer learning at three learning rates. The results showed that the model quickly converged with the best training accuracy of 100%, the best validation accuracy of 99.60%, and the best testing accuracy of 99.32%, when the transfer learning was combined the rough training of the fully connected layer, and fine-tuning training of all layers, where the 0.000 1 learning rate was set. Furthermore, the recognition accuracy was increased by 5.46 percentage points and 32.29 percentage points respectively, compared with the Support Vector Machine (SVM) and Back Propagation Neural Network (BPNN). Consequently, the proposed deep learning can also be expected to better perform for the adhesive fish recognition, while adaptively adjust the features, according to actual data and the objects. The combined depthwise separable convolution network and transfer learning can be used to improve the running speed of model. Therefore, the new model can also be conveniently applied in smart terminals, such as mobile phones, particularly for the recognition of adhesive fish automatically and in real-time.