Abstract: Soil-dwelling termites are highly destructive pests that pose significant threats to hydraulic engineering structures and garden trees. These pests are notorious for causing extensive damage. Their activities were often characterized by mud covering, mud tunnel, and swarming hole. Early and accurate identification of these activity signs is essential for implementing timely and effective termite damage early warning systems and control measures. However, detecting the activity signs of soil-dwelling termites is a challenging task, primarily due to the complexity of natural environments, diverse background interference, and the frequent difficulty in distinguishing termite activity signs from their surrounding environment, particularly in cases where the colors of the signs and the background blend together. To address these practical challenges, this study proposed an advanced one-stage object detection algorithm named ACP-YOLOv5s. This algorithm was an improved version of the widely used YOLOv5s model and had been specifically optimized to enhance feature extraction and color perception capabilities in complex natural environments. The core innovation lied in the integration of an adaptive color perception module (ACP-Module) along with a dynamically adjustable threshold learning mechanism. The ACP-Module intelligently analyzed the color distribution of input images and dynamically determined the optimal color threshold range. This allowed the model to automatically adjust its sensitivity to colors based on varying image contents. This mechanism effectively mitigated detection instability caused by color confusion, significantly enhancing the stability and generalization capabilities of the model. As a result, the model performed exceptionally well even in complex scenarios with high levels of environmental noise and interference.To further enhance detection accuracy, a CARFE upsampling module was incorporated into the neck structure of the model. This module played a pivotal role in expanding the receptive field and reorganizing feature information, enabling the model to detect finer image details with a greater precision. The design of the CARFE module greatly enhanced feature fusion and transmission, which was critical to improve detection accuracy in real-world applications. This optimization not only boosted the model’s performance across various environmental conditions but also significantly improved its ability to detect subtle termite activity signs, such as mud covering, mud tunnel, and swarming hole. Extensive experimental validation of the ACP-YOLOv5s model demonstrated its superiority in detecting soil-dwelling termite activity signs. The model achieved a remarkable precision of 91.2%, outperforming several state-of-the-art models, including Faster R-CNN, YOLOv5s, YOLOv5m, YOLOv8, and YOLOv9, by 5.3, 5.0, 3.4, 7.9, and 0.1 percentage points, respectively. Furthermore, the model attained a mean average precision (mAP50) of 92.3%, representing improvements of 6.7, 2.9, 1.4, 2.2, and 0.4 percentage points over other models. These results underscored the effectiveness of ACP-YOLOv5s in enhancing detection accuracy and adaptability to complex environmental conditions. The ACP-YOLOv5s model represents a significant step forward in object detection technologies for pest management. Its development offers a powerful tool for identifying and mitigating soil-dwelling termite activity, providing substantial benefits for hydraulic engineering and garden tree maintenance. By ensuring the timely detection and precise control of termite infestations, this model makes meaningful contributions to infrastructure safety, ecological preservation, and the advancement of pest control technologies.