Abstract: Solar greenhouses have been widely used in modern precision agriculture in China. However, potential negative environmental impacts can also be introduced, including atmospheric pollution, soil contamination, and water pollution. Great challenges have been posed on the rational management of farmlands and resource utilization. Meanwhile, the manual collection of visual data cannot fully meet the rapid acquisition of information from large-scale greenhouse facilities. In this study, Chinese solar greenhouses (CSGs) were focused on detecting the damage levels from the remote sensing images using an improved YOLO-DLSG (YOLO-damage level of solar greenhouse) object detection model. Comparative experiments were conducted using high-resolution solar greenhouse remote sensing image datasets. A dataset was collected with the 3235 images of solar greenhouses and then subjected to data augmentation. The classification standards were also combined with the damage indicators specific to solar greenhouses. The damage levels were then established from the high-resolution remote sensing satellite images. The YOLOv5s lightweight improvement was employed to reconstruct the YOLOv5s backbone with the lightweight MobileNetV3 architecture. This restructure reduced the channel dimensions and network parameter volume. Furthermore, the upsampling operator in the neck layers was replaced and then updated. A lightweight upsampling operator was selected as CARAFE (content-aware reassembly feature embedding), using an upsampled kernel prediction module. The feature recombination module was reconstructed in the upsample structure. EVCBlock (explicit visual center block) was utilized to further reduce the computational complexity for the high accuracy of detection on rotating objects. Finally, the CIoU (complete intersection over union) loss function for the target bounding box regression was replaced with the EIoU (Enhanced Intersection over Union) loss, in order to accelerate the convergence of the prediction box regression loss function for the high accuracy of prediction box regression. The experimental results show that the improved model reduced the number of parameters and floating-point operations per second by 17.91 and 15.19 percentage points, with an increase of 0.4 and 0.8 percentage points in the precision and mean average precision, respectively. Compared with the current mainstream lightweight models, such as SSD, Faster R-CNN, Centernet, YOLOv3-spp, and YOLOv3-tiny, the improved YOLO-DLSG model demonstrated superior performance, in terms of accuracy, recall, detection speed, and model lightweight. Data collection and accuracy validation were conducted in the Yangling Agricultural Demonstration Zone in Shaanxi Province of China. An average recognition accuracy of 84.00% was achieved in the improved model. A challenging task was to determine the distribution, quantity, and extent of greenhouse damage in a cost-effective, efficient, and user-friendly manner. This finding can provide a simple and effective way for the rapid location and automatic quantity extraction of solar greenhouses with different damage levels.