Abstract:
Walnut trees are characterized by their tall stature, numerous branches, and complex structures, making it challenging to directly determine the spatial positions of pruning points. Pruning points are typically determined based on the branch bifurcation points, along with branch types and corresponding pruning amounts to establish specific locations. Therefore, accurately locating the bifurcation points of walnut tree branches is a critical aspect of ensuring precise pruning point localization. This study proposes a method for locating branch bifurcation points and selecting pruning positions for walnut trees, based on an improved UNet model and image processing techniques.First, data collection, augmentation, and enhancement were conducted, followed by detailed annotation of walnut tree trunks and branches to construct the dataset. The dataset was divided into training and validation sets in a 9:1 ratio, containing 3168 and 352 images, respectively. Subsequently, a semantic segmentation model suitable for this dataset was selected. Specifically, the Multidimensional Collaborative Attention (MCA) module was integrated into the VGG16-UNet architecture to fuse detailed feature outputs with the corresponding layer’s decoding operations. The model, named VMfd-UNet, employed focal loss and dice loss as loss functions for the training and validation sets, respectively.Experimental results demonstrated that VMfd-UNet achieved mean pixel accuracy (mPA) and mean intersection over union (mIoU) on the entire dataset that were higher by 4.86 and 4.85 percentage points, respectively, compared to the VGG16-UNet model. The VMfd-UNet performed exceptionally well on the validation set and outperformed other models utilizing different attention mechanisms. The mPA of the trunk and branches reached 96.71% and 90.27%, while the mIoU values were 90.42% and 79.86%, respectively. The model’s parameter count was 35.73M, and its computational cost was 156.93G. The improved model demonstrates excellent detection performance and robustness, capable of better segmenting walnut tree trunks and branches under natural environmental conditions.Next, the VMfd-UNet model was utilized to segment tree trunks and branches, generating corresponding masks. A mask refinement algorithm was used to extract the skeleton of the masks, followed by the use of an 8-neighborhood traversal algorithm on the skeleton graph to identify all skeleton intersection points. The depth distribution around the skeleton intersection points was analyzed, and the range of depth value changes was calculated based on the set of neighboring pixel depth values. Combined with gradient variation analysis, pseudo-bifurcation points were precisely filtered out, leaving only actual branch bifurcation points.Finally, considering the uneven spatial distribution of walnut tree branch bifurcation points, pruning positions were selected from branch bifurcation points in areas of higher spatial density. A sliding window method was used to detect the density of branch bifurcation points, followed by the application of the K-means clustering algorithm to cluster the 3D coordinates of bifurcation points within the window. The Euclidean distance was calculated to group closely located bifurcation points, and the centroid of each cluster was determined. The branch bifurcation point closest to the centroid was selected as the pruning position. A comparative visualization was conducted between the pruning positions identified by this method and manually marked pruning positions.Experimental analysis involving 50 randomly selected walnut tree images was conducted to statistically compare the pruning positions identified using this method with manually marked positions. The proposed method achieved an average localization accuracy of 83.2%, validating its effectiveness and practicality. This study provides a reference for accurately locating pruning points in walnut trees, as well as technical support for such applications.