• EI
    • CSA
    • CABI
    • 卓越期刊
    • CA
    • Scopus
    • CSCD
    • 核心期刊
Hu Shaojun, Geng Nan, Zhang Zhiyi, Yang Pei, He Dongjian. Interactive modeling method of outdoor trees based on sparse images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(9): 168-175. DOI: 10.3969/j.issn.1002-6819.2014.09.021
Citation: Hu Shaojun, Geng Nan, Zhang Zhiyi, Yang Pei, He Dongjian. Interactive modeling method of outdoor trees based on sparse images[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(9): 168-175. DOI: 10.3969/j.issn.1002-6819.2014.09.021

Interactive modeling method of outdoor trees based on sparse images

More Information
  • Received Date: August 08, 2013
  • Revised Date: March 22, 2014
  • Published Date: April 30, 2014
  • Abstract: The creation of realistic outdoor trees is a challenging problem in the area of modeling natural phenomenon because trees have complex geometric structures. Currently, there are two major methods to achieve this goal. One approach models trees from image sequences. This approach requires more than 16 images and wide view angles to reconstruct tree point clouds and camera pose, and the reconstruction process is not easily implemented for non-expert users because of some complex computer vision techniques. The other approach uses a laser range scanner to acquire a point cloud for modeling trees. However, we need expensive hardware to obtain the point cloud. Furthermore, the background segmentation, the hole filling and the registration process of a point cloud is very cumbersome. In this paper, we present a low cost interactive modeling method to reconstruct real-world trees from sparse images. Our method has the advantage of preserving the branch structures of real trees with few images and limited viewing angle. Based on two input photographs taken from different views with the coverage of 90 degrees and 4 to 7 in-between photographs, we developed an interactive editing system to extract the node positions, the thickness of branches and the tree hierarchy from the front view image. The interactive editing system consists of branch (or node) drawing and modifying, branch (or node) inserting and deleting, thickness modifying, Hermite spline interpolation and tree hierarchy reconstruction parts. Next, we chose main branches as references, and interactively matched the corresponding branches from the side view image by making use of the in-between images. Then, we adapted the node positions using the editing system to obtain the depth information for each branch. By combining the extracted two dimensional node positions and the thickness of branch from the front view and the depth information from the side view, we drew the three dimensional tree using generalized cylinders. However, the reconstructed tree model showed distortion where the branches in the distance appeared smaller and the branches at close range appeared larger compared with the branches in the photograph. It can be explained by the double perspective projection phenomenon where the real-world objects have been transformed twice through taking photographs and through perspective transformation in OpenGL. We propose a perspective calibration method to avoid the distortion of reconstructed tree models. Leaves are difficult to be identified from images even by the interactive method. Thus, we designed a leaf arrangement algorithm and added leaves to each branch according to leaf phyllotaxis. Finally, we demonstrated the realistic reconstruction of a variety of tree species including apple trees, cherry trees and maple trees. The number of nodes of the reconstructed trees ranges from 736 to 1250, and the average reconstruction time is around 80 minutes for a medium scale tree. The result showed that our method is effective to model real world trees having clear branches and sparse leaves.
  • [1]
    熊俊涛,邹湘军,陈丽娟,等. 采摘机械手对扰动荔枝的视觉定位[J]. 农业工程学报,2012,28(14):36-41.Xiong Juntao, Zou Xiangjun, Chen Lijuan, et al. Visual position of picking manipulator for disturbed litchi[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2012, 28(14): 36-41. (in Chinese with English abstract)
    [2]
    Jimenez A R, Ceres R, Pons J L. A survey of computer vision methods for locating fruit on trees[J]. Transaction of the ASAE, 2000, 43(6): 1911-1920.
    [3]
    Wilson P A, Chakrabrty S. The virtual plant: A new tool for the study and management of plant diseases[J]. Crop Protection, 1998, 17(3): 231-239.
    [4]
    Hua J, Kang M Z, de Reffye P. An Interactive plant pruning system based on GreenLab model: Implementation and case study[C]// Proceedings of 2011 IEEE International Conference on Computer Science and Automation Engineering, 2011: 185-188.
    [5]
    Prusinkiewicz P, Lindenmayer A. The algorithmic beauty of plants[M]. New York : Springer-Verlag Press, 1990.
    [6]
    Deussen O, Linterman B. Digital design of nature: Computer generated plants and organics[M]. New York: Springer-Verlag Press, 2005.
    [7]
    Talton J, Lou Y, Lesser S, et al. Metropolis procedural modeling[J]. ACM Trans. Graph., 2011, 30(2): 11-24.
    [8]
    Okabe M, Owada S, Igarashi T. Interactive design of botanical trees using freehand sketches and example based editing[J]. Comput. Graph. Forum, 2005, 24(3): 487-496.
    [9]
    Palubicki W, Horel K, Longay S, et al. Self-organizing tree models for image synthesis[J]. ACM Trans. Graph., 2009, 28(3): 1-10.
    [10]
    Alex R, Martin I, Drettakis G. Volumetric reconstruction and interactive rendering of trees from photographs[J]. ACM Trans. Graph., 2004, 23(3): 720-727.
    [11]
    Neubert B, Franken T, Deussen O. Approximate image- based tree-modeling using particle flows[J]. ACM Trans. Graph., 2007, 26(3): 88-95.
    [12]
    Quan L, Tan P, Zeng G, et al. Image-based plant modeling[J]. ACM Trans. Graph., 2006, 25(3): 599-604.
    [13]
    Tan P, Zeng G, Wang J, et al. Image-based tree modeling[J]. ACM Trans. Graph., 2007, 26(3): 87-93.
    [14]
    Xu H, Gossett N, Chen B. Knowledge and heuristic- based modeling of laser-scanned trees[J]. ACM Trans. Graph., 2007, 26(4): 19-31.
    [15]
    Livny Y, Yan F, Olson M, et al. Automatic reconstruction of tree skeletal structures from point clouds[J]. ACM Trans. Graph., 2010, 29(6): 151-158.
    [16]
    Livny Y, Pirk S, Cheng Z, et al. Texture-lobes for tree modeling[J]. ACM Trans. Graph., 2011, 30(4): 53-62.
    [17]
    Zhang Zhiyi, Yuan Lin. Building a 3D scanner system based on monocular vision[J]. Applied Optics, 2012, 51(11): 1638-1644.
    [18]
    胡包钢,赵星,严红平,等. 植物生长建模于可视化-回顾与展望[J]. 自动化学报,2001,27(6):816-835.Hu Baogang, Zhao Xing, Yan Hongping, et al. Plant growth modeling and visualization-Review and perspective[J]. Acta Automatica Sinica, 2001, 27(6): 816-835. (in Chinese with English abstract)
    [19]
    曹宏鑫,石春林,金之庆. 植物形态结构模拟与可视化研究进展[J]. 中国农业科学,2008,41(3):669-677.Cao Hongxin, Shi Chunlin, Jin Zhiqing. Advances in researches on plant morphological structure simulation and visualization[J]. Scientia Agricultura Sinica, 2008, 41(3): 669-677. (in Chinese with English abstract)
    [20]
    赵春江,陆声链,郭新宇,等. 数字植物及其技术体系探讨[J]. 中国农业科学,2010,43(10):2023-2030.Zhao Chunjiang, Lu Shenglian, Guo Xinyu, et al. Exploration of digital plant and its technology system[J]. Science Agricultura Sinica, 2010, 43(10): 2023-2030. (in Chinese with English abstract)
    [21]
    屈洪春,朱庆生,蔡林沁,等. 基于多尺度分析的虚拟植物分枝结构三维重建[J]. 系统仿真学报,2009,21(15):4718-4721.Qu Hongchun, Zhu Qingsheng, Cai Linqin, et al. Multiscales analysis-based branching structure 3D reconstruction for virtual plants[J]. Journal of System Simulation, 2009, 21(15): 4718-4721. (in Chinese with English abstract)
    [22]
    温维亮,郭新宇,陆声链,等. 隐式曲面在三维植物建模中的应用研究综述[J]. 工程图学学报,2010,31(3):1-10.Wen Weiliang, Guo Xinyu, Lu Shenglian, et al. A survey on applications of implicit surfaces to 3D plant modeling[J]. Journal of Engineering Graphics, 2010, 31(3): 1-10. (in Chinese with English abstract)
    [23]
    丁维龙,胡辰,程志君,等. 基于基因表达式编程的植物形态建模智能化方法[J]. 农业工程学报,2013,29(1):134-141.Ding Weilong, Hu Chen, Cheng Zhijun, et al. Intelligent modeling method for plant morphology based on gene expression programming[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2013, 29(1): 134-141. (in Chinese with English abstract)
    [24]
    郭浩,戈振扬,蒋海波,等. 基于体着色的植物构型三维重建和可视化模拟[J]. 农业工程学报,2010,26(10):195-200.Guo Hao, Ge Zhenyang, Jiang Haibo, et a1. Three dimensional reconstruction and visual modeling of plant architecture based on voxel coloring[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2010, 26(10): 195-200. (in Chinese with English abstract)
    [25]
    魏学礼,肖伯祥,郭新宇,等. 三维激光扫描技术在植物扫描中的应用分析[J]. 中国农学通报,2010,26(20):373-377.Wei Xueli, Xiao Boxiang, Guo Xinyu, et al. Analysis of applications of 3D laser scan technology in plant scanning[J]. Chinese Agricultural Science Bulletin, 2010, 26(20): 373-377. (in Chinese with English abstract)
    [26]
    方慧,胡令潮,何任涛,等. 植物三维信息采集方法的研究[J]. 农业工程学报,2012,28(3):142-147.Fang Hui, Hu Lingchao, He Rentao, et al. Research on plant three-dimensional information acquisition method[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2012, 28(3): 142-147. (in Chinese with English abstract)
    [27]
    袁晓敏,赵春江,温维亮,等. 番茄植株三维形态精确重构研究[J]. 农业机械学报,2012,43(12):204-210.Yuan Xiaomin, Zhao Chunjiang, Wen Weiliang et al. Detailed modeling of 3D configuration of tomato plant[J]. Transactions of the CSAM, 2012, 43(12): 204-210. (in Chinese with English abstract)
    [28]
    Godin C, Caraglio Y. A multiscale model of plant topological structures[J]. J. theor. Biol., 1998, 191(1): 1-46.
    [29]
    Comaniciu D, Meer P. Mean Shift: A robust approach toward feature space analysis[J]. IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24(5): 603-618.
    [30]
    Furukawa Y, Ponce J. Accurate, dense, and robust multi-view stereopsis[J]. IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32(8): 1362-1376.
    [31]
    Yilmaz A, Javed O, Shah M. Object tracking: A survey[J]. ACM Comput. Surv., 2006, 38(4): 1-45.
    [32]
    Yang H, Shao L, Zheng F, et al. Recent advances and trends in visual tracking: A review[J]. Neurocomputing, 2011, 74(18): 3823-3831.
  • Related Articles

    [1]LINGHU Kerui, TANG Lixia, WANG Yimin, LU Yang, ZHANG Zengliang, RAN Kui, HAN Zhen. Influence of coconut fiber meshes on the scouring characteristics of engineering accumulation slopes[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2025, 41(6): 128-136. DOI: 10.11975/j.issn.1002-6819.202411254
    [2]FAN Chao, WU Qiuping, ZHANG Guohui, ZENG Jiwu. Construction and test of mango fruit shape identification platform based on 3D point cloud data[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2024, 40(22): 290-296. DOI: 10.11975/j.issn.1002-6819.202401112
    [3]Fu Yuxing, Li Chengming, Zhu Jiang, Wang Baolong, Zhang Bin, Fu Wei. Three-dimensional model construction method and experiment of jujube tree point cloud using Alpha-shape algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(22): 214-221. DOI: 10.11975/j.issn.1002-6819.2020.22.024
    [4]Xia Chunhua, Shi Ying, Yin Wenqing. Obtaining and denoising method of three-dimensional point cloud data of plants based on TOF depth sensor[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(6): 168-174. DOI: 10.11975/j.issn.1002-6819.2018.06.021
    [5]Wu Sheng, Zhao Chunjiang, Guo Xinyu, Wen Weiliang, Xiao Boxiang, Wang Chuanyu. Method of fruit tree canopy leaf reconstruction based on point cloud[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(z1): 212-218. DOI: 10.11975/j.issn.1002-6819.2017.z1.032
    [6]Li Xuezeng, Huang Yanhe, Lin Jinshi, Jiang Fangshi, Hao Fuxing, Guan Jiachun, Xu Huangji, Yang Dandan, Wang Zhenhong. Effects of different width of scouring flumes on runoff and sediment yield of colluvial deposits of collapsing hill[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(9): 136-141. DOI: 10.11975/j.issn.1002-6819.2016.09.019
    [7]Shao Qing, Xu Tao, Yoshino Tatsuo, Song Nan, Zhu Hang. Design and experiment for grain storage monitoring system based on 3-D laser scanning technology[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(20): 262-267. DOI: 10.11975/j.issn.1002-6819.2015.20.036
    [8]Sun Zhihui, Lu Shenglian, Guo Xinyu, Wen Weiliang. Surfaces reconstruction of plant leaves based on point cloud data[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2012, 28(3): 184-190.
    [9]Wang Wenlong, Li Zhanbin, Li Peng, Lu Kexin, Guo Yanbiao, Ding Wenfeng, Zheng Liangyong. Scouring experiment of on-sites on undisturbed lands of Shenfu-Dongsheng coalfield[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2005, 21(13): 59-62.
    [10]Li Mian, Yao Wenyi, Chen Jiangnan, Ding Wenfeng, Yang Jianfeng, Li Li, Yang Chunxia. Experimental study on the effect of different grass coverages on the overland flow velocity[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2005, 21(12): 43-47.

Catalog

    Article views (4507) PDF downloads (1518) Cited by()
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return