Determination of soil moisture content by hyperspectral technology with CARS algorithm

Yu Lei; Zhu Yaxing; Hong Yongsheng; Xia Tian; Liu Muxing; Zhou Yong

doi:10.11975/j.issn.1002-6819.2016.22.019

Volume 32 Issue 22

Nov. 2016

Turn off MathJax

Article Contents

Abstract

References

Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE) > 2016 > 32(22): 138-145. > DOI: 10.11975/j.issn.1002-6819.2016.22.019

Yu Lei, Zhu Yaxing, Hong Yongsheng, Xia Tian, Liu Muxing, Zhou Yong. Determination of soil moisture content by hyperspectral technology with CARS algorithm[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2016, 32(22): 138-145. DOI: 10.11975/j.issn.1002-6819.2016.22.019

Citation:

PDF (2237 KB)

Determination of soil moisture content by hyperspectral technology with CARS algorithm

Yu Lei^1,2,
Zhu Yaxing^1,2,
Hong Yongsheng^1,2,
Xia Tian^1,2,
Liu Muxing^1,2,
Zhou Yong^1,2

1.
Key Laboratory for Geographical Process Analysis & Simulation, Hubei Province, Central China Normal University, Wuhan 430079, China
2.
College of Urban & Environmental Science, Central China Normal University, Wuhan 430079, China

More Information

Received Date: April 28, 2016
Revised Date: October 09, 2016
Published Date: November 14, 2016

Graphical Abstract

Abstract

Abstract

Abstract: Hyperspectral technology is a popular method of predicting soil moisture content nowadays, however, soil spectra include quantities of invalid redundant information, which is a serious bottleneck problem that could lead higher complexity and lower accuracy of prediction model. In this study, 96 fluvo-aquic soil samples were collected at 0-20 cm depth in fields in Qianjiang city, Hubei province, China, and then the samples were pretreated by air-drying, grinding and sieving in a laboratory. Samples with different soil moisture content (SMC, mass fraction of 0-40%) were prepared. For each sample, hyperspectral reflectance was measured by an ASD Field Spec3 instrument. Outliers with abnormal data were removed by Monte Carlo cross validation method. After that, the raw spectral reflectance was processed by Savitzky-Golay smoothing method and continuum removal method. Then, spectra of samples were divided into 2 subsets by Kennard-Stone algorithm. One subset was a calibration set with 47 samples and the other subset was a prediction set with 30 samples. The wavelength variables sensitive (SWV) to SMC were selected from the full-spectrum by competitive adaptive reweighted sampling (CARS) method, and they were considered as an optimal variable set. The multivariate calibrations were performed with partial least squares regression by using the full-spectrum (F-PLSR) and the optimal variables (CARS-PLSR), respectively. The prediction accuracy was assessed by comparing determination coefficients (R2), root mean squared error (RMSE) and relative percent deviation (RPD). Results showed that the SMC greatly affected soil spectral reflectance. The soil spectral reflectance reduced with the SMC increase, and 4 soil moisture absorption peaks were obvious around 450, 1 400, 1 900 and 2 200 nm. The peaks at 1 400 and 1 900 nm had the obvious redshift phenomenon. The SWV of samples with different moisture were obtained by CARS, and then an optimal variables set was generated including wavelengths of 443-449, 1 408-1 456, 1 916-1 943 and 2 209-2 225 nm. Using the CARS-PLSR model calibrated by the optimal variables, the predicting accuracy was improved compared to the F-PLSR model calibrated by the full-spectrum. The predicting accuracy of CARS-PLSR (R2, RMSE and RPD were 0.983, 0.0144 and 8.36, respectively) was higher than F-PLSR model (R2, RMSE and RPD were 0.976, 0.0166 and 7.21, respectively). Meanwhile, compared to F-PLSR model, the CARS-PLSR model reduced the variable numbers from 2 001 to 101. In brief, the CARS-PLSR model not only enhanced the forecasting capability but also reduced the model complexity. Thus, this approach could facilitate the development of soil moisture sensor in field in the near future.
- soil moisture,
- algorithms,
- models,
- hyperspectra,
- competitive adaptive reweighted sampling (CARS),
- variable selection,
- fluvo-aquic soil

FullText(HTML)

References (31)

References

[1]	宋韬，鲍一丹，何勇. 利用光谱数据快速检测土壤含水量的方法研究[J]. 光谱学与光谱分析，2009，29(3)：675－677.Song Tao, Bao Yidan, He Yong. Research on the method for rapid detection of soil moisture content using spectral data[J]. Spectroscopy and Spectral Analysis, 2009, 29(3): 675－677. (in Chinese with English abstract)
[2]	Amani M , Parsian S, Mirmazloumi S M, et al. Two new soil moisture indices based on the NIR-red triangle space of Landsat-8 data[J]. International Journal of Applied Earth Observation and Geoinformation, 2016, 50: 176－186.
[3]	Wang Quan, Li Pingheng, Pu Zhi, et al. Calibration and validation of salt-resistant hyperspectral indices for estimating soil moisture in arid land[J]. Journal of Hydrology, 2011, 408(3/4): 276－285.
[4]	Peng Jian, Shen Hong, He Sanwei, et al. Soil moisture retrieving using hyperspectral data with the application of wavelet analysis[J]. Environmental Earth Science, 2013, 69: 279－288.
[5]	何挺，王静，程烨，等. 土壤水分光谱特征研究[J]. 土壤学报，2006，43(6)：1027－1032.He Ting, Wang Jing, Cheng Ye, et al. Spectral features of soil moisture[J]. Acta Pedologica Sinica, 2006, 43(6): 1027－1032. (in Chinese with English abstract)
[6]	刘秀英，王力，常庆瑞，等. 利用包络线消除法反演黄绵土水分含量[J]. 武汉大学学报·信息科学版，2016，41(1)：1－5.Liu Xiuying, Wang Li, Chang Qingrui, et al. Prediction of moisture content in loess using continuum-removed method[J]. Geomatics and Information Science of Wuhan University, 2016, 41(1): 1－5. (in Chinese with English abstract)
[7]	郝勇，孙旭东，潘圆媛，等. 蒙特卡罗无信息变量消除方法用于近红外光谱预测果品硬度和表面色泽的研究[J]. 光谱学与光谱分析，2011，31(5)：1225－1229.Hao Yong, Sun Xudong, Pan Yuanyuan, et al. Detection of firmness and surface color of pear by near infrared spectroscopy based on Monte Carlo uninformative variables elimination method[J]. Spectroscopy and Spectral Analysis, 2011, 31(5): 1225－1229. (in Chinese with English abstract)
[8]	Blanco M, Coello J, Iturriaga H, et al. NIR calibration in non-linear systems: Different PLS approaches and artificial neural networks[J]. Chemometrics and Intelligent Laboratory Systems, 2000, 50(1): 75－82.
[9]	温珍才，孙通，许朋，等. 可见/近红外联合变量优选检测油茶籽油掺假[J]. 江苏大学学报：自然科学版，2015，36(6)：673－678.Wen Zhencai, Sun Tong, Xu Peng, et al. Adulteration detection of camellia oils by Vis/NIR spectroscopy and variable selection method[J]. Journal of Jiangsu University: Natural Science Edition, 2015, 36(6): 673－678. (in Chinese with English abstract)
[10]	于霜，刘国海，梅从立，等. 基于变量筛选的固态发酵值近红外检测[J]. 计算机与应用化学，2014，31(9)：1143－1146.Yu Shuang, Liu Guohai, Mei Congli, et al. NIRS detection of pH value in solid-state fermentation process based on variable selection method of CARS[J]. Computers and Applied Chemistry, 2014, 31(9): 1143－1146. (in Chinese with English abstract)
[11]	孙通，许文丽，林金龙，等. 可见/近红外漫透射光谱结合CARS变量优选预测脐橙可溶性固形物[J]. 光谱学与光谱分析，2012，32(12)：3229－3233.Sun Tong, Xu Wenli, Lin Qinlong, et al. Determination of soluble solids content in navel oranges by Vis/NIR diffuse transmission spectra combined with CARS method[J]. Spectroscopy and Spectral Analysis, 2012, 32(12): 3229－3233. (in Chinese with English abstract)
[12]	张华秀，李晓宁，范伟，等. 近红外光谱结合CARS变量筛选方法用于液态奶中蛋白质与脂肪含量的测定[J]. 分析测试学报，2010，29(5)：430－434.Zhang Huaxiu, Li Xiaoning, Fan Wei, et al. Determination of proteinand fat in liquid milk by NIR combined with CARS variables screening method[J]. Journal of Instrumental Analysis, 2010, 29(5): 430－434. (in Chinese with English abstract)
[13]	姜微，房俊龙，王树文，等. CARS-SPA 算法结合高光谱检测马铃薯还原糖含量[J]. 东北农业大学学报，2016，47(2)：88－95.Jiang Wei, Fang Junlong, Wang Shuwen, et al. Using CARS-SPA algorithm combined with hyperspectral to determine reducing sugars content in potatoes[J]. Journal of Northeast Agricultural University, 2016, 47(2): 88－95. (in Chinese with Eng-lish abstract)
[14]	张明锦，王明伟. 基于数据分箱的 CARS 方法用于基因表达谱的特征筛选[J]. 计算机与应用化学，2015，32(8)：1004－1006.Zhang Mingjin, Wang Mingwei. Use of Binning-based CARS method for feature selection from gene expression data[J]. Computers and Applied Chemistry, 2015, 32(8): 1004－1006. (in Chinese with English abstract)
[15]	武永峰，董一威，胡新，等. 基于近红外漫反射光谱的农田原位表层土壤含水量定量建模方法比较[J]. 光谱学与光谱分析，2015，35(12)：3416－3421.Wu Yongfeng, Dong Yiwei, Hu Xin, et al. Quantification of agricultural in-situ surface soil moisture content using near infrared diffuse reflectance spectroscopy: a comparison of modeling methods[J]. Spectroscopy and Spectral Analysis, 2015, 35(12): 3416－3421. (in Chinese with English abstract)
[16]	刘洋，丁潇，刘焕军，等. 黑土土壤水分反射光谱特征定量分析与预测[J]. 土壤学报，2014，51(5)：1021－1026.Liu Yang, Ding Xiao, Liu Huanjun, et al. Quantitative analysis of reflectance spectrum of black soil as affected by soil moisture for prediction of soil moisture in black soil[J]. Acta Pedologica Sinica, 2014, 51(5): 1021－1026. (in Chinese with English abstract)
[17]	洪永胜，于雷，耿雷，等. 应用DS算法消除室内几何测试条件对土壤高光谱数据波动性的影响[J]. 华中师范大学学报，2016，50(2)：303－308.Hong Yongsheng, Yu Lei, Geng Lei, et al. Using direct standardization algorithm to eliminate the effect of laboratory geometric parameters in soil hyperspectral data fluctuate characteristic[J]. Journal of Central Normal Unversity, 2016, 50(2): 303－308. (in Chinese with English abstract)
[18]	Xu Qingsong, Liang Yizeng. Monte Carlo cross validation[J]. Chemom Intell Lab Systems, 2001, 56(1): 1－11.
[19]	刘智超，蔡文生，邵学广. 蒙特卡洛交叉验证用于近红外光谱奇异样本的识别[J]. 中国科学，2008，38(4)：316－323.Liu Zhichao, Cai Wensheng, Shao Xueguang. Using Monte Carlo cross-validation to identify the singular sample of near-infrared spectrum[J]. Science in China, 2008, 38(4): 316－323. (in Chinese with English abstract)
[20]	李水芳，单杨，范伟，等. 基于MCCV奇异样本筛选和CARS变量选择法对蜂蜜pH值和酸度的近红外光谱检测[J]. 食品科学，2011，32(8)：182－185.Li Shuifang, Shan Yang, Fan Wei, et al. Analysis of pH and acidity of honcy by near infrared spectroscopy based on MCCV outlier detection and CARS variable selection[J]. Food Science, 2011, 32(8): 182－185. (in Chinese with English abstract)
[21]	刘翠玲，孙晓荣，吴静珠，等. 基于NIR的小麦粉异常样本剔除方法研究[J]. 农机化研究，2014(4)：46－49.Liu Cuiling, Sun Xiaorong, Wu Jingzhu, et al. Study on elimination of abnormal wheat powder samples based on NIR[J]. Journal of Agricultural Mechanization Research, 2014 (4): 46－49. (in Chinese with English abstract)
[22]	谢伯承，薛绪掌，刘伟东，等. 基于包络线法对土壤光谱特征的提取及其分析[J]. 土壤学报，2005，42(1)：171－175.Xie Bocheng, Xue Xuzhang, Liu Weidong, et al. Hull-curve-method-based extraction and analysis of soil spectral characteristics[J]. Acta Pedologica Sinica, 2005, 42(1): 171－175. (in Chinese with English abstract)
[23]	Kennard R W, Stone L A. Computer aided design of experiments[J]. Technometrics, 1969, 11(1): 137－148.
[24]	李华，王菊香，邢志娜，等. 改进的K/S算法对近红外光谱模型传递影响的研究[J]. 光谱学与光谱分析，2011，31(2)：362－365.Li Hua, Wang Juxiang Xing Zhina, et al. Influence of improved Kennard/Stone algorithm on the calibration transfer in Near-Infrared spectroscopy[J] Spectroscopy and Spectral Analysis, 2011, 31(2): 362－365. (in Chinese with English abstract)
[25]	肖海斌，赵进辉，袁海超，等. 同步荧光光谱结合CARS变量优选预测猪肉中四环素残留含量[J]. 光学精密工程，2013，21(10)：2513－2519.Xiao Haibin, Zhao Jinhui, Yuan Haichao, et al. Determination of tetracycline content in pork by synchronous fluorescence spectroscopy with CARS method[J]. Optics and Precision Engineering, 2013, 21(10): 2513－2519. (in Chinese with English abstract)
[26]	Jiang Hui, Zhang Hang, Chen Quansheng, et al. Identi?cation of solid state fermentation degree with FT-NIR spectroscopy: Comparison of wavelength variable selection methods of CARS and SCARS[J]. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2015, 149: 1－7.
[27]	于雷，洪永胜，耿雷，等. 基于偏最小二乘回归的土壤有机质含量高光谱估算[J]. 农业工程学报，2015，31(14)：103－109.Yu Lei, Hong Yongsheng, Geng Lei et al. Hyperspectral estimation of soil organic matter content based on partial least squares regression[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2015, 31(14): 103－109. (in Chinese with English abstract)
[28]	彭杰，周清，张杨珠，等. 有机质对土壤光谱特性的影响研究[J]. 土壤学报，2013，50(3)：517－524.Peng Jie, Zhou Qing, Zhang Yangzhu, et al. Effect of soil organic matter on spectral characteristics of soil[J]. Acta Pedologica Sinica, 2013, 50(3): 517－524. (in Chinese with English abstract)
[29]	Guo Ying, Ni Yongnian, Kokot S. Evaluation of chemical components and properties of the jujube fruit using near infrared spectroscopy and chemometrics[J]. Spectrochimica Acta Part A: Molecular and Biomolecular Spectroscopy, 2016, 153(15): 79－86.
[30]	Gomez C, Lagacherie P, Coulouma G. Continuum removal versus PLSR method for clay and calcium carbonate content estimation from laboratory and airborne hyperspectral measurements[J]. Geoderma, 2008, 148(2): 141－148.
[31]	彭小婷，高文秀，王俊杰. 基于包络线去除和偏最小二乘的土壤参数光谱反演[J]. 武汉大学学报·信息科学版，2014，39(7)：862－866.Peng Xiaoting, Gao Wenxiu, Wang Junjie. Inversion of soil parameters from hyperspectra based on continuum removal and partial least squares regression[J]. Geomatics and Information Science of Wuhan University, 2014, 39(7): 862－866. (in Chinese with English abstract)

[1]	NIE Haoliang, YANG Junfang, YANG Yunma, HUANG Shaohui, ZHANG Jing, WANG Jingxia, YANG Huimin, YANG Wenfang, XING Suli, JIA Liangliang. Effects of long-term deep tillage, straw returning and nutrient management on organic carbon mineralization of fluvo-aquic soil[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2024, 40(18): 70-80. DOI: 10.11975/j.issn.1002-6819.202406027
[2]	WANG Shuo, SUN Mengyu, LI Changqing, SUN Zhimei. Transformation characteristics of different nitrogen fertilizers and the regulatory effect of ATS on urea-N transformation in calcareous-fluvo-aquic soil[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(24): 100-108. DOI: 10.11975/j.issn.1002-6819.202309155
[3]	WANG Fei'er, LIN Shan, ZHANG Xiuling, SHEN Yaoyao, ZHANG Shuiqing, JIANG Yanyun, HU Ronggui. Effects of microplastic input and straw addition on nitrogen leaching in fluvo-aquic soil and yellow-brown soil[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2023, 39(23): 94-102. DOI: 10.11975/j.issn.1002-6819.202307293
[4]	Liu Juan, Zhang Naiming, Zhang Shuxiang. Accumulative loss risk of phosphorus and its environmental threshold in fluvo-aquic soil[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2020, 36(20): 8-16. DOI: 10.11975/j.issn.1002-6819.2020.20.002
[5]	Song Wen, Zhang Min, Wu Kening, Li Li, Zhao Huafu, Li Junying. Test on nondestructive detection of farmland solum structure in fluvo-aquic soil area using ground penetrating radar[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2018, 34(16): 129-138. DOI: 10.11975/j.issn.1002-6819.2018.16.017
[6]	Cai Lianghong, Ding Jianli. Wavelet transformation coupled with CARS algorithm improving prediction accuracy of soil moisture content based on hyperspectral reflectance[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2017, 33(16): 144-151. DOI: 10.11975/j.issn.1002-6819.2017.16.019
[7]	Guo Doudou, Huang Shaomin, Zhang Shuiqing, Nie Shengwei. Comparative analysis of various hyperspectral prediction models of fluvo-aquic soil organic matter[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2014, 30(21): 192-200. DOI: 10.3969/j.issn.1002-6819.2014.21.023
[8]	Gong Fufei, Zha Yan, Wu Xueping, Huang Shaomin, Xu Minggang, Zhang Huimin, Liu Hailong, Jiang Zhiwei, Wang Xiaobin, Cai Dianxiong. Analysis on basic soil productivity change of winter wheat in fluvo-aquic soil under long-term fertilization[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2013, 29(12): 120-129.
[9]	Liao Qinhong, Gu Xiaohe, Li Cunjun, Chen Liping, Huang Wenjiang, Du Shizhou, Fu Yuanyuan, Wang Jihua. Estimation of fluvo-aquic soil organic matter content from hyperspectral reflectance based on continuous wavelet transformation[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2012, 28(23): 132-139.
[10]	Wu Man, Liu Junling, Xu Minggang, Zhang Wenju, Wu Haiwen. Effects of organic matter on natural-remediation process of typical copper-zinc polluted soils[J]. Transactions of the Chinese Society of Agricultural Engineering (Transactions of the CSAE), 2011, 27(14): 211-217.