Generating STICS XML files from tabulated data

library(SticsRFiles)

Introduction

This document has been generated using the latest STICS version V10.0.

Package functions to be used

There is not at the moment an integrated function which is able to produce all needed parameters files. Individual functions are dedicated to each parameter file kind and may be called individually or gathered in a single script for doing all tasks at once. Use examples are given in next parts below.

Getting a table file example

Getting inputs Excel example file from the SticsRFiles package

In the following example /path/to/target/dir must be replaced with a real path depending on the operating system (i.e. starting with c:/ for windows for example)

# Getting the file in the current directory
usm_xl_file <- download_usm_xl()

# Getting the file in a specic directory
xl_dir <- "/path/to/xl/dir" # or something like C:/path/to/xl/dir" for Windows

usm_xl_file <- download_usm_xl(out_dir = xl_dir)

#> [1] "inputs_stics_example.xlsx  has been copied in directory
#> "/path/to/xl/dir
#> Warning in download_usm_xl(file = "inputs_stics_example.xlsx", out_dir = workspace_path): Error copying files:
#> /tmp/RtmpU3Shko/csv/V10.0/inputs_stics_example.xlsx

Getting an USMs inputs CSV example file from the SticsRFiles package

# Getting the file in the current directory
usm_csv_file <- download_usm_csv(file = "inputs_stics_example_USMs.csv")

# Getting the file in a specific directory
csv_dir <- "/path/to/csv/dir" # or something like C:/path/to/xl/dir" for Windows

usm_csv_file <- download_usm_csv(file = "inputs_stics_example_USMs.csv",
                                 out_dir = csv_dir)

#> [1] inputs_stics_example_USMs.csv  has been copied in directory
#> "/path/to/csv/dir
#> Warning in download_usm_xl(file = csv_name, out_dir = dest_dir, stics_version = version_name, : Error copying files:
#> /tmp/RtmpU3Shko/csv/V10.0/inputs_stics_example_USMs.csv

Description of tables expected content and use

As told in the previous section, parameters data may be stored either in Excel files containing several sheets or in several equivalent individual CSV files.

Sheets content and CSV files content is the same, only their use is different. But, CSV files are easily traced under version control systems and they also can be generated programmatically with the help of functions dedicated for example to generating crop management parameters like irrigation, fertilization (interventions and doses calculations).

Important information

The USMs data which are necessary for generating the usms.xml must be all provided either through an Excel sheet or a CSV file; as shown in the USMs sheet example.

Other data stored in Excel or CSV files must be adapted to the use cases here after:

Specificities of Excel sheets

Excel sheet names are completely free, but must be meaningful regarding the kind of parameters set they contain.

In the example, useful sheets for generating XML parameters files are named: USMs, Ini, Soils, Tec, Station

Common tables specifications

The first column is used for storing identifiers or files names (or prefix) and the USMs table contains identifiers of the other linked tables in columns names finit, nomsol, fstation, ftec_1, ftec_2 with respect to initialization files, soil name, station files and crop management files (for main crops and optionally associated crops).

The first column names of data tables are automatically detected and must contain keywords prefix to do so as listed: usm_, ini_, soil_, sta_, tec_

These first columns may contain either names identifiers or files names according to data type:

Other columns of tables are named with parameters names according to the following rules:

Here is the actual correspondence table of keywords to be used in parameters tables (i.e excel files for example) in replacement of real names in XML files:

As an example, looking in the Tec sheet of the previously downloaded Excel file, irrigation parameters for setting days and amount of water supplied, are respectively named julapI_1, julapI_2,… and doseI_1, doseI_2,… instead of using absolute_value/% and amount.

Data consistency checks in tables

For the moment, no consistency checks (between Excel sheets or CSV equivalents files) functionalities have been included in the package in order to prevent missing generated files or misspelled file names or identifiers.

So, one must take care of links between xml files names for example files names defined for usms related to names declared for example in Excel sheets Tec, Station, and so on.

In future developments, checks will be included to run be run prior to files generations, for preventing errors.

USMs table example

The Excel file which contains parameters values to take into account for STICS inputs may contain specific sheets for each kind of parameters groups, i.e. USMs, crop management, soil, weather station, initializations (except plant parameters and general parameters).

The following table contains data extracted from a sheet named USMs containing a list of USMS parameters or from his equivalent CSV example file.

usm_name datedebut datefin finit nomsol fstation fclim1 fclim2 culturean nbplantes codesimul fplt_1 ftec_1 flai_1 fobs_1 fplt_2 ftec_2 flai_2 fobs_2
USM_2017_T1_CI 199 263 USM_2017_T1_ini.xml USM_T1 climatex_sta.xml climatex.2017 climatex.2017 1 1 0 rapeseed_plt.xml USM_2017_T1_CI_tec.xml USM_2017_T1_CI.obs
USM_2018_T1 264 570 USM_2017_T1_ini.xml USM_T1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 wheat_plt.xml USM_2018_T1_tec.xml USM_2018_T1.obs
BIN_CAN_05_SEC_220-0-0_34K_CANPC05T3_Q 199 263 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2017 2 1 0 proto_potato_plt.xml BIN_CAN_05_SEC_220-0-0_34K_CANPC05T3_Q_tec.xml
BIN_AGT_04_IRR_220-0-0_33K_AGT04T2_Q 264 570 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 proto_potato_plt.xml BIN_AGT_04_IRR_220-0-0_33K_AGT04T2_Q_tec.xml
AGA_ARB_13_IRR_220-0-0_37K_ARB13_C 199 263 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2017 2 1 0 proto_potato_plt.xml AGA_ARB_13_IRR_220-0-0_37K_ARB13_C_tec.xml
AGA_ARB_13_SEC_220-0-0_37K_ARB13_C 264 570 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 proto_potato_plt.xml AGA_ARB_13_SEC_220-0-0_37K_ARB13_C_tec.xml
FRA_ARB_11_SEC_220-0-0_38K_E 199 263 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2017 2 1 0 proto_potato_plt.xml FRA_ARB_11_SEC_220-0-0_38K_E_tec.xml
MAG_ARB_09_SEC_220-0-0_38K_E 264 570 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 proto_potato_plt.xml MAG_ARB_09_SEC_220-0-0_38K_E_tec.xml
MAG_ARV_12_IRR_220-0-0_36K_ARV12_C 199 263 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2017 2 1 0 proto_potato_plt.xml MAG_ARV_12_IRR_220-0-0_36K_ARV12_C_tec.xml
MAG_ARV_12_SEC_220-0-0_36K_ARV12_C 264 570 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 proto_potato_plt.xml MAG_ARV_12_SEC_220-0-0_36K_ARV12_C_tec.xml
FRA_ARB_12_SEC_220-0-0_31K_ARB12_C 199 263 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2017 2 1 0 proto_potato_plt.xml FRA_ARB_12_SEC_220-0-0_31K_ARB12_C_tec.xml
FRA_ARB_13_SEC_220-0-0_37K_ARB13_C 264 570 Vill11_ini.xml LF1 climatex_sta.xml climatex.2017 climatex.2018 2 1 0 proto_potato_plt.xml FRA_ARB_13_SEC_220-0-0_37K_ARB13_C_tec.xml

Generating the usms XML file (usms.xml)

The gen_usms_xml function is working on a data.frame object that can be loaded either from an Excel file sheet or from a csv file. Here after code shows how to get data from the downloaded examples Excel and CSV files.

Getting data

# Reading the Excel file
usms_param <- read_params_table(usm_xl_file, sheet_name = "USMs")
# Or
# Reading the CSV file
usms_param <- read_params_table(usm_csv_file)

Generating the file


# Output file path
out_file <- "/path/to/file/usms.xml"
# or something like C:/path/to/file/usms.xml" for Windows

# Generating a new usms.xml file, for all xl_param lines
gen_usms_xml(file = out_file, param_df = usms_param)

usms.xml file content

Example of the first usm parameters:

#> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
#> <usms version="10.0">
#>   <usm nom="USM_2017_T1_CI">
#>     <datedebut>199</datedebut>
#>     <datefin>263</datefin>
#>     <finit>USM_2017_T1_ini.xml</finit>
#>     <nomsol>USM_T1</nomsol>
#>     <fstation>climatex_sta.xml</fstation>
#>     <fclim1>climatex.2017</fclim1>
#>     <fclim2>climatex.2017</fclim2>
#>     <culturean>1</culturean>
#>     <nbplantes>1</nbplantes>
#>     <codesimul>0</codesimul>
#>     <plante dominance="1">
#>       <fplt>rapeseed_plt.xml</fplt>
#>       <ftec>USM_2017_T1_CI_tec.xml</ftec>
#>       <flai>null</flai>
#>       <fobs>USM_2017_T1_CI.obs</fobs>
#>     </plante>
#>     <plante dominance="2">
#>       <fplt>null</fplt>
#>       <ftec>null</ftec>
#>       <flai>null</flai>
#>       <fobs></fobs>
#> ...
#> </usms>

Generating the sols XML file (sols.xml)

As for gen_usms_xml, gen_sols_xml function is working on a data.frame object loaded from an Excel file sheet. The following code only shows here after the sols.xml file generation section.

Getting data

# Reading the Excel file
soils_param <- read_params_table(usm_xl_file, sheet_name = "Soils")

Generating the file


# Output file path
out_file <- "/path/to/file/sols.xml"
# or something like C:/path/to/file/usms.xml" for Windows

# Generating a new sols.xml file, for all xl_param lines
gen_sols_xml(file = out_file, param_df = soils_param)

sols.xml file content

Example of the first sol parameters sub-list:

#> <?xml version="1.0" encoding="UTF-8"?>
#> <sols version="10.0">
#>   <sol nom="USM_T1">
#>     <param format="real" max="60.0" min="0.0" nom="argi">20.35</param>
#>     <param format="real" max="0.5" min="0.05" nom="norg">0.1</param>
#>     <param format="real" max="60.0" min="10.0" nom="profhum">40</param>
#>     <param format="real" max="100.0" min="0.0" nom="calc">0.52</param>
#>     <param format="real" max="9.0" min="4.0" nom="pH">8.23</param>
#>     <param format="real" max="0.5" min="0.0" nom="concseuil">0.2</param>
#>     <param format="real" max="0.6" min="0.05" nom="albedo">0.22</param>
#>     <param format="real" max="50.0" min="0.0" nom="q0">9.63</param>
#>     <param format="real" max="1.0" min="0.0" nom="ruisolnu">0</param>
#>     <param format="real" max="1000.0" min="10.0" nom="obstarac">200</param>
#>     <param format="real" max="100.0" min="5.0" nom="pluiebat">50</param>
#>     <param format="real" max="2.0" min="0.0" nom="mulchbat">0.5</param>
#>     <param format="real" max="150.0" min="10.0" nom="zesx">60</param>
#>     <param format="real" max="10.0" min="-10.0" nom="cfes">5</param>
#>     <param format="real" max="0.2" min="0.01" nom="z0solnu">0.01</param>
#>     <param format="real" max="20.0" min="8.0" nom="CsurNsol">0</param>
#>     <param nom="finert">0.65</param>
#>     <param format="real" max="5.0" min="0.0" nom="penterui">0.33</param>
#>     <option choix="2" nom="pebbles" nomParam="codecailloux">
#>       <choix code="1" nom="yes"/>
#>       <choix code="2" nom="no"/>
#>     </option>
#>     <option choix="1" nom="macroporosity" nomParam="codemacropor">
#>       <choix code="1" nom="yes"/>
#>       <choix code="2" nom="no"/>
#>     </option>
#>     <option choix="2" nom="cracks (case of swelling clay soils)" nomParam="codefente">
#> ...
#>     </tableau>
#> ...
#> </sols>

Generating multiple crop management XML files (*_tec.xml)

Getting data

# Reading the Excel file
tec_param <- read_params_table(usm_xl_file, sheet_name = "Tec")

Generating files

# *_tec.xml files, one for each xl_param line
gen_tec_xml(param_df = tec_param, out_dir = workspace_path)

Example of a tec parameters file content sub-list:

#> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
#> <fichiertec version="10.0">
#>   <formalisme nom="supply of organic residus">
#>     <ta nb_interventions="1" nom="interventions">
#>       <ta_entete nb_colonnes="7">
#>         <colonne nom="julres"/>
#>         <colonne nom="coderes"/>
#>         <colonne nom="qres"/>
#>         <colonne nom="Crespc"/>
#>         <colonne nom="CsurNres"/>
#>         <colonne nom="Nminres"/>
#>         <colonne nom="eaures"/>
#>       </ta_entete>
#>       <intervention nb_colonnes="7">
#>         <colonne nom="julres">110</colonne>
#>         <colonne nom="coderes">1</colonne>
#>         <colonne nom="qres">9</colonne>
#>         <colonne nom="Crespc">42</colonne>
#>         <colonne nom="CsurNres">90</colonne>
#>         <colonne nom="Nminres">0</colonne>
#>         <colonne nom="eaures">7</colonne>
#>       </intervention>
#>     </ta>
#>   </formalisme>
#>   <formalisme nom="soil tillage">
#>     <option choix="2" nom="Automatic calculation of the depth of residues incorporation in function of proftrav" nomParam="code_auto_profres">
#>       <choix code="1" nom="yes">
#>         <param format="real" max="0.0" min="1.0" nom="resk">-999</param>
#>         <param format="real" max="0.0" min="10.0" nom="resz">-999</param>
#>       </choix>
#> ...
#> </fichiertec>

Generating multiple initialization XML files (*_ini.xml)

Getting data

# Reading the Excel file
ini_param <- read_params_table(usm_xl_file, sheet_name = "Ini")

Generating files

# *_ini.xml files, one for each xl_param line
gen_ini_xml(param_df = ini_param, out_dir = workspace_path)

XML file content

Example of a ini parameters file content sub-list:

#> <?xml version="1.0" encoding="UTF-8"?>
#> <initialisations version="10.0">
#>   <nbplantes>1</nbplantes>
#>   <plante dominance="1">
#>     <stade0>snu</stade0>
#>     <lai0>0</lai0>
#>     <magrain0>0</magrain0>
#>     <zrac0>0</zrac0>
#>     <option choix="2" nom="Simulation of Nitrogen and Carbon reserves" nomParam="code_acti_reserve">
#>       <choix code="1" nom="yes">
#>         <maperenne0>0</maperenne0>
#>         <QNperenne0>0</QNperenne0>
#>         <masecnp0>0</masecnp0>
#>         <QNplantenp0>0</QNplantenp0>
#>       </choix>
#>       <choix code="2" nom="no">
#>         <masec0>0</masec0>
#>         <QNplante0>0</QNplante0>
#>         <restemp0>0</restemp0>
#> ...
#> </initialisations>

Generating multiple meteorological station XML files (*_sta.xml)

Getting data

# Reading the Excel file
sta_param <- read_params_table(usm_xl_file, sheet_name = "Station")

Generating files

# *_sta.xml files, one for each xl_param line
gen_sta_xml(param_df = sta_param, out_dir = workspace_path)

XML file content

Example of a sta parameters file content sub-list:

#> <?xml version="1.0" encoding="UTF-8" standalone="no"?>
#> <fichiersta version="10.0">
#>   <formalisme nom="Weather station">
#>     <param format="real" max="10.0" min="2.0" nom="zr">2.5</param>
#>     <param format="real" max="10.0" min="0.0" nom="NH3ref">0</param>
#>     <param format="real" max="3.0" min="0.0" nom="concrr"/>
#>     <param format="real" max="90.0" min="-90.0" nom="latitude">49</param>
#>     <param format="real" max="1200.0" min="800.0" nom="patm">1000</param>
#>     <param format="real" max="25.0" min="4.0" nom="aclim">20</param>
#>   </formalisme>
#>   <formalisme nom="climate">
#>     <option choix="1" nom="reading OR calculation of PET" nomParam="codeetp">
#>       <choix code="1" nom="PET-Penman_reading"/>
#>       <choix code="2" nom="PET-Penman_calculation"/>
#>       <choix code="3" nom="PET-Shuttleworth-Wallace_calculation"/>
#>       <choix code="4" nom="PET-Priestley-Taylor_calculation">
#>         <param format="real" max="2.0" min="1.0" nom="alphapt">1.25</param>
#>       </choix>
#>     </option>
#>     <option choix="1" nom="climate change" nomParam="codeclichange">
#>       <choix code="1" nom="no"/>
#>       <choix code="2" nom="yes"/>
#>     </option>
#>     <option choix="1" nom="climate in altitude" nomParam="codaltitude">
#>       <choix code="1" nom="no"/>
#>       <choix code="2" nom="yes">
#>         <param format="real" max="2000.0" min="0.0" nom="altistation">440</param>
#>         <param format="real" max="2000.0" min="0.0" nom="altisimul">800</param>
#>         <param format="real" max="3.0" min="0.1" nom="gradtn">-0.5</param>
#>         <param format="real" max="3.0" min="0.1" nom="gradtx">-0.55</param>
#>         <param format="real" max="2000.0" min="0.0" nom="altinversion">500</param>
#>         <param format="real" max="3.0" min="0.1" nom="gradtninv">1.3</param>
#>         <param format="real" max="1.0" min="0.0" nom="cielclair">0.8</param>
#>         <option choix="1" nom="option.adret.or.ubac" nomParam="codadret">
#>           <choix code="1" nom="adret(south)"/>
#>           <choix code="2" nom="ubac(north)">
#>             <param format="real" max="5.0" min="-5.0" nom="ombragetx">-1.4</param>
#>           </choix>
#>         </option>
#>       </choix>
#>     </option>
#> ...
#> </fichiersta>