Running the STEMMUS_SCOPE model with different dataset¶
This notebook shows how to run the STEMMUS_SCOPE model uisng the Executable
file, including preprocessing and postprocessing. For postprocesisng, the file required_netcdf_variables.csv
can be found at https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/utils/csv_to_nc/required_netcdf_variables.csv.
See the Running the STEMMUS_SCOPE model on a system for different ways e.g. Executable file, Matlab, Octave.
The STEMMUS_SCOPE
executable file is located in a repository https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/exe/STEMMUS_SCOPE.
Which datasets are available?¶
The config file shows how to set the paths to different data needed by the model. Check the data source on Snelius and CRIB and modify the config file accordingly:
On CRIB¶
- Site data:
- path to data:
/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/input
, - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib.txt
- path to data:
- Global data:
- path to data
/data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/global_data
, - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_crib_global.txt
- path to data
On Snellius¶
- Site data:
- path to data:
/projects/0/ttse0619/
, - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius.txt
- path to data:
- Global data:
- path to data:
/projects/0/ttse0619/global_data
, - config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius_global.txt
- path to data:
Choose how do you want to run the model:
Using site data¶
import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save
# user must provide the correct path
path_to_config_file = "./my_config_template.txt"
path_to_exe_file = "./STEMMUS_SCOPE"
If you run the model on your own machine, make sure LD_LIBRARY_PATH
is set correctly. To do this, uncomment the cell below and run it:
# # Set LD_LIBRARY_PATH
# matlab_path = !whereis MATLAB
# matlab_path = matlab_path.s.split(": ")[1]
# os.environ['LD_LIBRARY_PATH'] = (
# f"{matlab_path}/MATLAB_Runtime/R2023a/runtime/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/sys/os/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/extern/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64")
# print(os.environ['LD_LIBRARY_PATH'])
/usr/local/MATLAB/MATLAB_Runtime/R2023a/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64
# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)
# setup the model
# here you can change the location and start & end time
config_path = model.setup(
Location="ZA-Kru",
StartTime="2001-01-01T00:00",
EndTime="2001-01-02T01:30",
)
# new config file genertaed to run the model
print(f"New config file {config_path}")
# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')
New config file /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_config.txt Model input dir /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/ Model output dir /home/sarah/temp/ecoextreml/test/output/ZA-Kru_2024-01-24-1125/
# run the model
result = model.run()
result
'Reading config from /home/sarah/temp/ecoextreml/test/input/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_config.txt\nFinished model initialization\nThe calculations start now\nFinalizing STEMMUS_SCOPE\n'
# save output in netcdf format
required_netcdf_variables = "./required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)
/home/sarah/temp/ecoextreml/test/output/ZA-Kru_2024-01-24-1125/ZA-Kru_2024-01-24-1125_STEMMUS_SCOPE.nc
Using global data¶
import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save
# user must provide the correct path
path_to_config_file = "./my_config_template_global.txt"
path_to_exe_file = "./STEMMUS_SCOPE"
If you run the model on your own machine, make sure LD_LIBRARY_PATH
is set correctly. To do this, uncomment the cell below and run it:
# # Set LD_LIBRARY_PATH
# matlab_path = !whereis MATLAB
# matlab_path = matlab_path.s.split(": ")[1]
# os.environ['LD_LIBRARY_PATH'] = (
# f"{matlab_path}/MATLAB_Runtime/R2023a/runtime/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/sys/os/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/extern/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64")
# print(os.environ['LD_LIBRARY_PATH'])
/usr/local/MATLAB/MATLAB_Runtime/R2023a/runtime/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/os/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/extern/bin/glnxa64:/usr/local/MATLAB/MATLAB_Runtime/R2023a/sys/opengl/lib/glnxa64
# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)
# setup the model
config_path = model.setup(
Location="(61.8474, 24.2948)",
StartTime="2014-01-10T00:00",
EndTime="2014-01-15T00:00",
)
# new config file genertaed to run the model
print(f"New config file {config_path}")
# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')
/home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16 result = blockwise( /home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16 result = blockwise( /home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16 result = blockwise( /home/sarah/miniconda3/envs/pystemmusscope/lib/python3.10/site-packages/dask/array/core.py:4836: PerformanceWarning: Increasing number of chunks by factor of 16 result = blockwise(
New config file /home/sarah/temp/ecoextreml/test/input/global_N61-847_E24-295_2024-02-02-1620/global_2024-02-02-1620_config.txt Model input dir /home/sarah/temp/ecoextreml/test/input/global_N61-847_E24-295_2024-02-02-1620/ Model output dir /home/sarah/temp/ecoextreml/test/output/global_N61-847_E24-295_2024-02-02-1620/
# run the model
result = model.run()
result
# save output in netcdf format
required_netcdf_variables = "./required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)