Running the STEMMUS_SCOPE model on a system¶
This notebook shows steps to run the STEMMUS_SCOPE model, including preprocessing and postprocessing. STEMMUS_SCOPE files are located in a repository https://github.com/EcoExtreML/STEMMUS_SCOPE. Then specify the path to a config file e.g.config_file_template.txt
, executable file STEMMUS_SCOPE
or source code STEMMUS_SCOPE/src
in the cells below. There are already config files for users on Snellius and CRIB, see config_file_snellius.txt
, config_file_crib.txt
. Depending on your system (OS, Matlab availability, etc.), there are a few ways you can run the model. Choose how do you want to run the model:
If MATLAB Runtime is installed on a Unix-like system, you can run STEMMUS_SCOPE using the executable file, for example on Snellius or WSL.
If Matlab is installed, you can run STEMMUS_SCOPE from the source code, for example on Snellius or CRIB.
If latest Octave including required packages is installed, you can run STEMMUS_SCOPE from the source code, for example on CRIB or your own machine.
1. Using executable file¶
import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save
# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_exe_file = "~/STEMMUS_SCOPE/exe/STEMMUS_SCOPE"
If you run the model on your own machine, make sure LD_LIBRARY_PATH
is set correctly. To do this, uncomment the cell below and run it:
# # Set LD_LIBRARY_PATH
# matlab_path = !whereis MATLAB
# matlab_path = matlab_path.s.split(": ")[1]
# os.environ['LD_LIBRARY_PATH'] = (
# f"{matlab_path}/MATLAB_Runtime/v910/runtime/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/v910/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/v910/sys/os/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/v910/extern/bin/glnxa64:"
# f"{matlab_path}/MATLAB_Runtime/v910/sys/opengl/lib/glnxa64")
# print(os.environ['LD_LIBRARY_PATH'])
# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)
# setup the model
# here you can change the location and start & end time
config_path = model.setup(
Location="ZA-Kru",
StartTime="2001-01-01T00:00",
EndTime="2001-01-02T01:30",
)
# new config file genertaed to run the model
print(f"New config file {config_path}")
# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')
New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1642/
# run the model
result = model.run()
result
b'Opening log file: /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642//java.log.5036\nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt\nThe calculations start now\nThe calculations end now\n'
# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)
2. Using Matlab¶
import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save
# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_model_src = "~/STEMMUS_SCOPE/src"
# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")
# setup the model
config_path = model.setup(
Location="ZA-Kru",
StartTime="2001-01-01T00:00",
EndTime="2001-01-02T01:30",
)
# new config file genertaed to run the model
print(f"New config file {config_path}")
# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')
New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1651/
If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform <echo $DISPLAY> in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly.
# For CRIB users
%env DISPLAY=:1.0
# run the model
result = model.run()
result
b'MATLAB is selecting SOFTWARE OPENGL rendering.\nOpening log file: /home/alidoost/java.log.37351\n\n < M A T L A B (R) >\n Copyright 1984-2021 The MathWorks, Inc.\n R2021a Update 3 (9.10.0.1684407) 64-bit (glnxa64)\n May 27, 2021\n\n \nTo get started, type doc.\nFor product information, visit www.mathworks.com.\n \nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt\nThe calculations start now\nThe calculations end now\n'
# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)
3. Using Octave¶
import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save
# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_model_src = "~/STEMMUS_SCOPE/src"
# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Octave")
# setup the model
# here you can change the location and start & end time
config_path = model.setup(
Location="ZA-Kru",
StartTime="2001-01-01T00:00",
EndTime="2001-01-02T01:30",
)
# new config file genertaed to run the model
print(f"New config file {config_path}")
# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')
New config file /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt Model input dir /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ Model output dir /data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/
# run the model
result = model.run()
result
Reading config from /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt The calculations start now The calculations end now
# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)
/data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_STEMMUS_SCOPE.nc