Different ways to run the model

This notebook shows steps to run the STEMMUS_SCOPE model, including preprocessing and postprocessing. For postprocesisng, the file required_netcdf_variables.csv can be found at https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/utils/csv_to_nc/required_netcdf_variables.csv.

First, check the Getting started page on documentation of STEMMUS_SCOPE and pystemmusscope.

Depending on your system (OS, Matlab availability, etc.), there are a few ways you can run the model. Choose how do you want to run the model:

Executable file

  If MATLAB Runtime is installed on a Unix-like system, you can run STEMMUS_SCOPE using the executable file, for example on Snellius or WSL.

Matlab

  If Matlab is installed, you can run STEMMUS_SCOPE from the source code, for example on Snellius or CRIB.

Octave

  If latest Octave including required packages is installed, you can run STEMMUS_SCOPE from the source code, for example on CRIB or your own machine.

1. Using executable file¶

In [16]:

import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save

import os from PyStemmusScope import StemmusScope from PyStemmusScope import save

In [17]:

# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_exe_file = "~/STEMMUS_SCOPE/exe/STEMMUS_SCOPE"

# user must provide the correct path path_to_config_file = "~/my_config_template.txt" path_to_exe_file = "~/STEMMUS_SCOPE/exe/STEMMUS_SCOPE"

If you run the model on your own machine, make sure LD_LIBRARY_PATH is set correctly. To do this, uncomment the cell below and run it:

In [18]:

# # Set LD_LIBRARY_PATH
# matlab_path = !whereis MATLAB
# matlab_path = matlab_path.s.split(": ")[1]
# os.environ['LD_LIBRARY_PATH'] = (
#     f"{matlab_path}/MATLAB_Runtime/v910/runtime/glnxa64:"
#     f"{matlab_path}/MATLAB_Runtime/v910/bin/glnxa64:"
#     f"{matlab_path}/MATLAB_Runtime/v910/sys/os/glnxa64:"
#     f"{matlab_path}/MATLAB_Runtime/v910/extern/bin/glnxa64:"
#     f"{matlab_path}/MATLAB_Runtime/v910/sys/opengl/lib/glnxa64")
# print(os.environ['LD_LIBRARY_PATH'])

# # Set LD_LIBRARY_PATH # matlab_path = !whereis MATLAB # matlab_path = matlab_path.s.split(": ")[1] # os.environ['LD_LIBRARY_PATH'] = ( # f"{matlab_path}/MATLAB_Runtime/v910/runtime/glnxa64:" # f"{matlab_path}/MATLAB_Runtime/v910/bin/glnxa64:" # f"{matlab_path}/MATLAB_Runtime/v910/sys/os/glnxa64:" # f"{matlab_path}/MATLAB_Runtime/v910/extern/bin/glnxa64:" # f"{matlab_path}/MATLAB_Runtime/v910/sys/opengl/lib/glnxa64") # print(os.environ['LD_LIBRARY_PATH'])

In [19]:

# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)

# create an an instance of the model model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_exe_file)

In [20]:

# setup the model
# here you can change the location and start & end time
config_path = model.setup(
    Location="ZA-Kru",
    StartTime="2001-01-01T00:00",
    EndTime="2001-01-02T01:30",
)

# new config file genertaed to run the model
print(f"New config file {config_path}")

# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')

# setup the model # here you can change the location and start & end time config_path = model.setup( Location="ZA-Kru", StartTime="2001-01-01T00:00", EndTime="2001-01-02T01:30", ) # new config file genertaed to run the model print(f"New config file {config_path}") # see input and output paths generated by the model print(f'Model input dir {model.config["InputPath"]}') print(f'Model output dir {model.config["OutputPath"]}')

New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt
Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/
Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1642/

In [21]:

# run the model
result = model.run()
result

# run the model result = model.run() result

Out[21]:

b'Opening log file:  /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642//java.log.5036\nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1642/ZA-Kru_2022-10-27-1642_config.txt\nThe calculations start now\nThe calculations end now\n'

In [16]:

# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)

# save output in netcdf format required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv" nc_file_name = save.to_netcdf(config_path, required_netcdf_variables) print(nc_file_name)

2. Using Matlab¶

In [1]:

import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save

import os from PyStemmusScope import StemmusScope from PyStemmusScope import save

In [4]:

# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_model_src = "~/STEMMUS_SCOPE/src"

# user must provide the correct path path_to_config_file = "~/my_config_template.txt" path_to_model_src = "~/STEMMUS_SCOPE/src"

In [5]:

# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

# create an an instance of the model model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

In [6]:

# setup the model
config_path = model.setup(
    Location="ZA-Kru",
    StartTime="2001-01-01T00:00",
    EndTime="2001-01-02T01:30",
)

# new config file genertaed to run the model
print(f"New config file {config_path}")

# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')

# setup the model config_path = model.setup( Location="ZA-Kru", StartTime="2001-01-01T00:00", EndTime="2001-01-02T01:30", ) # new config file genertaed to run the model print(f"New config file {config_path}") # see input and output paths generated by the model print(f'Model input dir {model.config["InputPath"]}') print(f'Model output dir {model.config["OutputPath"]}')

New config file /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt
Model input dir /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/
Model output dir /gpfs/home2/alidoost/test_matlab/output/ZA-Kru_2022-10-27-1651/

  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform <echo $DISPLAY> in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly.

In [ ]:

# For CRIB users
%env DISPLAY=:1.0

# For CRIB users %env DISPLAY=:1.0

In [7]:

# run the model
result = model.run()
result

# run the model result = model.run() result

Out[7]:

b'MATLAB is selecting SOFTWARE OPENGL rendering.\nOpening log file:  /home/alidoost/java.log.37351\n\n                            < M A T L A B (R) >\n                  Copyright 1984-2021 The MathWorks, Inc.\n             R2021a Update 3 (9.10.0.1684407) 64-bit (glnxa64)\n                                May 27, 2021\n\n \nTo get started, type doc.\nFor product information, visit www.mathworks.com.\n \nReading config from /gpfs/home2/alidoost/test_matlab/input/ZA-Kru_2022-10-27-1651/ZA-Kru_2022-10-27-1651_config.txt\nThe calculations start now\nThe calculations end now\n'

In [ ]:

# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)

# save output in netcdf format required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv" nc_file_name = save.to_netcdf(config_path, required_netcdf_variables) print(nc_file_name)

3. Using Octave¶

In [1]:

import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save

import os from PyStemmusScope import StemmusScope from PyStemmusScope import save

In [2]:

# user must provide the correct path
path_to_config_file = "~/my_config_template.txt"
path_to_model_src = "~/STEMMUS_SCOPE/src"

# user must provide the correct path path_to_config_file = "~/my_config_template.txt" path_to_model_src = "~/STEMMUS_SCOPE/src"

In [3]:

# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Octave")

# create an an instance of the model model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Octave")

In [4]:

# setup the model
# here you can change the location and start & end time
config_path = model.setup(
    Location="ZA-Kru",
    StartTime="2001-01-01T00:00",
    EndTime="2001-01-02T01:30",
)

# new config file genertaed to run the model
print(f"New config file {config_path}")

# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')

# setup the model # here you can change the location and start & end time config_path = model.setup( Location="ZA-Kru", StartTime="2001-01-01T00:00", EndTime="2001-01-02T01:30", ) # new config file genertaed to run the model print(f"New config file {config_path}") # see input and output paths generated by the model print(f'Model input dir {model.config["InputPath"]}') print(f'Model output dir {model.config["OutputPath"]}')

New config file /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt
Model input dir /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/
Model output dir /data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/

In [5]:

# run the model
result = model.run()
result

# run the model result = model.run() result

Reading config from /data/private/Sarah/test_run/input/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_config.txt
The calculations start now
The calculations end now

In [6]:

# save output in netcdf format
required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)

# save output in netcdf format required_netcdf_variables = "~/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv" nc_file_name = save.to_netcdf(config_path, required_netcdf_variables) print(nc_file_name)

/data/private/Sarah/test_run/output/ZA-Kru_2022-10-28-1159/ZA-Kru_2022-10-28-1159_STEMMUS_SCOPE.nc

In [ ]: