Site and global datasets on CRIB

This notebook shows how to run the STEMMUS_SCOPE model uisng source code, including preprocessing and postprocessing. See the Different ways to run the model for different ways e.g. Executable file, Matlab, Octave.

Also, check the Getting started page on documentation of STEMMUS_SCOPE and pystemmusscope.

For postprocesisng, the file required_netcdf_variables.csv can be found at https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/utils/csv_to_nc/required_netcdf_variables.csv.

Which datasets are available?¶

The config file shows how to set the paths to different data needed by the model. Check the data source on Snelius and CRIB and modify the config file accordingly:

On CRIB¶

• Site data:
  • path to data: /data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/input,
  • config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib.txt
• Global data:
  • path to data /data/shared/EcoExtreML/STEMMUS_SCOPEv1.0.0/global_data,
  • config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/config_file_crib_global.txt

On Snellius¶

• Site data:
  • path to data: /projects/0/ttse0619/,
  • config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius.txt
• Global data:
  • path to data: /projects/0/ttse0619/global_data,
  • config file template: https://github.com/EcoExtreML/STEMMUS_SCOPE/blob/main/run_model_on_snellius/config_file_snellius_global.txt

Choose how do you want to run the model with MATLAB on CRIB:

Using site data

Using global data

1. Using site data¶

In [3]:

import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save

import os from PyStemmusScope import StemmusScope from PyStemmusScope import save

In [4]:

# user must provide the correct path
path_to_config_file = "./config_file_crib.txt"
path_to_model_src = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/src"

# user must provide the correct path path_to_config_file = "./config_file_crib.txt" path_to_model_src = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/src"

In [5]:

# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

# create an an instance of the model model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

In [6]:

# setup the model
# here you can change the location and start & end time
config_path = model.setup(
    Location="ZA-Kru",
    StartTime="2001-01-01T00:00",
    EndTime="2001-01-02T01:30",
)

# new config file genertaed to run the model
print(f"New config file {config_path}")

# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')

# setup the model # here you can change the location and start & end time config_path = model.setup( Location="ZA-Kru", StartTime="2001-01-01T00:00", EndTime="2001-01-02T01:30", ) # new config file genertaed to run the model print(f"New config file {config_path}") # see input and output paths generated by the model print(f'Model input dir {model.config["InputPath"]}') print(f'Model output dir {model.config["OutputPath"]}')

New config file /data/private/test_run/input/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_config.txt
Model input dir /data/private/test_run/input/ZA-Kru_2024-01-29-1142/
Model output dir /data/private/test_run/output/ZA-Kru_2024-01-29-1142/

  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform <echo $DISPLAY> in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly.

In [7]:

# For CRIB users
%env DISPLAY=:1.0

# For CRIB users %env DISPLAY=:1.0

env: DISPLAY=:1.0

In [8]:

# run the model
result = model.run()
result

# run the model result = model.run() result

Out[8]:

'\n                            < M A T L A B (R) >\n                  Copyright 1984-2021 The MathWorks, Inc.\n                  R2021a (9.10.0.1602886) 64-bit (glnxa64)\n                             February 17, 2021\n\n \nTo get started, type doc.\nFor product information, visit www.mathworks.com.\n \nReading config from /data/private/test_run/input/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_config.txt\nFinished model initialization\nThe calculations start now\nFinalizing STEMMUS_SCOPE\n'

In [9]:

# save output in netcdf format
required_netcdf_variables = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)

# save output in netcdf format required_netcdf_variables = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv" nc_file_name = save.to_netcdf(config_path, required_netcdf_variables) print(nc_file_name)

/data/private/test_run/output/ZA-Kru_2024-01-29-1142/ZA-Kru_2024-01-29-1142_STEMMUS_SCOPE.nc

2. Using global data¶

In [1]:

import os
from PyStemmusScope import StemmusScope
from PyStemmusScope import save

import os from PyStemmusScope import StemmusScope from PyStemmusScope import save

In [2]:

# user must provide the correct path
path_to_config_file = "./config_file_crib_global.txt"
path_to_model_src = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/src"

# user must provide the correct path path_to_config_file = "./config_file_crib_global.txt" path_to_model_src = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/src"

In [3]:

# create an an instance of the model
model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

# create an an instance of the model model = StemmusScope(config_file=path_to_config_file, model_src_path=path_to_model_src, interpreter="Matlab")

In [ ]:

# setup the model
config_path = model.setup(
    Location="(61.8474, 24.2948)",
    StartTime="2014-01-01T00:00",
    EndTime="2014-01-15T00:00",
)

# new config file genertaed to run the model
print(f"New config file {config_path}")

# see input and output paths generated by the model
print(f'Model input dir {model.config["InputPath"]}')
print(f'Model output dir {model.config["OutputPath"]}')

# setup the model config_path = model.setup( Location="(61.8474, 24.2948)", StartTime="2014-01-01T00:00", EndTime="2014-01-15T00:00", ) # new config file genertaed to run the model print(f"New config file {config_path}") # see input and output paths generated by the model print(f'Model input dir {model.config["InputPath"]}') print(f'Model output dir {model.config["OutputPath"]}')

  If you are on CRIB, to use Jupyter interface and MATLAB license manager at the same time, first perform <echo $DISPLAY> in a terminal in the Remote Desktop of Crib Platform, and check the output which should be similar to :1.0 . Then execute the following line accordingly.

In [1]:

# for CRIB users
%env DISPLAY=:1.0

# for CRIB users %env DISPLAY=:1.0

env: DISPLAY=:1.0

In [ ]:

# run the model
result = model.run()
result

# run the model result = model.run() result

In [ ]:

# save output in netcdf format
required_netcdf_variables = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv"
nc_file_name = save.to_netcdf(config_path, required_netcdf_variables)
print(nc_file_name)

# save output in netcdf format required_netcdf_variables = "/home/jovyan/private/Sarah/STEMMUS_SCOPE/utils/csv_to_nc/required_netcdf_variables.csv" nc_file_name = save.to_netcdf(config_path, required_netcdf_variables) print(nc_file_name)