Getting Started¶

This chapter provides a step-by-step guide to quickly get started with the SURF-NEMO platform. It includes instructions for downloading and installing the SURF Virtual Machine (VM) and all required SURF packages. Additionally, this guide will show you how to compile the source codes, run a case study experiment in the Gulf of Taranto, and analyze the results. Finally, you’ll learn how to modify the user-configuration file for executing and analyzing new experiments.

The template experiment makes it easier to run simulations without requiring in-depth knowledge of the underlying scientific details. For most applications, only a limited number of default values need to be adjusted. However, for more advanced use cases, such as changing turbulence models or adjusting numerical schemes, users should consult the NEMO User Manual for a deeper understanding. There are also video tutorials available online here, designed for beginners to walk through the basic features of the SURF platform step by step.

Download and Install SURF Virtual Machine (VM)¶

The SURF platform is distributed as a Virtual Machine (VM) image, packaged within a ZIP archive. The naming scheme for the releases follows the format surf_vm_VERSION.zip, where VERSION represents the version number (e.g., surf_vm_1.01.zip for the current version). The instructions below explain how to download, install and configure the SURF VM in Oracle VirtualBox.

Installing Oracle VirtualBox¶

Go to the VirtualBox download page and download the VirtualBox base package (version >=6) corresponding to your operating system (Windows, Mac, or Linux).

Figure 3.1. Downloads VirtualBox base package.
Follow the installation instructions to install VirtualBox on your system.
In addition to the base package, download the VirtualBox Extension Pack, which adds functionality like USB device support and remote desktop support. Double-click the downloaded extension pack and follow the installation steps. Please install the same version extension pack as your installed version of the VirtualBox base package.

Figure 3.2. Downloads VirtualBox Extension Pack.

Downloading and Installing SURF VM¶

Visit the SURF platform website and download the current version of the SURF Virtual Machine (e.g., surf_vm_1.01.zip). The default installation path for VirtualBox VMs is typically /Users/USERNAME/VirtualBox VMs/ on macOS, with similar paths for other operating systems. To download the SURF-VM directly from the terminal, use the following commands:
```
cd /Users/USERNAME/VirtualBox VMs;
wget https://www.surf-platform.org/repository/surf_vm/surf_vm_1.01/surf_vm_1.01.zip
```
In the virtual machines is installed the Debian GNU/Linux 8.11 (jessie) operating system and all the necessary libraries required. The Guest Additions have been also installed to optimize the guest operating system for better performance and usability.

Figure 3.3. Downloads SURF Virtual Machine.
Unzip the archive file into the VirtualBox directory:
```
unzip surf_vm_1.01.zip
```
Open VirtualBox, navigate to Machine > Add, and select the surf.vbox file to add the SURF VM to the list of Virtual Machine. This file is an XML file that contains settings of the Machine.

Figure 3.4. Add SURF-VM in VirtualBox.
Start the VM from the VirtualBox Manager by selecting it and pressing Start button at the top of the window. The VM Login should look like the figure 3.5 .

Figure 3.5. Start SURF-VM.
Log in using the following credentials:
- Username: surf
- Password: surf2019

Disk Partitions mounted on the SURF Virtual Machine¶

The SURF VM package includes two VirtualBox Disk Image (VDI) files:

surf.vdi: Contains the Debian GNU/Linux 8.11 (jessie) operating system.
surf_scratch.vdi: Contains source code files, sample datasets, and experiments.

It is divided into two main partitions:

the disk /dev/sda "mounted" as filesystems to the root directory /
the disk /dev/sdb "mounted" in the directory /scratch.

From the guest operating system you can see the list of partitions by typing the following command:

sudo fdisk -l

Shared Folders Between Host and Guest Systems¶

You can mount other physical hard disks with VirtualBox (see the VirtualBox Manual for details). VirtualBox has the ability to mount a shared folder between host and guest in order to access files of your host system from within the guest system. There are a few steps involved:

Shut down the VM, go to Settings > Shared Folders in VirtualBox.
Click the Plus button and select the folder you want to share. Check the Auto-mount option and click OK.
Restart the VM, and the shared folder will appear in /media with the prefix "sf_".

Configuring the SURF Virtual Machine¶

By default, the VM surf is configurated as in table Table 3.1 . You can keep all defaults parameters or if it is not adequate for your application you can change settings. To change the configuration you need to shut down the virtual OS before you can edit settings.

Select the surf VM in the VirtualBox Manager, right-click it and choose Setting.
increase/decrease the number of cores based on your performance desires.
increase/decrease the number of GB of RAM allocated to your VM according to the size of your computational domain.
increase/decrease the video memory and scale factor of your screen

VM_install_setting.png.png — Figure 3.7. Change VM configurations.

If you want to add more storage space to a VM you can also expande the virtual hard disk. There are a few steps involved:

With the VM Power off, open a terminal and move to the location of the surf_scratch.vdi file that you want to resize,

At the terminal prompt, type the command:

VBoxManage modifyhd surf_scratch.vdi --resize SIZE_MB

Restart the SURF VM and open the GParted application from the Application Menu
Select the /dev/sdb partition (an unlocated drive space is now available). Resize to the unalocated area

Table 3.1: Summary of Virtual Machine fields and configurations.
Parameter	Description	Values
Name	Name given to the VM	surf
Guest OS	Operating system running on this VM	Debian Linux
Memory	Amount of memory available to this VM	2 GB
Cores	Number of CPU cores being used by this VM	2
Disk Capacity	Total disk capacity available to this VM	40 GB
Network Adapters	Number of network adapters available to this VM	1
IP Address	IP address assigned to the VM	x

Download and Install SURF packages¶

The Virtual Machine you have installed does not contain the SURF packages (source codes and static datasets) and you need to download and install them. The scratch directory /scratch follows the directory structure as shown in Figure B.1. The SURF packages are packaged and distributed as a GZIP Compressed Tar Archive file (with a .tar.gz extension). The naming scheme for the releases follows the format packageName_VERSION.tar.gz, where VERSION is a number (e.g. surf_nemo_1.01.tar.gz for the current version of the surf_nemo package).

The instructions below explain how to install the packages in the SURF VM.

Once logged in the VM surf, download the current version of the SURF-NEMO (surf_nemo_1.01.tar.gz) and SURF-DATASETS (surf_datasets_1.01.tar.gz) packages directly from the SURF platform website and save it in the directory /scratch/surf/surf_install/releases/ (for simplicity, we abbreviate this location as $SURF_RELEASES). To download the SURF-NEMO and SURF-DATASETS packages, execute the following commands::
```
cd /scratch/surf/surf_install/releases
wget https://www.surf-platform.org/repository/surf_nemo/surf_nemo_1.01/surf_nemo_1.01.tar.gz
wget https://www.surf-platform.org/repository/surf_datasets/surf_datasets_1.01/surf_datasets_1.01.tar.gz
```
Go to the directory $SURF_RELEASES and run the installation bash script install.sh followed by the package name. Use the following commands to install both the SURF-NEMO and SURF-DATASETS packages:
```
cd /scratch/surf/surf_install/releases
install.sh surf_nemo_1.01.tar.gz
install.sh surf_datasets_1.01.tar.gz
```
The installation process will extract the archive in the directory /scratch/surf/surf_nemo/ and /scratch/surf/surf_datasets/, respectively, and will create a symbolic link current in this directory that points to the extracted folder (for simplicity, we abbreviate this location as $SURF_NEMO, $SURF_DATASETS, respectively).

For a detailed description of the directory structure and contents of each package refer to the Appendix B.

Compiling the source code¶

Once the installation of the SURF-NEMO package is complete, you need to compile the source codes in order to create the executable files needed to perform specific tasks. The executable files should not be recreated unless you need to modify the source code.

The compilation process is handled using the Unix/Linux make utility and the following tools:

Fortran 90 Compiler – used for compiling the core numerical models.
C Preprocessor (cpp) – processes C-style macros.
MPI Library – required for running simulations in parallel mode.
netCDF Library – allows reading and writing of data in portable netCDF format.

All of these tools are already installed and configured on the SURF platform, so no additional setup is required.

To compile the source codes go to the directory /scratch/surf/surf_nemo/current/scripts/ and run the compilation bash script compile.sh followed by the package name (or by the word 'all' to compile all the packages):

cd /scratch/surf/surf_nemo/current/scripts;
./compile_codes.sh all

Compilation could take a few minutes and it will create the executable files for each program present in the SURF-NEMO package.

Running the Gulf of Taranto Case Study¶

In this case study, the SURF platform is used to implemente an high-resolution model in the Gulf of Taranto, located in the northern Ionian Sea. The simulation starts on 5 October 2014 at 00:00 and run until 7 October 2014 at 24:00. This section provides step-by-step instructions on how to set up, run, and analyze the downscaling experiment.

Setting Up the Experiment¶

Download the case study datasets (gulfTaranto_20141005.tar.gz) from the (SURF website). To download the SURF-VM, you can use the following commands:

cd /scratch/surf/indata_offline;
wget https://www.surf-platform.org/repository/surf_nemo/surf_nemo_1.01/case_studies/gulfTaranto_20141005/gulfTaranto_20141005_indata.tar.gz

Extract them to the directory /scratch/surf/indata_offline/ using the following command:
```
tar -zxvf gulfTaranto_20141005_indata.tar.gz
```
Note If you want to change the local repository path to some other location of your choice make sure to change the path in the user configuration file.
Create a new folder named gulfTaranto_20141005 in the /scratch/from_GUI/ directory. This folder will serve as the Experiment ID, uniquely identifying the experiment:
```
cd /scratch/from_GUI;
mkdir gulfTaranto_20141005
```
Copy the configuration file setParFree.json from /scratch/surf/surf_nemo/current/ into the newly created directory /scratch/from_GUI/gulfTaranto_20141005/. This file contains the configuration settings specific to this case study:
```
cp /scratch/surf/surf_nemo/current/setParFree.json /scratch/from_GUI/gulfTaranto_20141005/
```

Running the Experiment¶

To execute the experiment, navigate to the scripts directory and run the experiment using the run_exp.jl Julia script. Be sure to pass the experiment ID (in this case, gulfTaranto_20141005) as an argument:

cd /scratch/surf/surf_nemo/current/scripts;
julia run_exp.jl gulfTaranto_20141005

This command will generate a new folder named gulfTaranto_20141005 in the /scratch/surf/experiments/ directory. The newly created directory will follow a structure similar to the one shown in Figure B.1 (refer to the Appendix B for more details)

User can activate/deactivate specific tasks by setting logical parameters in the set_lrun section of the setParFree.json configuration file. Set each task to True or False depending on whether you want it enabled or disabled:

lrun_childMeshmask: Enable the CHILD-MESHMASK GENERATION task.
lrun_regridPreAtm: Enable the ATMOSPHERIC-DATA-REGRIDDING task.
lrun_regridPreOceIC: Enable the OCEAN-IC-DATA-REGRIDDING task.
lrun_regridPreOceBC: Enable the OCEAN-BC-DATA-REGRIDDING task.
lrun_regridPreTideBC: Enable the TIDAL-DATA-REGRIDDING task.
lrun_regridPreWeights: Set to True to compute the WEIGHT-FILEs for remapping in the regridding phase, or False to copy existing ones.
lrun_ocean: Enable the execution of the NEMO model.

Here's an example of how the set_lrun section might look in setParFree.json:

{
  "id": "A001",
  "title": "set_lrun",
  "items": [
    { "name": "lrun_childMeshmask", "value": "True" },
    { "name": "lrun_regridPreAtm", "value": "True" },
    { "name": "lrun_regridPreOceIC", "value": "True" },
    { "name": "lrun_regridPreOceBC", "value": "True" },
    { "name": "lrun_regridPreTideBC", "value": "True" },
    { "name": "lrun_regridPreWeights", "value": "True" },
    { "name": "lrun_ocean", "value": "True" }
  ]
}

Post-processing the results¶

The SURF platform includes a comprehensive suite of open-source tools designed for data analysis and visualization. Users can explore and visualize their data using the pre-installed NcView software, which provides interactive graphical capabilities. software, which provides interactive graphical capabilities. For more advanced data processing and visualization, the platform also provides a suite of NCL (NCAR Command Language) and Python scripts, giving users the flexibility to perform complex analyses and create customized visual outputs.

Visualizing the results with Ncview¶

NcView is a user-friendly tool designed for the quick visualization of netCDF data files. While its functionalities are limited, it is perfect for an initial exploration of your data. Typically you would use ncview to get a quick and easy, push-button look at your netCDF files. Users can create simple data movies, view data along different dimensions, inspect actual values, modify color maps, invert data, and more.

To launch NcView, use the following command, specifying the dataset you want to visualize:

ncview SURF_1h_20141006_20141006_grid_T.nc

Below is an example of the NcView user interface:

../assets/img/ncview.png — Figure 3.10. Screenshot of using NcView.

Analyzing and Visualizing results using NCAR graphic packages¶

The NCAR Graphics libraries offer advanced capabilities for the visualization of scientific data. NCL (NCAR Command Language) is an open-source, interpreted language designed to facilitate the analysis and visualization of geoscientific data. SURF-NEMO includes an extensive suite of NCL functions for post-processing tasks, such as:

Visualizing input and output datasets
Comparing child and parent model fields
Comparing simulation results with in-situ or satellite datasets
Converting datasets into various formats

Example visualizations generated using NCAR Graphics:

../assets/img/velxy_z000_t035.png — (A) Surface current.

../assets/img/tempxy_z000_t035.png — (A) Surface current.

To post-process the results of an existing experiment, you need to execute the Julia script run_postProc.jl followed by the experiment ID. Example for the case study experiment type the following command:

cd /scratch/surf/experiments/gulfTaranto_20141005/code/ocean/scripts/;
julia run_postproc.jl gulfTaranto_20141005

Post-processing tasks can be activated or deactivated by setting logical parameters in the set_lrun_post and set_visual_lplot sections of the configuration file setParFree.json. These parameters control various post-processing tasks, such as plotting specific domains, input fields, and comparison plots.

Set each task to True or False depending on whether you want it enabled or disabled:

lrun_visDom to enable the plotting of the user-defined domains.
lrun_visIndata to enable the plotting of the Indata Bat, Atm, OceIC, OceBC fields.
lrun_visExtrapdata to enable the plotting of the Extrapdata Atm, OceIC, OceBC fields.
lrun_visRegriddata to enable the execution of the OCEAN-IC-DATA-REGRIDDING task.
lrun_visOutdata to enable the execution of the OCEAN-BC-DATA-REGRIDDING task.
lrun_chlVSpar if you want to compute (=True) or just copy (=False) the WEIGHT-FILEs for REMAPPING in the Regridding phase.
lrun_surfVSctd enables the execution of the NEMO code.
lrun_surfVSsat enables the execution of the NEMO code.
lrun_surfVSmooring enables the execution of the NEMO code.
lrun_surfVSferrybox enables the execution of the NEMO code.

Here's an example of how the set_lrun_post section might look in setParFree.json:

{
  "id": "B000",
  "title": "set_lrun_post",
  "items": [
    { "name": "lrun_visDom", "value": "True" },
    { "name": "lrun_visIndata", "value": "True" },
    { "name": "lrun_visExtrapdata", "value": "True" },
    { "name": "lrun_visRegriddata", "value": "True" },
    { "name": "lrun_visOutdata", "value": "True" },
    { "name": "lrun_chlVSpar", "value": "True" },
    { "name": "lrun_surfVSctd", "value": "True" },
    { "name": "lrun_surfVSsat", "value": "True" },
    { "name": "lrun_surfVSmooring", "value": "True" },
    { "name": "lrun_surfVSferrybox", "value": "True" }
  ]
}

User can also configure which fields are plotted by modifying the set_visual_lplot section in the setParFree.json file:

lplotMesh: Enable plotting of the Child MeshMask fields.
lplotBat: Enable plotting of the Bathymetry fields.
lplotAtm: Enable plotting of the Atmospheric fields.
lplotOceIC: Enable plotting of the Ocean Initial Condition fields.
lplotOceBC: Enable plotting of the Open Boundary Condition Ocean fields.
lplotOceBCbdy: Enable plotting of the Open Boundary Condition Ocean fields.
lplotTide: Enable plotting of the Tidal fields.
lplotTidebdy: Enable plotting of the Tidal fields.
lplotOceOut: Enable plotting of the Output Ocean fields.

Here’s an example of the set_visual_lplot section in setParFree.json:

{
   "id": "B001",
   "title": "set_visual_lplot",
   "items": [
      { "name": "lplotMesh", "value": "True" },
      { "name": "lplotBat", "value": "True" },
      { "name": "lplotAtm", "value": "True" },
      { "name": "lplotOceIC", "value": "True" },
      { "name": "lplotOceBC", "value": "True" },
      { "name": "lplotOceBCbdy", "value": "True" },
      { "name": "lplotTide", "value": "True" },
      { "name": "lplotTidebdy", "value": "True" },
      { "name": "lplotOceOut", "value": "True" }
   ]
}

Make a new experiments¶

Let's assume you want create a new experiment to study the circulation of the Sermilik fjord in Greenland from 1 February 2017 at 00:00 to 7 February 2017 at 24:00. You can follow these steps:

First, select an appropriate name for your experiment ID. For instance, you could name it something like greenlandFjord_20170201
Navigate to the from_GUI directory, and create a new folder for this experiment.
```
cd /scratch/from_GUI/; mkdir greenlandFjord_20170201
```
For this experiment, you will need a base configuration file to customize. Copy the template configuration file /scratch/surf/surf_nemo/current/setParFree.json in the directory /scratch/from_GUI/greenlandFjord_20170201
```
cp /scratch/surf/surf_nemo/current/setParFree.json ./greenlandFjord_20170201/
```
Modify the user configuration file setParFree.json as needed for the Sermilik fjord experiment. You should adjust configuration parameters like domain, time period, and specific physical parameters.
```
param1 = xxx
param2 = xxx 
param3 = xxx 
param4 = xxx
```
Once your configuration file is ready, it’s time to run the experiment. Navigate to the scripts directory where the simulation script run_exp.jl is located, and execute the simulation using Julia, passing the experiment ID greenlandFjord_20170201 as an argument.
```
cd /scratch/surf/surf_nemo/current/scripts/;
julia run_exp.jl greenlandFjord_20170201
```
Once the simulation has successfully completed, the next step is to analyze and visualize the results. Run the post-processing Julia script run_postproc.jl, followed by the experiment ID greenlandFjord_20170201:
```
cd /scratch/surf/surf_nemo/current/scripts/;
julia run_postproc.jl greenlandFjord_20170201
```

In principle you can simply use the template model and modify it to your needs, and not be too much concerned with the input files they create. But our advice is never to use the template model as black boxes. It is therefore important to understand how the codes work, which options they have and how their input files are structured.

Multiple downscaling experiments¶

SURF-NEMO package includes multiple nesting capability (i.e. consecutive nested models can be implemented with increasing grid resolutions). Let’s assume you want to increase the spatial resolution of an existing experiment, such as the template experiment gulfTaranto_20141005, downscaling it to a finer resolution of 800 meters. Below are the steps to achieve this.

Go to the existing experiment directory

cd /scratch/surf/experiments/gulfTaranto_20141005/

Modify the user configuration file setParFree.json as needed for the multiple nesting experiment. You should adjust configuration parameters like domain, time period, and specific physical parameters.
```
param1 = xxx
param2 = xxx 
param3 = xxx 
param4 = xxx
```
Once the configuration file is ready, from the directory /scratch/surf/experiments/gulfTaranto_20141005/code/ocean/scripts/, execute the Julia script run_exp.jl followed by the experiment ID gulfTaranto_20141005
```
cd /scratch/surf/experiments/gulfTaranto_20141005/code/ocean/scripts/;
julia run_exp.jl gulfTaranto_20141005
```
After the simulation has completed, you can analyze and visualize the results. Run the Julia script run_postproc.jl, followed by the experiment ID gulfTaranto_20141005:
```
cd /scratch/surf/experiments/gulfTaranto_20141005/code/ocean/scripts/;
julia run_postproc.jl gulfTaranto_20141005
```