Acessing Amazon Web Services (AWS)

Step 1: Acquire AWS Access Files

A JEDI Master (Mark) will provide you with three files:

  1. Academy-oregon.pem
  2. aws_init.sh
  3. aws_init.csh

The first contains an ssh key that is necessary to log into AWS instances. The other two files are shell scripts that you will use to set up your AWS access. Both have an identical function and you will use one or the other depending on whether you use the bash shell or some variant of the C shell (e,g. csh or tcsh).

The first file, Academy-oregon.pem will be used throughout the week. However, you will need a new aws_init initialization script at the start of each day. The reason for this is because we wish to use our AWS resources efficiently. At the end of each day, we will halt our fleet of AWS Elastic Compute Cloud (EC2) instances because we won’t be using them overnight. The disk content will be saved (so you can pick up where you left off the previous day), but the servers will be shut down. When we restart them the next morning they will have different ip addresses. The new script that will be provided each day will contain the ip addresses that are valid for that day.

In order for the ssh key file to work, you will have to make sure that you disable read permission for anyone other than the user (if you don’t do this, ssh will throw an error):

chmod 400 Academy-oregon.pem

Step 2: Log in to AWS

If you use bash, enter this from a terminal command line:

source aws_init.sh  # bash

Or, alternatively, if you use tcsh or csh, enter

source aws_init.csh  # tcsh, csh

The script will prompt you for your group number; enter it as requested and hit return.

Each of these scripts does two things. First, it sets the environment variable AWSIP to be equal to the ip address of your group’s EC2 instance (compute node). Second, it defines the alias ssh-aws, which you can use to log in to your group’s EC2 instance. So, to log into your group’s EC2 instance, just enter:

ssh-aws

You will likely get a warning asking if you are sure you want to continue connecting. Enter yes to add this ip address to ssh’s list of known hosts.

Step 3: Initialize Your AWS Environment

After you successfully log in to your group’s EC2 instance, source the startup script in the home directory - be prepared to enter your group number at the prompt:

source startup.sh

The most important function of this script is to set up the GitHub login credentials for your group so you can access the JEDI code repositories without being prompted for a user name and password. Be sure you enter the correct group number (you can run it again if necessary) because this will allow you to participate in team exercises on GitHub such as pushes, pull requests, and code reviews. The script also sets up a few other environment variables that you will need to compile and run JEDI.

Step 4: Build and Run JEDI

The source code for the FV3 bundle is in the subdirectory jedi/fv3-bundle but you will compile and run it in the build directory jedi/build. So, to compile JEDI, run the following commands.

cd ~/jedi/build
rm -rf *
ecbuild --build=release ../fv3-bundle
make update
make -j4

Each step may take a few minutes so please be patient. The rm -rf * step is not necessary the first time you build the code but it is often useful to clean the build directory before you start a new build. The make update step will pull the latest code changes from the GitHub repositories.

Now you are ready to run JEDI. You can start by running the test suite as follows:

ctest

You can also run other applications as described elsewhere.

Contact Mark Miesch (miesch@ucar.edu) if you have any problems or questions.