################################### Acessing Amazon Web Services (AWS) ################################### Step 1: Acquire AWS Access Files -------------------------------- A JEDI Master (Mark) will provide you with three files: 1. :code:`academy-virginia.pem` 2. :code:`aws_init.sh` 3. :code:`aws_init.csh` The first contains an ssh key that is necessary to log into AWS instances. The other two files are shell scripts that you will use to set up your AWS access. Both have an identical function and you will use one or the other depending on whether you use the :code:`bash` shell or some variant of the C shell (e,g. :code:`csh` or :code:`tcsh`). The first file, :code:`academy-virginia.pem` will be used throughout the week. However, **you will need a new** :code:`aws_init` **initialization script at the start of each day**. The reason for this is because we wish to use our AWS resources efficiently. At the end of each day, we will halt our fleet of AWS Elastic Compute Cloud (EC2) instances because we won't be using them overnight. The disk content will be saved (so you can pick up where you left off the previous day), but the servers will be shut down. When we restart them the next morning they will have different ip addresses. The new script that will be provided each day will contain the ip addresses that are valid for that day. In order for the ssh key file to work, you will have to make sure that you disable read permission for anyone other than the user (if you don't do this, ssh may throw an error): .. code:: bash chmod 400 academy-virginia.pem We will have 20 groups, with 2 padawans per group. Each group has their own computer in the computer lab, their own AWS compute node, and their own GitHub account. The group numbers are assigned in the table below. .. pull-quote:: **It is important to know your group number** Making a mistake here will mean that you will be working in the work space of another group. ======== ========== ==================== Group Machine GitHub user name ======== ========== ==================== 1 112 jedi-ac01 2 114 jedi-ac02 3 116 jedi-ac03 4 118 jedi-ac04 5 120 jedi-ac05 6 122 jedi-ac06 7 124 jedi-ac07 8 126 jedi-ac08 9 128 jedi-ac09 10 130 jedi-ac10 11 132 jedi-ac11 12 134 jedi-ac12 13 136 jedi-ac13 14 138 jedi-ac14 15 140 jedi-ac15 16 142 jedi-ac16 17 144 jedi-ac17 18 146 jedi-ac18 19 148 jedi-ac19 20 150 jedi-ac20 ======== ========== ==================== Step 2: Log in to AWS --------------------- If you use :code:`bash`, enter this from a terminal command line: .. code:: bash source aws_init.sh # bash Or, alternatively, if you use :code:`tcsh` or :code:`csh`, enter .. code:: bash source aws_init.csh # tcsh, csh The script will prompt you for your group number; enter it as requested and hit return. Each of these scripts does two things. First, it sets the environment variable :code:`AWSIP` to be equal to the ip address of your group's EC2 instance (compute node). Second, it defines the alias :code:`ssh-aws`, which you can use to log in to your group's EC2 instance. So, to log into your group's EC2 instance, just enter: .. code:: bash ssh-aws You will likely get a warning asking if you are sure you want to continue connecting. Enter :code:`yes` to add this ip address to ssh's list of known hosts. Step 3: Initialize Your AWS Environment ---------------------------------------- After you successfully log in to your group's EC2 instance, source the startup script in the home directory - be prepared to enter your group number at the prompt: .. code:: bash source startup.sh The most important function of this script is to set up the GitHub login credentials for your group so you can access the JEDI code repositories without being prompted for a user name and password. Be sure you enter the correct group number (you can run it again if necessary) because this will allow you to participate in team exercises on GitHub such as pushes, pull requests, and code reviews. The script also sets up a few other environment variables that you will need to compile and run JEDI. Next, you will need to load the jedi module. This contains the compilers, the MPI library, and the other software dependencies that you will need to build and run JEDI. The JEDI test and application nodes have a number of modules available but the academy node only has a version of the code compiled with version 7.4 of the gnu compiler suite (gcc, g++, gfortran) and version 3.2.1 of OpenMPI. To load this module, enter: .. code:: bash module load jedi/gnu-openmpi This is really a meta-module that loads a number of JEDI dependencies. To see a list of the software you now have access to, enter .. code:: bash module list Step 4: Build and Run JEDI -------------------------- Now you are ready to build JEDI. The source code for the FV3 bundle is in the subdirectory :code:`jedi/src/fv3-bundle` but you will compile and run it in the build directory :code:`jedi/build`. To do so, enter the following commands .. code:: bash cd ~/jedi/build rm -rf * ecbuild --build=Release ../src/fv3-bundle make update make -j4 Each step may take a few minutes so please be patient. The :code:`rm -rf *` step is not necessary the first time you build the code but it is good practice to clean the build directory before you start a new build. The :code:`--build=Release` option is not necessary - it may be omitted - but if there, it tells ecbuild to optimize the code so it runs more efficiently. The :code:`make update` step will pull the latest code changes from the GitHub repositories. Now you are ready to run JEDI. You can start by running the test suite as follows: .. code:: bash ctest You can also run other applications as described elsewhere. Contact Mark Miesch (miesch@ucar.edu) if you have any problems or questions.