Wednesday Exercise 1.1: Try an OSG Connect Software Module

In this exercise, we will submit a similar job to yesterday, but use OSG Connect's built-in Python software module to run our software.

Setup

Make sure you are logged into training.osgconnect.net (the OSG Connect submit server for this workshop). cd into your directory within the /local-scratch folder (cd /local-scratch/<USERNAME>). Copy the fib.py script from yesterday into this folder.

Modules on OSG Connect

  1. The software installed in the OSG Connect software repository is able to viewed and used via a module system. To see the available software modules, you can type:

    [email protected] $ module avail
    
  2. If you want to search for a specific module, you can use the module spider command. For this example, we want to use python, so let's look for it:

    [email protected] $ module spider python
    

    What is the name of the available Python modules?

  3. Finally, in order to use the available software, the software module has to be "loaded." First, let's check which version of Python is available by default:

    [email protected] $ python --version
    
  4. Now, what happens after we load the python/3.7.0 module?

    [email protected] $ module load python/3.7.0
    [email protected] $ python --version
    

    Note that we won't be actually running Python on this server, but we'll use the same command inside the job to "activate" the Python installation.

Using Modules in Jobs

  1. To use the modules we've just seen in jobs, we can load it via a script. Take a moment to consider which commands should go into this script, and then proceed.

  2. The job's executable script should look like this:

    #!/bin/bash
    
    module load python/3.7.0
    python3 fib.py 90
    
  3. The submit file should like something like the submit files you used yesterday. Besides including requests for cpus, memory and disk, and transferring the python script, the submit file for this job should include a list of arguments that ensures the job will only run on servers where the OSG Connect software repository and modules are available:

    requirements = (HAS_MODULES =?= true) && (OSGVO_OS_STRING == "RHEL 7") && (OpSys == "LINUX")
    
  4. Submit this job. Does it produce the expected results?