Wednesday Exercise 1.9: Using Docker¶
In this exercise, you will install run the same Python script as the previous exercises, but using a Docker container.
Containers are another way to manage software installations. We don't have the time to go fully into the details of building and using containers, but can use a publicly available Python container to run our Python job.
One caveat for using containers: not all systems will support them. HTCondor has built-in features for using Docker and many Open Science Grid resources have Singularity installed, but they are still not in widespread use.
Submit File Changes¶
- Make a copy of your submit file from the previous exercise.
- Add the following three lines to the submit file or modify existing lines to match the lines below: \
universe = docker docker_image = python requirements = (OpSysMajorVer == 7)
\ Here we are requesting HTCondor's Docker universe and using a pre-built python image that, by default, will be pulled from a public website of Docker images called DockerHub. \ The requirements line will ensure that we run on computers whose operating system can support Docker.
- Adjust the executable and arguments lines. The executable can now be the Python script itself, with the appropriate arguments: \
executable = fib.py arguments = 5
- Finally, we no longer need to transfer a Python tarball (whether source code or pre-built) or our Python script. You can remove both from the
line of the submit file.
- Open the Python script and add the following line at the top: \
\ This will ensure that the script uses the version of Python that comes in the Docker container.
Once these steps are done, submit the job.