Skip to content

Container Development Guide

This document contains instructions for OSG Technology Team members, including:

  • How to to develop OSG Software container images that are automatically pushed to Docker Hub that adhere to our container release policy
  • How to build a new version of an existing image
  • How to manage tags for images in the OSG DockerHub organization
  • Tips for container image development

Creating New OSG Software Containers

OSG Software service container images intended for OSG site admin use need to be automatically updated once per week to pick up any OS updates, as well as upon any changes to the images themselves. To do this, we use GitHub Actions to:

  1. Build new images on commits to master or main
  2. Update the docker-software-base on a schedule, which triggers builds for all image repos through repository dispatch
  3. Push container images to Docker Hub and the OSG Container Registry (Harbor)

Code for new container images should now be stored in the GitHub repository opensciencegrid/images; see below for instructions.

The previous convention was to use individual GitHub repos for each image; these are described below.

Add image code to the opensciencegrid/images repo

Images are automatically built from the subdirectories of the opensciencegrid directory. A set of images will be built for each subdirectory, with each set containing multiple images based on OSG release series (e.g. 3.5, 3.6) and release level (e.g. testing, release).

  1. Fork and clone the https://github.com/opensciencegrid/images repo
  2. Create a subdirectory under opensciencegrid/
  3. Create a README.md file describing the software provided by the image
  4. Create a LICENSE file containing the Apache 2.0 license text
  5. Create a Dockerfile building from the OSG Software Base image:

    ARG BASE_OSG_SERIES=3.6
    ARG BASE_YUM_REPO=release
    
    FROM opensciencegrid/software-base:$BASE_OSG_SERIES-<DISTRO VERSION>-$BASE_YUM_REPO
    
    # Previous instance has gone out of scope
    ARG BASE_OSG_SERIES=3.6
    ARG BASE_YUM_REPO=release
    
    LABEL maintainer OSG Software <[email protected]>
    
    RUN yum install -y <PACKAGE(S)> && \
        yum clean all && \
        rm -rf /var/cache/yum/*
    

    Replacing <DISTRO VERSION> with the Enterprise Linux version abbreviation (e.g., el7, el8), and <PACKAGE(S)> with the RPM(s) you'd like to provide in this image.

Hardcoding OSG series or release level

If you do not want to build your image for all release series (for example, it's 3.6-only), or you do not want to build your image for all release levels (for example, always build from release), hardcode those instead of using the arguments, as in:

ARG BASE_OSG_SERIES=3.6

FROM opensciencegrid/software-base:$BASE_OSG_SERIES-el8-release

Prepare the Docker Hub repository

  1. Ask the Software Manager to create a Docker Hub repo in the OSG organization. The name should generally match the subdirectory name under the images repo.
  2. Go to the permissions tab and give the robots and technology teams Read & Write access

Old Image Repositories

Some container images are stored in individual repositories under the opensciencegrid organization, with names prefixed with docker- and with the container repository topic.

New images should not be created this way, but existing images may need to be updated or fixed; this section describes their layout and mechanics.

Old image repos will have:

  1. A README.md file describing the software provided by the image
  2. A LICENSE file containing the Apache 2.0 license text
  3. A Dockerfile based off of the OSG Software Base image:

    FROM opensciencegrid/software-base:<OSG RELEASE SERIES>-<EL MAJOR VERSION>-release
    
    LABEL maintainer OSG Software <[email protected]>
    
    RUN yum update -y && \
        yum clean all && \
        rm -rf /var/cache/yum/*
    
    RUN yum install -y <PACKAGE> && \
        yum clean all && \
        rm -rf /var/cache/yum/*
    

    Replacing <PACKAGE> with the name of the RPM you'd like to provide in this container image, <OSG RELEASE SERIES> with the OSG release series version (e.g., 3.6), and <EL MAJOR VERSION> with the Enterprise Linux major version (e.g., 7).

    The BASE_OSG_SERIES and BASE_YUM_REPO arguments may or may not be used.

  4. The pre-defined Publish OSG Software container image workflow, found under the Actions tab.

    The user "osg-bot" needs to have the "Write" role for this repo in order to trigger automatic builds.

  5. Access to the following organizational secrets

    • DOCKER_USERNAME
    • DOCKER_PASSWORD
    • OSG_HARBOR_ROBOT_USER
    • OSG_HARBOR_ROBOT_PASSWORD

    The repo may also have access to the REPO_ACCESS_TOKEN organization secret, if it needs to send dispatches to another repository (e.g. docker-software-base).

  6. A Docker Hub repository with a name matching the GitHub repo name, without the docker- prefix, with Read & Write access for the robots and technology teams.

In addition, for repository dispatch from docker-software-base, they are listed in the GitHub Actions workflow for docker-software-base, in the dispatch-repo: list (under jobs: dispatch: strategy: matrix:).

Triggering Container Image Builds

To build a new version of an existing container image, e.g. for a new RPM version of software in the container, you can kick off a new build in one of two ways:

  • If there are no changes necessary to the container packaging: go to the GitHub repository's latest build under Actions, e.g. https://github.com/opensciencegrid/docker-frontier-squid/actions/, and click "Re-run jobs" -> "Re-run all jobs".
  • If changes need to be made to the container packaging: submit a pull request with your changes to the relevant GitHub repository and request that another team member review it. Once merged into master or main, a GitHub Actions build should start automatically.

If the GitHub Actions build completes successfully, you should shortly see new fresh and timestamp tags appear in the DockerHub repository.

Automatic weekly rebuilds

If the repo's GitHub Actions are configured as above, container images will automatically rebuild, and therefore pick up new packages available in minefield once per week.

Managing Tags in DockerHub

Adding tags

Images that have passed acceptance testing should be tagged as stable:

  1. Install the jq utility:

    yum install jq
    
  2. Get the sha256 repo digest of the image that the user has tested. All you need is the part that starts with sha256:... (aka the <DIGEST>).

    A Kubernetes user can get the digest from the "Image ID" line obtained by running:

     kubectl describe pod <POD>
    

    A Docker user can get the digest by running:

     docker image inspect <IMAGE NAME>:<TAG> | jq '.[0].Id'
    
  3. (Optional) If you are tagging multiple images, you can enter your Docker Hub username and password into environment variables, to avoid having to re-type them. Otherwise the script will prompt for them.

    read user     # enter dockerhub username
    read -s pass  # enter dockerhub password
    export user pass
    
  4. Run the Docker container image tagging command from release-tools:

    ./dockerhub-tag-fresh-to-stable.sh <IMAGE NAME> <DIGEST>
    

Removing tags

Run the Docker container image pruning command from release-tools:

./dockerhub-prune-tags.py <IMAGE NAME>

Making Slim Containers

Here are some resources for creating slim, efficient containers: