Container Development Guide¶

This document contains instructions for OSG Technology Team members, including:

How to to develop OSG Software container images that are automatically pushed to Docker Hub that adhere to our container release policy
How to build a new version of an existing image
How to manage tags for images in the OSG DockerHub organization
Tips for container image development

Creating New OSG Software Containers¶

OSG Software service container images intended for OSG site admin use need to be automatically updated once per week to pick up any OS updates, as well as upon any changes to the images themselves. To do this, we use GitHub Actions to:

Build new images on commits to master or main
Update the docker-software-base on a schedule, which triggers builds for all image repos through repository dispatch
Push container images to Docker Hub and the OSG Container Registry (Harbor)

Code for new container images should now be stored in the GitHub repository opensciencegrid/images; see below for instructions.

The previous convention was to use individual GitHub repos for each image; these are described below.

Add image code to the opensciencegrid/images repo¶

Images are automatically built from the subdirectories of the opensciencegrid directory. A set of images will be built for each subdirectory, with each set containing multiple images based on OSG release series (e.g. 3.5, 3.6) and release level (e.g. testing, release).

Fork and clone the https://github.com/opensciencegrid/images repo
Create a subdirectory under opensciencegrid/
Create a README.md file describing the software provided by the image
Create a LICENSE file containing the Apache 2.0 license text

Create a Dockerfile building from the OSG Software Base image:

ARG BASE_OSG_SERIES=3.6
ARG BASE_YUM_REPO=release

FROM opensciencegrid/software-base:$BASE_OSG_SERIES-<DISTRO VERSION>-$BASE_YUM_REPO

# Previous instance has gone out of scope
ARG BASE_OSG_SERIES=3.6
ARG BASE_YUM_REPO=release

LABEL maintainer OSG Software <[email protected]>

RUN yum install -y <PACKAGE(S)> && \
    yum clean all && \
    rm -rf /var/cache/yum/*

Replacing <DISTRO VERSION> with the Enterprise Linux version abbreviation (e.g., el7, el8), and <PACKAGE(S)> with the RPM(s) you'd like to provide in this image.

Hardcoding OSG series or release level

If you do not want to build your image for all release series (for example, it's 3.6-only), or you do not want to build your image for all release levels (for example, always build from release), hardcode those instead of using the arguments, as in:

ARG BASE_OSG_SERIES=3.6

FROM opensciencegrid/software-base:$BASE_OSG_SERIES-el8-release

Prepare the Docker Hub repository¶

Ask the Software Manager to create a Docker Hub repo in the OSG organization. The name should generally match the subdirectory name under the images repo.
Go to the permissions tab and give the robots and technology teams Read & Write access

Old Image Repositories¶

Some container images are stored in individual repositories under the opensciencegrid organization, with names prefixed with docker- and with the container repository topic.

New images should not be created this way, but existing images may need to be updated or fixed; this section describes their layout and mechanics.

Old image repos will have:

A README.md file describing the software provided by the image
A LICENSE file containing the Apache 2.0 license text
A Dockerfile based off of the OSG Software Base image:
```
FROM opensciencegrid/software-base:<OSG RELEASE SERIES>-<EL MAJOR VERSION>-release

LABEL maintainer OSG Software <[email protected]>

RUN yum update -y && \
    yum clean all && \
    rm -rf /var/cache/yum/*

RUN yum install -y <PACKAGE> && \
    yum clean all && \
    rm -rf /var/cache/yum/*
```
Replacing <PACKAGE> with the name of the RPM you'd like to provide in this container image, <OSG RELEASE SERIES> with the OSG release series version (e.g., 3.6), and <EL MAJOR VERSION> with the Enterprise Linux major version (e.g., 7).

The BASE_OSG_SERIES and BASE_YUM_REPO arguments may or may not be used.
The pre-defined Publish OSG Software container image workflow, found under the Actions tab.

The user "osg-bot" needs to have the "Write" role for this repo in order to trigger automatic builds.
Access to the following organizational secrets
- DOCKER_USERNAME
- DOCKER_PASSWORD
- OSG_HARBOR_ROBOT_USER
- OSG_HARBOR_ROBOT_PASSWORD
The repo may also have access to the REPO_ACCESS_TOKEN organization secret, if it needs to send dispatches to another repository (e.g. docker-software-base).
A Docker Hub repository with a name matching the GitHub repo name, without the docker- prefix, with Read & Write access for the robots and technology teams.

In addition, for repository dispatch from docker-software-base, they are listed in the GitHub Actions workflow for docker-software-base, in the dispatch-repo: list (under jobs: dispatch: strategy: matrix:).

Triggering Container Image Builds¶

To build a new version of an existing container image, e.g. for a new RPM version of software in the container, you can kick off a new build in one of two ways:

If there are no changes necessary to the container packaging: go to the GitHub repository's latest build under Actions, e.g. https://github.com/opensciencegrid/docker-frontier-squid/actions/, and click "Re-run jobs" -> "Re-run all jobs".
If changes need to be made to the container packaging: submit a pull request with your changes to the relevant GitHub repository and request that another team member review it. Once merged into master or main, a GitHub Actions build should start automatically.

If the GitHub Actions build completes successfully, you should shortly see new fresh and timestamp tags appear in the DockerHub repository.

Automatic weekly rebuilds

If the repo's GitHub Actions are configured as above, container images will automatically rebuild, and therefore pick up new packages available in minefield once per week.

Managing Tags in DockerHub¶

Adding tags¶

Images that have passed acceptance testing should be tagged as stable:

Install the jq utility:
```
yum install jq
```
Get the sha256 repo digest of the image that the user has tested. All you need is the part that starts with sha256:... (aka the <DIGEST>).

A Kubernetes user can get the digest from the "Image ID" line obtained by running:
```
 kubectl describe pod <POD>
```
A Docker user can get the digest by running:
```
 docker image inspect <IMAGE NAME>:<TAG> | jq '.[0].Id'
```
(Optional) If you are tagging multiple images, you can enter your Docker Hub username and password into environment variables, to avoid having to re-type them. Otherwise the script will prompt for them.
```
read user     # enter dockerhub username
read -s pass  # enter dockerhub password
export user pass
```
Run the Docker container image tagging command from release-tools:
```
./dockerhub-tag-fresh-to-stable.sh <IMAGE NAME> <DIGEST>
```

Removing tags¶

Run the Docker container image pruning command from release-tools:

./dockerhub-prune-tags.py <IMAGE NAME>

Making Slim Containers¶

Here are some resources for creating slim, efficient containers: