Container Development Guide¶
This document contains instructions for OSG Technology Team members, including:
- How to to develop OSG Software container images that are automatically pushed to Docker Hub that adhere to our container release policy
- How to build a new version of an existing image
- How to manage tags for images in the OSG DockerHub organization
- Tips for container image development
Creating New OSG Software Containers¶
OSG Software service container images intended for OSG site admin use need to be automatically updated once per week to pick up any OS updates, as well as upon any changes to the images themselves. To do this, we use GitHub Actions to:
- Build new images on commits to
master
- Update the
docker-software-base
on a schedule, which triggers builds for all image repos through repository dispatch - Push container images to Docker Hub.
Prepare the GitHub repository¶
- Create a Git repository in the
opensciencegrid
organization whose name is prefixed withdocker-
, e.g.docker-frontier-squid
- Create a
README.md
file describing the software provided by the image - Create a
LICENSE
file containing the Apache 2.0 license text - Set the repository topic to
container
-
Create a
Dockerfile
based off of the OSG Software Base image:FROM opensciencegrid/software-base:fresh LABEL maintainer OSG Software <[email protected]> RUN yum update -y && \ yum clean all && \ rm -rf /var/cache/yum/* RUN yum install -y <PACKAGE> --enablerepo=osg-testing && \ yum clean all && \ rm -rf /var/cache/yum/*
Replacing
<PACKAGE>
with the name of the RPM you'd like to provide in this container image -
Add the pre-defined OSG Software container publishing GitHub Actions workflow. From the GitHub repository, perform the following steps:
- Go to the
Actions
tab - Select the
Publish OSG Software container image
workflow (you may have to clickAdd new workflow
first if the repository has existing workflows) - Click
Start commit
thenCommit new file
- Go to the
- Give write permissions to the "osg-bot" user for this GitHub repo, navigating to:
- "Settings"
- "Manage access"
- "Invite teams or people"
- Search for and select "osg-bot"
- Choose the "Write" role, and click the button to Add osg-bot to the repo. (The osg-bot user needs this permission in order to trigger automatic builds.)
- Ask the Software Manager to give this repo access to the
DOCKER_USERNAME
andDOCKER_PASSWORD
organizational secrets
Repository dispatch
Any repository that sends dispatches to another repository (e.g. docker-software-base
, docker-compute-entrypoint
)
needs access to the REPO_ACCESS_TOKEN
organization secret.
Ask the Software Manager for access.
Prepare the Docker Hub repository¶
- Ask the Software Manager to create a Docker Hub repo in the OSG organization.
The name should generally match the GitHub repo name, without the
docker-
prefix. - Go to the permissions tab and give the
robots
andtechnology
teamsRead & Write
access
Set up repository dispatch from docker-software-base¶
- Edit the
GitHub Actions workflow for docker-software-base,
and add the new GitHub repo name to the
dispatch-repo:
list (underjobs:
dispatch:
strategy:
matrix:
). - Make a Pull Request for your change.
Triggering Container Image Builds¶
To build a new version of an existing container image, e.g. for a new RPM version of software in the container, you can kick off a new build in one of two ways:
- If there are no changes necessary to the container packaging: go to the GitHub repository's latest build under Actions, e.g. https://github.com/opensciencegrid/docker-frontier-squid/actions/, and click "Re-run jobs" -> "Re-run all jobs".
- If changes need to be made to the container packaging: submit a pull request with your changes to the relevant
GitHub repository and request that another team member review it.
Once merged into
master
, a GitHub Actions build should start automatically.
If the GitHub Actions build completes successfully, you should shortly see new fresh
and timestamp tags appear in the
DockerHub repository.
Automatic weekly rebuilds
If the repo's GitHub Actions are configured as above, container images will automatically rebuild, and therefore pick up new packages available in minefield once per week.
Managing Tags in DockerHub¶
Adding tags¶
Images that have passed acceptance testing should be tagged as stable
:
-
Install the
jq
utility:yum install jq
-
Get the sha256 repo digest of the image that the user has tested. All you need is the part that starts with
sha256:...
(aka the<DIGEST>
).
A Kubernetes user can get the digest from the "Image ID" line obtained by running:
kubectl describe pod <POD>
A Docker user can get the digest by running:
docker image inspect <IMAGE NAME>:<TAG> | jq '.[0].Id'
-
(Optional) If you are tagging multiple images, you can enter your Docker Hub username and password into environment variables, to avoid having to re-type them. Otherwise the script will prompt for them.
read user # enter dockerhub username read -s pass # enter dockerhub password export user pass
-
Run the Docker container image tagging command from release-tools:
./dockerhub-tag-fresh-to-stable.sh <IMAGE NAME> <DIGEST>
Removing tags¶
Run the Docker container image pruning command from release-tools:
./dockerhub-prune-tags.py <IMAGE NAME>
Making Slim Containers¶
Here are some resources for creating slim, efficient containers: