OSG Release Series¶
An OSG release series is a sequence of OSG software releases that are intended to provide a painless upgrade path. For example, the 3.2 release series contains OSG software 3.2.0, 3.2.1, 3.2.2, and so forth. A release series corresponds to a set of Yum software repositories, including ones for development, testing, and production use. The Yum repositories for one release series are completely distinct from the repositories for a different release series, even though they share many common packages. A particular release within a series is a snapshot of packages and their exact versions at one point in time. When you install software from a release series, say 3.2, you get the most current versions of software packages within that series, regardless of the current release version.
When a new series is released, it is an opportunity for the OSG Technology area to add major new software packages, make substantial updates to existing packages, and remove obsolete packages. When a new series is initially released, most packages are identical to the previous release, but two adjacent series will diverge over time.
Our goal is, within a series, that one may upgrade their OSG services via
yum update cleanly and without any necessary
config file changes or excessive downtime.
OSG Release Series¶
Since the start of the RPM-based OSG software stack, we have offered the following release series:
OSG 3.5 started August 2019. The main differences between it and 3.4 were the introduction of the HTCondor 8.8 and 8.9 series; also the RSV monitoring probes, EL6 support, and CREAM support were all dropped.
OSG 3.4 started June 2017 and will reach its end-of-life in November 2020. The main differences between it and 3.3 are the removal of edg-mkgridmap, GUMS, BeStMan, and VOMS Admin Server packages.
OSG 3.3 started in August 2015 and was end-of-lifed in May 2018. While the files have not been removed, it is strongly recommended that it not be installed anymore. The main differences between 3.3 and 3.2 are the dropping of EL5 support, the addition of EL7 support, and the dropping of Globus GRAM support.
OSG 3.2 started in November 2013, and was end-of-lifed in August 2016. The main differences between it and 3.1 were the introduction of glideinWMS 3.2, HTCondor 8.0, and Hadoop/HDFS 2.0; also the gLite CE Monitor system was dropped in favor of osg-info-services.
OSG 3.1 started in April 2012, and was end-of-lifed in April 2015. Historically, there were 3.0.x releases as well, but there was no separate release series for 3.0 and 3.1; we simply went from 3.0.10 to 3.1.0 in the same repositories.
There is one more OSG Series called "upcoming" which contains major updates planned for a future release series.
The yum repositories for upcoming (
osg-upcoming-testing) are available from all OSG 3.x series, and
individual packages can be installed from Upcoming without needing to update entirely to a new series.
Note, however, that packages in the "upcoming" repositories are tested against the most recent OSG series.
As of the time of writing,
osg-upcoming is meant to work with OSG 3.5.
Installing an OSG Release Series¶
See the yum repositories document for instructions on installing the OSG repositories.
Updating to OSG 3.5¶
OS Version Support
OSG 3.5 only supports EL7
If you have an existing installation based on OSG release version <= 3.4 (which will be referred to as the old series), and want to upgrade to 3.5 (the new series), we recommend the following procedure:
First, remove the old series yum repositories:
[email protected] # rpm -e osg-release
This step ensures that any local modifications to
*.repofiles will not prevent installing the new series repos. Any modified
*.repofiles should appear under
*.rpmsaveextension. After installing the new OSG repositories (the next step) you may want to apply any changes made in the
*.rpmsavefiles to the new
Install the OSG repositories:
[email protected] # yum install https://repo.opensciencegrid.org/osg/3.5/osg-3.5-el7-release-latest.rpm
Clean yum cache:
[email protected] # yum clean all --enablerepo=*
[email protected] # yum update
- Please be aware that running
yum updatemay also update other RPMs. You can exclude packages from being updated using the
--exclude=[package-name or glob]option for the
- Watch the yum update carefully for any messages about a
.rpmnewfile being created. That means that a configuration file had been edited, and a new default version was to be installed. In that case, RPM does not overwrite the edited configuration file but instead installs the new version with a
.rpmnewextension. You will need to merge any edits that have made into the
.rpmnewfile and then move the merged version into place (that is, without the
.rpmnewextension). Watch especially for
/etc/lcmaps.db, which every site is expected to edit.
- Please be aware that running
Remove any deprecated packages that were previously installed:
[email protected] # yum remove osg-version \ osg-control \ 'rsv*' \ glite-ce-cream-client-api-c \ glite-lbjp-common-gsoap-plugin \ xacml
If you did not have any of the above packages installed, Yum will not remove any packages:
No Match for argument: osg-version No Match for argument: osg-control No Match for argument: rsv* No Match for argument: glite-ce-cream-client-api-c No Match for argument: glite-lbjp-common-gsoap-plugin No Match for argument: xacml No Packages marked for removal
Running into issues?
If you are not having the expected result or having problems with Yum please see the Yum troubleshooting guide
Updating to HTCondor-CE 4.x¶
The OSG 3.5 release series contains HTCondor-CE 4, a major version upgrade from the previously released versions in the OSG. See the HTCondor-CE 4.0.0 release notes for an overview of the changes. In particular, this version includes a major reorganization of the default configuration so updates will require manual intervention. To update your HTCondor-CE host(s), perform the following steps:
Update all CE packages:
[email protected] # yum update htcondor-ce 'osg-ce*'
The new default
condor_mapfileis sufficient since HTCondor-CE no longer relies on GSI authentication between its daemons. If
/etc/condor-ce/condor_mapfile.rpmnewexists, replace your old
[email protected] # mv /etc/condor-ce/condor_mapfile.rpmnew /etc/condor-ce/condor_mapfile
Additionally, you may wish to make one or more of the following optional changes:
HTCondor-CE now disables batch system job retries by default. To re-enable job retries, set the following configuration in
ENABLE_JOB_RETRIES = True
For non-HTCondor sites that use remote CE requirements, the new version of HTCondor-CE accepts a simplified format. For example, a snippet from an example job route in the old format:
set_default_remote_cerequirements = strcat("Walltime == 3600 && AccountingGroup =="", x509UserProxyFirstFQAN, "\"");
May be rewritten as the following:
set_WallTime = 3600; set_AccountingGroup = x509UserProxyFirstFQAN; set_default_CERequirements = "Walltime,AccountingGroup";
Reload and restart the HTCondor-CE daemons:
Updating to HTCondor 8.8.x¶
The OSG 3.5 release series contains HTCondor 8.8, a major version upgrade from the previously released versions in the OSG. See the detailed update instructions below to update to HTCondor 8.8.
Updating to OSG Configure 3¶
The OSG 3.5 release series contains OSG-Configure 3, a major version upgrade from the previously released versions in the OSG. See the OSG Configure release notes for an overview of the changes. To update OSG Configure on your HTCondor-CE, perform the following steps:
If you haven't already, update to OSG 3.5.
If you have
/etc/osg/config.d/40-siteinfo.ini, delete it and specify
resourceshould match the resource name that's registered in OSG Topology.
/etc/osg/config.d/40-siteinfo.inito the resource group registered in OSG Topology, i.e. the name of the
.yamlfile in OSG Topology that contains the registered resouce above.
host_nameto the host name that is registered in OSG Topology. This may be different from the FQDN of the host if you're using a DNS alias, for example.
OSG Configure will warn about config options that it does not recognize; delete these options from the config to get rid of the warnings.
Updating to HTCondor 8.9.7+¶
Where to find HTCondor 8.9
For HTCondor hosts < 8.9.7 using the SciTokens CredMon, updates to HTCondor 8.9.7+ require manual intervention and a
corresponding update to
python2-scitokens-credmon, available in the OSG 3.5 release repository.
If you do not have the
python2-scitokens-credmon package installed, you may skip these instructions.
Otherwise, follow these steps for a seamless update to HTCondor 8.9.7+:
Determine if your HTCondor installation is configured to use the SciTokens CredMon:
# condor_config_val -v DAEMON_LIST
SEC_CREDENTIAL_MONITORare in the output of the above command, continue onto the next step. Otherwise, your installation is not configured to use SciTokens CredMon and you should skip the rest of these instructions.
Add the following to a file in
SEC_CREDENTIAL_DIRECTORY_OAUTH = /var/lib/condor/oauth_credentials CREDMON_OAUTH = /usr/bin/condor_credmon_oauth SEC_CREDENTIAL_MONITOR_OAUTH_LOG = $(LOG)/CredMonOAuthLog if version < 8.9.7 CREDMON_OAUTH = /usr/bin/scitokens_credmon endif
DAEMON_LIST = $(DAEMON_LIST), CREDD, SEC_CREDENTIAL_MONITOR
DAEMON_LIST = $(DAEMON_LIST), CREDD, CREDMON_OAUTH
Turn off the SchedD and CredMon daemons:
# condor_off -daemon SCHEDD # condor_off -daemon SEC_CREDENTIAL_MONITOR # condor_off -daemon CREDMON_OAUTH
Move the existing credential directory and set up a temporary symlink:
# mv $(condor_config_val SEC_CREDENTIAL_DIRECTORY) $(condor_config_val SEC_CREDENTIAL_DIRECTORY_OAUTH) # ln -s $(condor_config_val SEC_CREDENTIAL_DIRECTORY_OAUTH) $(condor_config_val SEC_CREDENTIAL_DIRECTORY)
Update HTCondor and SciTokens CredMon packages:
# yum -y upgrade python2-scitokens-credmon condor
If you are running Apache on this host, reload the Apache configuration:
# systemctl reload httpd.service
Turn the SchedD and CredMon daemons back on:
# condor_on -daemon CREDMON_OAUTH # condor_on -daemon SCHEDD
Clean up old CredMon configuration. Remove the following entries from your HTCondor configuration:
SEC_CREDENTIAL_DIRECTORY = /var/lib/condor/credentials SEC_CREDENTIAL_MONITOR = /usr/bin/scitokens_credmon SEC_CREDENTIAL_MONITOR_LOG = $(LOG)/CredMonLog
To allow for seamless HTCondor downgrades, update the
if version < 8.9.7block that you added in step 2.
if version < 8.9.7 CREDMON_OAUTH = /usr/bin/scitokens_credmon SEC_CREDENTIAL_DIRECTORY = $(SEC_CREDENTIAL_DIRECTORY_OAUTH) endif
Remove the symlink to the old credential directory that you created in step 5. This is whatever you had set
SEC_CREDENTIAL_DIRECTORYto before. For example:
# rm /var/lib/condor/credentials
Updating to HTCondor 8.8.x¶
The OSG 3.5 and 3.4 release series contain HTCondor 8.8, a major version upgrade from the previously released versions in the OSG. See the HTCondor 8.8 manual for an overview of the changes. To update HTCondor on your HTCondor-CE and/or HTCondor pool hosts, perform the following steps:
Update all HTCondor packages:
[email protected] # yum update 'condor*'
HTCondor pools only:
CONDOR_HOSTconfiguration changed in HTCondor 8.8. Additionally in OSG 3.5, the default security was changed to use FS and pool password. If you are experiencing issues with communication between hosts in your pool after the upgrade, the default OSG configuration is listed in
/etc/condor/config.d/00-osg_default_*.config: ensure that any default configuration is overriden with your own
CONDOR_HOST, and/or security configuration in subsequent files.
As of HTCondor 8.8, MOUNT_UNDER_SCRATCH has default values of
/var/tmp, which may cause issues if your OSG_WN_TMP is a subdirectory of either of these directories. If the partition containing your execute directories is large enough, we recommend setting your
/var/tmp. If that partition is not large enough, we recommend setting your
OSG_WN_TMPvariable to a directory outside of
HTCondor-CE hosts only: The HTCondor 8.8 series changed the default job route matching order from round-robin to first matching route. To use the old round-robin matching order, add the following configuration to
JOB_ROUTER_ROUND_ROBIN_SELECTION = True
Clean-up deprecated packages:
[email protected] # yum remove 'rsv*' glite-ce-cream-client-api-c