OSG Release Series¶
An OSG release series is a sequence of OSG software releases that are intended to provide a painless upgrade path. For example, the 3.2 release series contains OSG software 3.2.0, 3.2.1, 3.2.2, and so forth. A release series corresponds to a set of Yum software repositories, including ones for development, testing, and production use. The Yum repositories for one release series are completely distinct from the repositories for a different release series, even though they share many common packages. A particular release within a series is a snapshot of packages and their exact versions at one point in time. When you install software from a release series, say 3.2, you get the most current versions of software packages within that series, regardless of the current release version.
When a new series is released, it is an opportunity for the OSG Technology area to add major new software packages, make substantial updates to existing packages, and remove obsolete packages. When a new series is initially released, most packages are identical to the previous release, but two adjacent series will diverge over time.
Our goal is, within a series, that one may upgrade their OSG services via
yum update cleanly and without any necessary
config file changes or excessive downtime.
OSG Release Series¶
Since the start of the RPM-based OSG software stack, we have offered the following release series:
OSG 3.5 started August 2019. The main differences between it and 3.4 were the introduction of the HTCondor 8.8 and 8.9 series; also the RSV monitoring probes, EL6 support, and CREAM support were all dropped.
OSG 3.4 started June 2017 and will reach its end-of-life in November 2020. The main differences between it and 3.3 are the removal of edg-mkgridmap, GUMS, BeStMan, and VOMS Admin Server packages.
OSG 3.3 started in August 2015 and was end-of-lifed in May 2018. While the files have not been removed, it is strongly recommended that it not be installed anymore. The main differences between 3.3 and 3.2 are the dropping of EL5 support, the addition of EL7 support, and the dropping of Globus GRAM support.
OSG 3.2 started in November 2013, and was end-of-lifed in August 2016. The main differences between it and 3.1 were the introduction of glideinWMS 3.2, HTCondor 8.0, and Hadoop/HDFS 2.0; also the gLite CE Monitor system was dropped in favor of osg-info-services.
OSG 3.1 started in April 2012, and was end-of-lifed in April 2015. Historically, there were 3.0.x releases as well, but there was no separate release series for 3.0 and 3.1; we simply went from 3.0.10 to 3.1.0 in the same repositories.
There is one more OSG Series called "upcoming" which contains major updates planned for a future release series.
The yum repositories for upcoming (
osg-upcoming-testing) are available from all OSG 3.x series, and
individual packages can be installed from Upcoming without needing to update entirely to a new series.
Note, however, that packages in the "upcoming" repositories are tested against the most recent OSG series.
As of the time of writing,
osg-upcoming is meant to work with OSG 3.5.
Installing an OSG Release Series¶
See the yum repositories document for instructions on installing the OSG repositories.
Updating to OSG 3.5¶
OS Version Support
OSG 3.5 only supports EL7
If you have an existing installation based on OSG release version <= 3.4 (which will be referred to as the old series), and want to upgrade to 3.5 (the new series), we recommend the following procedure:
First, remove the old series yum repositories:
[email protected] # rpm -e osg-release
This step ensures that any local modifications to
*.repofiles will not prevent installing the new series repos. Any modified
*.repofiles should appear under
*.rpmsaveextension. After installing the new OSG repositories (the next step) you may want to apply any changes made in the
*.rpmsavefiles to the new
Install the OSG repositories:
[email protected] # yum install https://repo.opensciencegrid.org/osg/3.5/osg-3.5-el7-release-latest.rpm
Clean yum cache:
[email protected] # yum clean all --enablerepo=*
[email protected] # yum update
- Please be aware that running
yum updatemay also update other RPMs. You can exclude packages from being updated using the
--exclude=[package-name or glob]option for the
- Watch the yum update carefully for any messages about a
.rpmnewfile being created. That means that a configuration file had been edited, and a new default version was to be installed. In that case, RPM does not overwrite the edited configuration file but instead installs the new version with a
.rpmnewextension. You will need to merge any edits that have made into the
.rpmnewfile and then move the merged version into place (that is, without the
.rpmnewextension). Watch especially for
/etc/lcmaps.db, which every site is expected to edit.
- Please be aware that running
Remove any deprecated packages that were previously installed:
[email protected] # yum remove osg-version \ osg-control \ 'rsv*' \ glite-ce-cream-client-api-c \ glite-lbjp-common-gsoap-plugin \ xacml
If you did not have any of the above packages installed, Yum will not remove any packages:
No Match for argument: osg-version No Match for argument: osg-control No Match for argument: rsv* No Match for argument: glite-ce-cream-client-api-c No Match for argument: glite-lbjp-common-gsoap-plugin No Match for argument: xacml No Packages marked for removal
Running into issues?
If you are not having the expected result or having problems with Yum please see the Yum troubleshooting guide
Updating to HTCondor-CE 4.x¶
The OSG 3.5 release series contains HTCondor-CE 4, a major version upgrade from the previously released versions in the OSG. See the HTCondor-CE 4.0.0 release notes for an overview of the changes. In particular, this version includes a major reorganization of the default configuration so updates will require manual intervention. To update your HTCondor-CE host(s), perform the following steps:
Update all CE packages:
[email protected] # yum update htcondor-ce 'osg-ce*'
The new default
condor_mapfileis sufficient since HTCondor-CE no longer relies on GSI authentication between its daemons. If
/etc/condor-ce/condor_mapfile.rpmnewexists, replace your old
[email protected] # mv /etc/condor-ce/condor_mapfile.rpmnew /etc/condor-ce/condor_mapfile
Additionally, you may wish to make one or more of the following optional changes:
HTCondor-CE now disables batch system job retries by default. To re-enable job retries, set the following configuration in
ENABLE_JOB_RETRIES = True
For non-HTCondor sites that use remote CE requirements, the new version of HTCondor-CE accepts a simplified format. For example, a snippet from an example job route in the old format:
set_default_remote_cerequirements = strcat("Walltime == 3600 && AccountingGroup =="", x509UserProxyFirstFQAN, "\"");
May be rewritten as the following:
set_WallTime = 3600; set_AccountingGroup = x509UserProxyFirstFQAN; set_default_CERequirements = "Walltime,AccountingGroup";
Reload and restart the HTCondor-CE daemons:
Updating to HTCondor 8.8.x¶
The OSG 3.5 release series contains HTCondor 8.8, a major version upgrade from the previously released versions in the OSG. See the detailed update instructions below to update to HTCondor 8.8.
Updating to OSG Configure 3¶
The OSG 3.5 release series contains OSG-Configure 3, a major version upgrade from the previously released versions in the OSG. See the OSG Configure release notes for an overview of the changes. To update OSG Configure on your HTCondor-CE, perform the following steps:
If you haven't already, update to OSG 3.5.
If you have
/etc/osg/config.d/40-siteinfo.ini, delete it and specify
resourceshould match the resource name that's registered in OSG Topology.
/etc/osg/config.d/40-siteinfo.inito the resource group registered in OSG Topology, i.e. the name of the
.yamlfile in OSG Topology that contains the registered resouce above.
host_nameto the host name that is registered in OSG Topology. This may be different from the FQDN of the host if you're using a DNS alias, for example.
OSG Configure will warn about config options that it does not recognize; delete these options from the config to get rid of the warnings.
Updating to HTCondor 8.8.x¶
The OSG 3.5 and 3.4 release series contain HTCondor 8.8, a major version upgrade from the previously released versions in the OSG. See the HTCondor 8.8 manual for an overview of the changes. To update HTCondor on your HTCondor-CE and/or HTCondor pool hosts, perform the following steps:
Update all HTCondor packages:
[email protected] # yum update 'condor*'
HTCondor pools only:
CONDOR_HOSTconfiguration changed in HTCondor 8.8. Additionally in OSG 3.5, the default security was changed to use FS and pool password. If you are experiencing issues with communication between hosts in your pool after the upgrade, the default OSG configuration is listed in
/etc/condor/config.d/00-osg_default_*.config: ensure that any default configuration is overriden with your own
CONDOR_HOST, and/or security configuration in subsequent files.
As of HTCondor 8.8, MOUNT_UNDER_SCRATCH has default values of
/var/tmp, which may cause issues if your OSG_WN_TMP is a subdirectory of either of these directories. If the partition containing your execute directories is large enough, we recommend setting your
/var/tmp. If that partition is not large enough, we recommend setting your
OSG_WN_TMPvariable to a directory outside of
HTCondor-CE hosts only: The HTCondor 8.8 series changed the default job route matching order from round-robin to first matching route. To use the old round-robin matching order, add the following configuration to
JOB_ROUTER_ROUND_ROBIN_SELECTION = True
Clean-up deprecated packages:
[email protected] # yum remove 'rsv*' glite-ce-cream-client-api-c