Installing the StashCache Origin¶
This document describes how to install a StashCache origin service. This service allows an organization to export its data to the StashCache data federation.
The origin must be registered with the OSG prior to joining the data federation. You may start the registration process prior to finishing the installation by using this link along with the basic information like:
- Resource name and hostname.
- VO associated with this origin server (will be used to determine the origin's namespace prefix).
- Administrative and security contact.
This guide covers the procedure for exporting world-readable data; publishing proprietary data is still experimental.
Before starting the installation process, consider the following points:
- Operating system: A RHEL 7 or compatible operating systems is strongly recommended. If you have special needs that require you to run your origin server on a RHEL 6-based host, contact us at [email protected]
- User IDs: If they do not exist already, the installation will create the Linux user IDs
- Host certificate: The StashCache server uses a host certificate to advertise to a central collector. The host certificate documentation provides more information on setting up host certificates.
- Network ports: The StashCache Origin service requires the following ports open:
- Inbound TCP port 1094 for file access via the XRootD protocol
- Outbound TCP port 1213 to
redirector.osgstorage.orgfor connecting to the data federation
- Outbound UDP port 9619 for reporting to
- Hardware requirements: We recommend that a StashCache origin has at least 1Gbps connectivity and 8GB of RAM. We suggest that several gigabytes of local disk space be available for log files.
As with all OSG software installations, there are some one-time steps to prepare in advance:
- Ensure the host has a supported operating system. A RHEL 7-based operating system is strongly recommended.
- Obtain root access to the host
- Prepare the required Yum repositories
- Install CA certificates
Installing the StashCache origin¶
The StashCache daemon consists of an XRootD server and an HTCondor-based service for collecting and reporting monitoring data about the origin. To simplify installation, OSG provides convenience RPMs that install all required software with a single command:
[email protected] # yum install stashcache-origin-server
For this installation guide, we assume that the data to be exported to the federation is mounted at
and owned by the
Configuring the Origin Server¶
stashcache-daemon package provides a default configuration file,
/etc/xrootd/xrootd-stashcache-origin-server.cfg, which must be customized for your origin.
The most common lines to customize are:
oss.localroot /stash: Change the
localrootto the location where your data is mounted on the origin server; default is
all.export /<YOUR VO>: A sub-directory within the
localrootdirectory that will be exported. If multiple directories must be exported, you may specify
For example, if the HCC VO would like to set up an origin server exporting from the mountpoint
but only export the subdirectories
they would use the following configuration:
oss.localroot /mnt/bigdata all.export /hcc/bio/datasets all.export /hcc/hep/generators
With this configuration, the data under
/mnt/bigdata/hcc/bio/datasets would be available under the StashCache path
/hcc/bio/datasets, the data under
/mnt/bigdata/hcc/hep/generators would be available under the StashCache path
/hcc/hep/generators, and no other data would be available via StashCache.
The StashCache namespace is global within a data federation. Directories you export must not collide with directories provided by other origin servers.
The best way to do this is to create a directory named after your VO or project, place all files you want to distribute within that directory, and export only that directory or its subdirectories.
Managing the Origin Service¶
The origin service consists of the following systemd units:
||The xrootd daemon, which performs the data transfers|
||The "cluster management service" daemon, which integrates the origin into the data federation.|
||See CA documentation for more info|
These services must be managed with
systemctl. As a reminder, here are common service commands (all run as
|To...||On EL7, run the command...|
|Start a service||
|Stop a service||
|Enable a service to start on boot||
|Disable a service from starting on boot||
Verifying the Origin Server¶
Once your server has been registered with the OSG and started, perform the following steps to verify that it is functional.
To verify that your origin is correctly advertising its availability, run the following command from the origin server:
[[email protected] ~]$ xrdmapc -r --list s redirector.osgstorage.org:1094 0**** redirector.osgstorage.org:1094 Srv ceph-gridftp1.grid.uchicago.edu:1094 Srv stashcache.fnal.gov:1094 Srv stash.osgconnect.net:1094 Srv origin.ligo.caltech.edu:1094 Srv csiu.grid.iu.edu:1094
The output should list the hostname of your origin server.
Testing directory export¶
To verify that the directories you are exporting are visible from the redirector, run the following command from the origin server:
[[email protected] ~]$ xrdmapc -r --verify --list s redirector.osgstorage.org:1094 <exported dir> 0*rv* redirector.osgstorage.org:1094 >+ Srv ceph-gridftp1.grid.uchicago.edu:1094 ? Srv stashcache.fnal.gov:1094 [not authorized] >+ Srv stash.osgconnect.net:1094 - Srv origin.ligo.caltech.edu:1094 ? Srv csiu.grid.iu.edu:1094 [connect error]
Your server should be marked with a
>+ to indicate that it contains the given path and the path was accessible.
Testing file access¶
To verify that you can download a file from the origin server, use the
Place a test file in the exported dir.
stashcp is available in the
Run the following command:
[[email protected]]$ stashcp <test file> /tmp/testfile
If successful, there should be a file at
/tmp/testfile with the contents of the test file on your origin server.
If unsuccessful, you can pass the
-d flag to
stashcp for debug info.
Registering the Origin¶
To be part of the OSG StashCache Federation, your origin must be
registered with the OSG. The service type is
XRootD origin server.
The resource must also specify which VOs it will serve data from.
To do this, add an
AllowedVOs list, with each line specifying a VO whose StashCache data the resource is willing to host.
MY_STASHCACHE_ORIGIN: Service: XRootD origin server Description: StashCache origin server AllowedVOs: - GLOW - OSG
You can use the special value
ANY to indicate that the origin will serve data from any VO that puts data on it.
In addition to the origin allowing a VOs via the
that VO must also allow the origin in its
See the page on getting your VO's data into StashCache.