U.S. CMS
Search
uscms.org  www 

File Transfer

Data Transfer to and from the cmslpc cluster

At Fermilab, access to User Analysis Farm (UAF) goes through cmslpc-sl7.fnal.gov. This can be accessed using Secure Copy (scp) or sftp.

The following storage areas on NFS are available for users, be sure to reference the appropriate directory and use your username, and the first letter (here u) of your username:

/uscms/homes/u/username
/uscms_data/d1/username

To transfer a file to the cmslpc cluster:

Usage: scp file_name username@cmslpc-sl7.fnal.gov:/uscms/homes/u/username

e.g.

Other Computer $ scp zprime705.jdf username@cmslpc-sl7.fnal.gov:/uscms/homes/u/username
zprime705.jdf        100% |*****************************|   286       00:00


To transfer a file from the cmslpc cluster:

Usage: scp username@cmslpc-sl7.fnal.gov:/uscms/homes/u/username/source destination

e.g.

Other Computer $ scp username@cmslpc-sl7.fnal.gov:/uscms/homes/u/username/zprime_jj_10001.root .
zprime_jj_10001.root 100% |*****************************|  8880 KB    00:01


Copy files on the EOS T3_US_FNALLPC SE

For instructions how to copy files from/to the EOS Storage Element, follow the instructions on the Using EOS at the LPC website.

Copy files using gfal-copy from another site's Storage Element

Instructions are on the CRAB3 FAQ on how to use gfal tools to find and copy files from another site's Storage Element

  • Note: if you have errors with gfal-copy, you may wish to do the gfal-copy from a clean environment. You can do that on the command line by starting with this: env -i X509_USER_PROXY=${X509_USER_PROXY}. This also works for xrdcp. Examples are shown for gsiftp and now WebDAV (production default transfer method). If you need to create the target directory in webdav, use the -p option. Examples and alternate copy methods below:
    • env -i X509_USER_PROXY=${X509_USER_PROXY} gfal-copy -pf filename gsiftp://cmseos-gridftp.fnal.gov//eos/uscms/store/user/filename
    • env -i X509_USER_PROXY=${X509_USER_PROXY} gfal-copy -pf filename davs://cmseos.fnal.gov//eos/uscms/store/user/username/newdirectory/filename
  • Alternately: The copyfiles.py script can be used to copy single files or a directory of files using gfal-copy or xrdcp from another site to T3_US_FNALLPC
  • Note: The GetSiteInfo.py script can be useful to get the information of the site's endpoint to obtain a single file through gfal-copy, it is used by the copyfiles.py script above

File access with Xrootd

The WorkBookXrootdService twiki describes how to access files using Xrootd
Webmaster | Last modified: Wednesday, 27-Jul-2022 11:29:10 CDT