uscms.org  www 

Using FNAL EOS at the LPC: Additional Commands

This is a secondary set of possibly useful EOS commands. Go to Using EOS at the LPC to find instructions for basic EOS use and operations.

Where do I find more information about EOS commands available?

What are some useful scripts for EOS?

  • Kevin Pedro's github utilities, see eosdu in particular
    • chmod 755 eosdu
    • ./eosdu -h /store/user/username
    • ./eosdu.sh -rh /store/user/lpcgroupname

  • There are some useful scripts in the github.com/FNALLPC/lpc-scripts area
  • These scripts are also copied to cvmfs: /cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts, mounted on all grid sites
    • For each script there should be a help, for instance python /cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts/copyfiles.py --help
    • The xrdcp_recursive.py script can be used to copy multiple files from user filesystems to the cmslpc EOS
    • The recursiveFileList.py script is used by the xrdcpRecursive.py script above
    • The copyfiles.py script can be used to copy multiple files from local AND remote filesystems to the cmslpc EOS

  • From the interactive node command line type alias to find out what commands are aliased for your login
    [username@cmslpc333 WorkDir]$  alias
    eosgrpquota	eos root://cmseos.fnal.gov quota | grep -A4
    eosls	eos root://cmseos.fnal.gov ls
    eosmkdir	eos root://cmseos.fnal.gov mkdir
    eosquota	eos root://cmseos.fnal.gov quota | grep -B4 `whoami`
    eosrm	eos root://cmseos.fnal.gov rm
    xrdfsls	xrdfs root://cmseos.fnal.gov ls    

What are some errors and fixes for EOS?

Errors duplicated from Main EOS page

  • Having an active version of CMSSW environment (for example CMSSW_9_2_4) may cause problems with the commands, giving an error such as:
    [username@cmslpc333 ~]$ eosls /store/user/CERNusername
    eos: symbol lookup error: eos: undefined symbol: _ZN5XrdCl4File4OpenERKSsNS_9OpenFlags5FlagsENS_6Access4ModeEt
    • Solve this error by doing the commands in a new terminal without a CMSSW environment
    • This should not affect xrdcp commands

  • If you changed shell by hand, the eos aliases (eosls, eosquota, etc.) are not loaded.

  • Condor batch jobs at the cmslpc do not have access to the interactive /usr/bin/eos commands, so commands such as xrdcp root://cmseos.fnal.gov or xrdfs root://cmseos.fnal.gov ls should be used within batch jobs

  • CMSSW_7_4_7 gives the following when trying to xrdcp at the end of a condor job to EOS:
    • xrdcp: symbol lookup error: xrdcp: undefined symbol: _ZN14EnvInitializerC1Ev
    • The solution is to move to a newer CMSSW for the xrdcp file transfer step, such as CMSSW_8_0_25 or CMSSW_9_2_3

  • crab checkwrite --site=T3_US_FNALLPC
    • Note that this error at the end is a crab-based error and can be safely ignored, for other errors, follow instructions higher up to make a CMS Storage Space Request
    • Exception AttributeError: AttributeError("'NoneType' object has no attribute 'rmtree'",) in <bound method TempDirectory.__del__ of <WMCore.Services.Requests.TempDirectory object at 0x2b9e16a1b7d0>> ignored

  • If you xrdcp -r from root://cmseos.fnal.gov//store/user/username to root://cmseos.fnal.gov//store/user/username, it will fail with Error indexing remote directory..

How do I use EOS commands at the LPC on the interactive node command line?

Login to cmslpc-el9.fnal.gov, use eos commands with this structure (where command is the eos command you put in yourself. These eos root://cmseos.fnal.gov commands only work on the interactive nodes and not the condor worker nodes.

[username@cmslpc333 ~]$ eos root://cmseos.fnal.gov command

Examples with the EOS find command:

du type command

This provides all files there with respective size. You can pipe this to a grep to do selection.

eos root://cmseos.fnal.gov find --size  

find with xurl

This runs find and gives the xurl path

eos root://cmseos.fnal.gov find --xurl

find with count

This maxes out at 100,000 files. All "user" find commands are limited to 100,000 lines of output. Note that sysadmins don't have this limit, so if more information is required, one can contact LPC Computing support to get the information from the sysadmins.

eos root://cmseos.fnal.gov find --count 

find with childcount

This counts number of a files in a directory, use with a directory -d

eos root://cmseos.fnal.gov find --childcount -d /store/user/username

Given that all "user" find commands are limited to 100,000 lines of output, we use the following example to not count too many subdirectories deep:

eos root://cmseos.fnal.gov find --childcount -d --maxdepth 1 /store/user/username

find by age

This finds files older than 365 days in a directory

eos root://cmseos.fnal.gov find -ctime +365 /store/user/lpcgroupname

For that command it will report all files/directories in an area, but for those that meet the requirement in the -ctime will give the ctime (creation time) of the file, for instance:
eos root://cmseos.fnal.gov find -ctime +1000 /store/user/lpcgroupname
path=/eos/uscms/store/user/lpcgroupname/FilesFrom2015/slimming.C ctime=1422242193.927462722

The first file is from 2015, and the directory that follows (with no ctime or path reported) has a recent modification date.

More EOS copy options:

Note: If you are copying with the intent to move files from one location on FNAL EOS to another, please use EOS mv commands, as mv simply changes the file metadata on the EOS management node and is much faster and less prone to any file transfer errors.

cp help:

eos root://cmseos.fnal.gov cp --help

cp example one file:

eos cp root://cmseos.fnal.gov///store/user/differentuser/miniaod.root root://cmseos.fnal.gov//store/user/tonjes/miniaodCopy.root

cp --recursive:

This will work with the local path, can copy local to remote. If you are copying EOS files from one area on FNAL EOS to another directory on FNAL EOS, please consider doing renaming instead. EOS renaming is faster and better as it only changes the file index on the management node.

eos root://cmseos.fnal.gov cp --recursive

Take note of the directions below to copy a whole directory, the command is formatted differently, and is a solution in case you get the warning: found zero files to copy! error:

cp example directory recursive:

  • Set the EOS_MGM_URL environment variable to root://cmseos.fnal.gov
    • bash: export EOS_MGM_URL=root://cmseos.fnal.gov - note you may wish to put this in your ~/.bash_profile
    • tcsh as default shell: setenv EOS_MGM_URL root://cmseos.fnal.gov - note you may wish to put this in your ~/.tcshrc
  • Use a command like (one of the few places it's ok to use /eos/uscms):
    eos cp -r /eos/uscms/store/user/username/TTbarNtuples/ /eos/uscms/store/group/lpcgroupname/TTbarNtuples/
  • Note that the / at the end of the directory name is essential in formatting the command above
  • Another example, here it copies the pre_exercises directory into the destination, as well any subdirectories that are inside pre_exercises
    eos cp -r /eos/uscms/store/user/cmsdas/2021/pre_exercises/ /eos/uscms/store/user/cmsdas/2022/
  • Example copying from FNAL EOS to a local directory (this DOES NOT work copying local to FNAL EOS, you must use xrdcp for that), note that the directories that are in EOS must have content, empty directories will not be copied:
    eos cp -r /eos/uscms/store/user/tonjes/testing/ ~/nobackup/.

cp xrdcp many files:

To copy many files at the end of a condor batch job for instance, we have: for i in *.root; do xrdcp $i root://cmseos.fnal.gov//store/user/lpcgroupname/MyAnalysis/$i;. However, you can instead use xrdcp recursive, this is from the man page on xrootd.org:

xrdcp -r | --recursive

recursively copy all files starting at the given source directory. This option is only supported for local files.

  • Note: xrdcp -r root://cmseos.fnal.gov//store/user/username/Subdirectory LocalDirectory (to a local file area), may fail for some users with: Error indexing remote directory..
    • This is due to the environment of that user (sometimes CMSSW environment). You can issue the command from a fresh environment in line with the env command, like this:
    • env -i X509_USER_PROXY=${X509_USER_PROXY} xrdcp -r root://cmseos.fnal.gov//store/user/cmsdas/2021/short_exercises/BTag BTag

EOS file renaming and moving:

EOS file rename

If you want to just rename a folder/file without copying inside eos, i.e., change metadata:

eos root://cmseos.fnal.gov file rename /store/user/username/testing_old.root /store/user/username/testing_Name_v2.txt

EOS file move options

eos root://cmseos.fnal.gov mv --help
eos root://cmseos.fnal.gov mv /store/user/tonjes/step4_test.root /store/group/lpcci2dileptons/tonjes_test.root

EOS and xrootd:

Note that these are both xrootd access methods and fine at the cmslpc. Non-xrootd commands should never use /eos/uscms.


FNAL EOS and CERN EOS access at either site:

Here are some examples on how to access CERN EOS files at FNAL:
cernbox/ CERN EOS home area (where u is the first initial of your CERN username from a-z):
[username@cmslpc333 ~]$ eos root://eosuser.cern.ch/ ls /eos/user/u/username 

[username@cmslpc333 ~]$ eos root://eoscms.cern.ch/ ls /store/user 

How to access FNAL EOS files at CERN:
[username@lxplus095 ~]$ eos root://cmseos.fnal.gov ls /store/user 

Checksum files on EOS:

One can do checksum with xrdcp (which copies the file while calculating checksum), as well as xrootd (does not copy the file).

Two examples (note the long command - scroll right):

  1. xrdcp copies the file, here it copies it locally (.), which may not be ideal depending on the file size.
    [username@cmslpc333 CMSSW_9_2_4]$  xrdcp --cksum md5:print root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe .
    md5: 87ee2aedc42926e66d0041645c2ceba4 .//dynlo.lhe 260873891
    Check your local copy like this:
    [tonjes@cmslpc333 ~]$ md5sum dynlo.lhe 
    87ee2aedc42926e66d0041645c2ceba4  dynlo.lhe
  2. xrootd calculates the adler32 checksum:
    [username@cmslpc333 CMSSW_9_2_4]$  xrdfs root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe query checksum /store/user/drberry/dynlo.lhe
    eos 1e053b51
    Check your local copy like this:
    [tonjes@cmslpc333 ~]$  adler32 dynlo.lhe
Webmaster | Last modified: Monday, 06-May-2024 10:30:13 CDT