Using FNAL EOS at the LPC: Additional Commands

This is a secondary set of possibly useful EOS commands. Go to Using EOS at the LPC to find instructions for basic EOS use and operations.

Where do I find more information about EOS commands available?

Using EOS at the LPC

Keep in mind all the things you things you should NOT do on the CMS LPC EOS
From the interactive node command line: eos root://cmseos.fnal.gov command --help, where command is the command such as ls

EOS at CERN knowledgebase, be aware that CERN may be running a more recent version of EOS than the CMS LPC

What are some useful scripts for EOS?

Kevin Pedro's github utilities, see eosdu in particular

chmod 755 eosdu
./eosdu -h /store/user/username
./eosdu.sh -rh /store/user/lpcgroupname

There are some useful scripts in the github.com/FNALLPC/lpc-scripts area
These scripts are also copied to cvmfs: /cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts, mounted on all grid sites

For each script there should be a help, for instance python /cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts/copyfiles.py --help
The xrdcp_recursive.py script can be used to copy multiple files from user filesystems to the cmslpc EOS
The recursiveFileList.py script is used by the xrdcpRecursive.py script above
The copyfiles.py script can be used to copy multiple files from local AND remote filesystems to the cmslpc EOS

From the interactive node command line type alias to find out what commands are aliased for your login

[username@cmslpc333 WorkDir]$  alias
eosgrpquota	eos root://cmseos.fnal.gov quota | grep -A4
eosls	eos root://cmseos.fnal.gov ls
eosmkdir	eos root://cmseos.fnal.gov mkdir
eosquota	eos root://cmseos.fnal.gov quota | grep -B4 `whoami`
eosrm	eos root://cmseos.fnal.gov rm
xrdfsls	xrdfs root://cmseos.fnal.gov ls

What are some errors and fixes for EOS?

Errors duplicated from Main EOS page

Having an active version of CMSSW environment (for example CMSSW_9_2_4) may cause problems with the commands, giving an error such as:
```
[username@cmslpc333 ~]$ eosls /store/user/CERNusername
eos: symbol lookup error: eos: undefined symbol: _ZN5XrdCl4File4OpenERKSsNS_9OpenFlags5FlagsENS_6Access4ModeEt
```
- Solve this error by doing the commands in a new terminal without a CMSSW environment
- This should not affect xrdcp commands
If you changed shell by hand, the eos aliases (eosls, eosquota, etc.) are not loaded.

Do the commands from a new login shell or request a new default login shell.

Condor batch jobs at the cmslpc do not have access to the interactive /usr/bin/eos commands, so commands such as xrdcp root://cmseos.fnal.gov or xrdfs root://cmseos.fnal.gov ls should be used within batch jobs

CMSSW_7_4_7 gives the following when trying to xrdcp at the end of a condor job to EOS:
- xrdcp: symbol lookup error: xrdcp: undefined symbol: _ZN14EnvInitializerC1Ev
- The solution is to move to a newer CMSSW for the xrdcp file transfer step, such as CMSSW_8_0_25 or CMSSW_9_2_3
crab checkwrite --site=T3_US_FNALLPC

Note that this error at the end is a crab-based error and can be safely ignored, for other errors, follow instructions higher up to make a CMS Storage Space Request


Exception AttributeError: AttributeError("'NoneType' object has no attribute 'rmtree'",) in <bound method TempDirectory.__del__ of <WMCore.Services.Requests.TempDirectory object at 0x2b9e16a1b7d0>> ignored

If you xrdcp -r from root://cmseos.fnal.gov//store/user/username to root://cmseos.fnal.gov//store/user/username, it will fail with Error indexing remote directory..

This is because xrdcp -r only works from local files. Instead, use either the xrdcpRecursive.py script, the copyfiles.py, or use eos cp -pr root://cmseos.fnal.gov type commands.

How do I use EOS commands at the LPC on the interactive node command line?

Login to cmslpc-el9.fnal.gov, use eos commands with this structure (where command is the eos command you put in yourself. These eos root://cmseos.fnal.gov commands only work on the interactive nodes and not the condor worker nodes.

[username@cmslpc333 ~]$ eos root://cmseos.fnal.gov command

Examples with the EOS find command:

du type command

This provides all files there with respective size. You can pipe this to a grep to do selection.


eos root://cmseos.fnal.gov find --size

find with xurl

This runs find and gives the xurl path


eos root://cmseos.fnal.gov find --xurl

find with count

This maxes out at 100,000 files. All "user" find commands are limited to 100,000 lines of output. Note that sysadmins don't have this limit, so if more information is required, one can contact LPC Computing support to get the information from the sysadmins.


eos root://cmseos.fnal.gov find --count

find with childcount

This counts number of a files in a directory, use with a directory -d


eos root://cmseos.fnal.gov find --childcount -d /store/user/username

Given that all "user" find commands are limited to 100,000 lines of output, we use the following example to not count too many subdirectories deep:


eos root://cmseos.fnal.gov find --childcount -d --maxdepth 1 /store/user/username

find by age

This finds files older than 365 days in a directory


eos root://cmseos.fnal.gov find -ctime +365 /store/user/lpcgroupname

For that command it will report all files/directories in an area, but for those that meet the requirement in the -ctime will give the ctime (creation time) of the file, for instance:

eos root://cmseos.fnal.gov find -ctime +1000 /store/user/lpcgroupname
path=/eos/uscms/store/user/lpcgroupname/FilesFrom2015/slimming.C ctime=1422242193.927462722
/eos/uscms/store/user/lpcgroupname/FilesFrom2018/

The first file is from 2015, and the directory that follows (with no ctime or path reported) has a recent modification date.

More EOS copy options:

Note: If you are copying with the intent to move files from one location on FNAL EOS to another, please use EOS mv commands, as mv simply changes the file metadata on the EOS management node and is much faster and less prone to any file transfer errors.

cp help:


eos root://cmseos.fnal.gov cp --help

cp example one file:


eos cp root://cmseos.fnal.gov///store/user/differentuser/miniaod.root root://cmseos.fnal.gov//store/user/tonjes/miniaodCopy.root

cp --recursive:

This will work with the local path, can copy local to remote. If you are copying EOS files from one area on FNAL EOS to another directory on FNAL EOS, please consider doing renaming instead. EOS renaming is faster and better as it only changes the file index on the management node.


eos root://cmseos.fnal.gov cp --recursive

Take note of the directions below to copy a whole directory, the command is formatted differently, and is a solution in case you get the warning: found zero files to copy! error:

cp example directory recursive:

Set the EOS_MGM_URL environment variable to root://cmseos.fnal.gov
- bash: export EOS_MGM_URL=root://cmseos.fnal.gov - note you may wish to put this in your ~/.bash_profile
- tcsh as default shell: setenv EOS_MGM_URL root://cmseos.fnal.gov - note you may wish to put this in your ~/.tcshrc

Use a command like (one of the few places it's ok to use /eos/uscms):

eos cp -r /eos/uscms/store/user/username/TTbarNtuples/ /eos/uscms/store/group/lpcgroupname/TTbarNtuples/

Note that the / at the end of the directory name is essential in formatting the command above
Another example, here it copies the pre_exercises directory into the destination, as well any subdirectories that are inside pre_exercises
```
eos cp -r /eos/uscms/store/user/cmsdas/2021/pre_exercises/ /eos/uscms/store/user/cmsdas/2022/
```
Example copying from FNAL EOS to a local directory (this DOES NOT work copying local to FNAL EOS, you must use xrdcp for that), note that the directories that are in EOS must have content, empty directories will not be copied:
```
eos cp -r /eos/uscms/store/user/tonjes/testing/ ~/nobackup/.
```

cp xrdcp many files:

To copy many files at the end of a condor batch job for instance, we have: for i in *.root; do xrdcp $i root://cmseos.fnal.gov//store/user/lpcgroupname/MyAnalysis/$i;. However, you can instead use xrdcp recursive, this is from the man page on xrootd.org:


xrdcp -r | --recursive

recursively copy all files starting at the given source directory. This option is only supported for local files.

Note: xrdcp -r root://cmseos.fnal.gov//store/user/username/Subdirectory LocalDirectory (to a local file area), may fail for some users with: Error indexing remote directory..

This is due to the environment of that user (sometimes CMSSW environment). You can issue the command from a fresh environment in line with the env command, like this:
env -i X509_USER_PROXY=${X509_USER_PROXY} xrdcp -r root://cmseos.fnal.gov//store/user/cmsdas/2021/short_exercises/BTag BTag

EOS file renaming and moving:

EOS file rename

If you want to just rename a folder/file without copying inside eos, i.e., change metadata:


eos root://cmseos.fnal.gov file rename /store/user/username/testing_old.root /store/user/username/testing_Name_v2.txt

EOS file move options


eos root://cmseos.fnal.gov mv --help
eos root://cmseos.fnal.gov mv /store/user/tonjes/step4_test.root /store/group/lpcci2dileptons/tonjes_test.root

EOS and xrootd:

Note that these are both xrootd access methods and fine at the cmslpc. Non-xrootd commands should never use /eos/uscms.


root://cmseos.fnal.gov//eos/uscms/store/user
root://cmseos.fnal.gov//store/user

FNAL EOS and CERN EOS access at either site:

Here are some examples on how to access CERN EOS files at FNAL:
cernbox/ CERN EOS home area (where u is the first initial of your CERN username from a-z):

 
[username@cmslpc333 ~]$ eos root://eosuser.cern.ch/ ls /eos/user/u/username

CMS CERN EOS:

 
[username@cmslpc333 ~]$ eos root://eoscms.cern.ch/ ls /store/user

How to access FNAL EOS files at CERN:

 
[username@lxplus095 ~]$ eos root://cmseos.fnal.gov ls /store/user

Checksum files on EOS:

One can do checksum with xrdcp (which copies the file while calculating checksum), as well as xrootd (does not copy the file).

Two examples (note the long command - scroll right):

xrdcp copies the file, here it copies it locally (.), which may not be ideal depending on the file size.

 
[username@cmslpc333 CMSSW_9_2_4]$  xrdcp --cksum md5:print root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe .
[248.8MB/248.8MB][100%][==================================================][41.46MB/s]  
md5: 87ee2aedc42926e66d0041645c2ceba4 .//dynlo.lhe 260873891

Check your local copy like this:

 
[tonjes@cmslpc333 ~]$ md5sum dynlo.lhe 

87ee2aedc42926e66d0041645c2ceba4  dynlo.lhe

xrootd calculates the adler32 checksum:

 
[username@cmslpc333 CMSSW_9_2_4]$  xrdfs root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe query checksum /store/user/drberry/dynlo.lhe
eos 1e053b51