Using FNAL EOS at the LPC: Additional Commands
This is a secondary set of possibly useful EOS commands. Go to Using EOS at the LPC to find instructions for basic EOS use and operations.
Where do I find more information about EOS commands available?
- Using EOS at the LPC
- Keep in mind all the things you things you should NOT do on the CMS LPC EOS
- From the interactive node command line:
eos root://cmseos.fnal.gov command --help, where
commandis the command such as
- EOS at CERN knowledgebase, be aware that CERN may be running a more recent version of EOS than the CMS LPC
What are some useful scripts for EOS?
- Kevin Pedro's github utilities, see eosdu in particular
chmod 755 eosdu
./eosdu -h /store/user/username
./eosdu.sh -rh /store/user/lpcgroupname
- There are some useful scripts in the github.com/FNALLPC/lpc-scripts area
- These scripts are also copied to cvmfs:
/cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts, mounted on all grid sites
- For each script there should be a help, for instance
python /cvmfs/cms-lpc.opensciencegrid.org/FNALLPC/lpc-scripts/copyfiles.py --help
- The xrdcpRecursive.py script can be used to copy multiple files from user filesystems to the cmslpc EOS
- The recursiveFileList.py script is used by the xrdcpRecursive.py script above
- The copyfiles.py script can be used to copy multiple files from local AND remote filesystems to the cmslpc EOS
- From the interactive node command line type
aliasto find out what commands are aliased for your login
[username@cmslpc139 WorkDir]$ alias eosgrpquota eos root://cmseos.fnal.gov quota | grep -A4 eosls eos root://cmseos.fnal.gov ls eosmkdir eos root://cmseos.fnal.gov mkdir eosquota eos root://cmseos.fnal.gov quota | grep -B4 `whoami` eosrm eos root://cmseos.fnal.gov rm xrdfsls xrdfs root://cmseos.fnal.gov ls
What are some errors and fixes for EOS?Errors duplicated from Main EOS page
- Having an active version of CMSSW environment (for example CMSSW_9_2_4) may cause problems with the commands, giving an error such as:
[username@cmslpc125 ~]$ eosls /store/user/CERNusername eos: symbol lookup error: eos: undefined symbol: _ZN5XrdCl4File4OpenERKSsNS_9OpenFlags5FlagsENS_6Access4ModeEt
- Solve this error by doing the commands in a new terminal without a CMSSW environment
- This should not affect
- If you changed shell by hand, the eos aliases (
eosquota, etc.) are not loaded.
- Do the commands from a new login shell or request a new default login shell.
- Condor batch jobs at the cmslpc do not have access to the interactive
/usr/bin/eoscommands, so commands such as
xrdfs root://cmseos.fnal.gov lsshould be used within batch jobs
- CMSSW_7_4_7 gives the following when trying to
xrdcpat the end of a condor job to EOS:
xrdcp: symbol lookup error: xrdcp: undefined symbol: _ZN14EnvInitializerC1Ev
- The solution is to move to a newer CMSSW for the
xrdcpfile transfer step, such as CMSSW_8_0_25 or CMSSW_9_2_3
crab checkwrite --site=T3_US_FNALLPC
- Note that this error at the end is a crab-based error and can be safely ignored, for other errors, follow instructions higher up to make a CMS Storage Space Request
- If you
root://cmseos.fnal.gov//store/user/username, it will fail with
Error indexing remote directory..
Exception AttributeError: AttributeError("'NoneType' object has no attribute 'rmtree'",) in <bound method TempDirectory.__del__ of <WMCore.Services.Requests.TempDirectory object at 0x2b9e16a1b7d0>> ignored
How do I use EOS commands at the LPC on the interactive node command line?
cmslpc-sl7.fnal.gov, use eos commands with this structure (where
command is the eos command you put in yourself. These
eos root://cmseos.fnal.gov commands only work on the interactive nodes and not the condor worker nodes.
[username@cmslpc137 ~]$ eos root://cmseos.fnal.gov command
Examples with the EOS find command:
du type commandThis provides all files there with respective size. You can pipe this to a grep to do selection.
eos root://cmseos.fnal.gov find --size
find with xurlThis runs find and gives the xurl path
eos root://cmseos.fnal.gov find --xurl
find with countThis maxes out at 100,000 files. All "user" find commands are limited to 100,000 lines of output. Note that sysadmins don't have this limit, so if more information is required, one can contact LPC Computing support to get the information from the sysadmins.
eos root://cmseos.fnal.gov find --count
find with childcountThis counts number of a files in a directory, use with a directory -d
eos root://cmseos.fnal.gov find --childcount -d /store/user/usernameGiven that all "user" find commands are limited to 100,000 lines of output, we use the following example to not count too many subdirectories deep:
eos root://cmseos.fnal.gov find --childcount -d --maxdepth 1 /store/user/username
find by ageThis finds files older than 365 days in a directory
eos root://cmseos.fnal.gov find -ctime +365 /store/user/lpcgroupnameFor that command it will report all files/directories in an area, but for those that meet the requirement in the -ctime will give the ctime (creation time) of the file, for instance:
eos root://cmseos.fnal.gov find -ctime +1000 /store/user/lpcgroupname path=/eos/uscms/store/user/lpcgroupname/FilesFrom2015/slimming.C ctime=1422242193.927462722 /eos/uscms/store/user/lpcgroupname/FilesFrom2018/The first file is from 2015, and the directory that follows (with no ctime or path reported) has a recent modification date.
More EOS copy options:Note: If you are copying with the intent to move files from one location on FNAL EOS to another, please use EOS mv commands, as
mvsimply changes the file metadata on the EOS management node and is much faster and less prone to any file transfer errors.
eos root://cmseos.fnal.gov cp --help
cp example one file:
eos cp root://cmseos.fnal.gov///store/user/differentuser/miniaod.root root://cmseos.fnal.gov//store/user/tonjes/miniaodCopy.root
cp --recursive:This will work with the local path, can copy local to remote. If you are copying EOS files from one area on FNAL EOS to another directory on FNAL EOS, please consider doing renaming instead. EOS renaming is faster and better as it only changes the file index on the management node.
eos root://cmseos.fnal.gov cp --recursiveTake note of the directions below to copy a whole directory, the command is formatted differently, and is a solution in case you get the
warning: found zero files to copy!error:
cp example directory recursive:
- Set the
EOS_MGM_URLenvironment variable to
export EOS_MGM_URL=root://cmseos.fnal.gov- note you may wish to put this in your
- tcsh as default shell:
setenv EOS_MGM_URL root://cmseos.fnal.gov- note you may wish to put this in your
- Use a command like (one of the few places it's ok to use
eos cp -r /eos/uscms/store/user/username/TTbarNtuples/ /eos/uscms/store/group/lpcgroupname/TTbarNtuples/
- Note that the
/at the end of the directory name is essential in formatting the command above
cp xrdcp many files:To copy many files at the end of a condor batch job for instance, we have:
for i in *.root; do xrdcp $i root://cmeos.fnal.gov//store/user/lpcgroupname/MyAnalysis/$i;. However, you can instead use xrdcp recursive, this is from the man page on xrootd.org:
xrdcp -r | --recursiverecursively copy all files starting at the given source directory. This option is only supported for local files.
EOS file renaming and moving:
EOS file renameIf you want to just rename a folder/file without copying inside eos, i.e., change metadata:
eos root://cmseos.fnal.gov file rename /store/user/username/testing_old.root /store/user/username/testing_Name_v2.txt
EOS file move options
eos root://cmseos.fnal.gov mv --help eos root://cmseos.fnal.gov mv /store/user/tonjes/step4_test.root /store/group/lpcci2dileptons/tonjes_test.root
EOS and xrootd:Note that these are both xrootd access methods and fine at the cmslpc. Non-xrootd commands should never use /eos/uscms.
FNAL EOS and CERN EOS access at either site:Here are some examples on how to access CERN EOS files at FNAL:
[username@cmslpc127 ~]$ eos root://eoscms.cern.ch/ ls /store/user
How to access FNAL EOS files at CERN:
[username@lxplus095 ~]$ eos root://cmseos.fnal.gov ls /store/user
Checksum files on EOS:One can do checksum with
xrdcp(which copies the file while calculating checksum), as well as
xrootd(does not copy the file).
Two examples (note the long command - scroll right):
xrdcpcopies the file, here it copies it locally (
.), which may not be ideal depending on the file size.
[username@cmslpc127 CMSSW_9_2_4]$ xrdcp --cksum md5:print root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe . [248.8MB/248.8MB][100%][==================================================][41.46MB/s] md5: 87ee2aedc42926e66d0041645c2ceba4 .//dynlo.lhe 260873891Check your local copy like this:
[tonjes@cmslpc127 ~]$ md5sum dynlo.lhe 87ee2aedc42926e66d0041645c2ceba4 dynlo.lhe
xrootdcalculates the adler32 checksum:
[username@cmslpc127 CMSSW_9_2_4]$ xrdfs root://cmseos.fnal.gov//store/user/drberry/dynlo.lhe query checksum /store/user/drberry/dynlo.lhe eos 1e053b51Check your local copy like this:
[tonjes@cmslpc127 ~]$ adler32 dynlo.lhe 1e053b51