justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 21828.2@dunegpschedd02.fnal.gov

Jobsub ID21828.2@dunegpschedd02.fnal.gov
Workflow ID646
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-08-19 10:15:48
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_dissel
Last heartbeat2025-08-19 10:25:50
From worker nodeHostnamewn-lot-036.farm.nikhef.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-08-19 10:16:26
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_466_20231208T041157Z_gen_g4_detsim_hitreco__20240510T050636Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_775_20231207T000741Z_gen_g4_detsim_hitreco__20240510T032038Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_839_20231202T172615Z_gen_g4_detsim_hitreco__20240507T210509Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_183_20231122T082855Z_gen_g4_detsim_hitreco__20240503T064932Z_reco2.root
JobscriptExit code0
Real time9m (549s)
CPU time1m (113s = 20%)
Max RSS bytes1201852416 (1146 MiB)
Outputting started2025-08-19 10:25:35
Output files
Finished2025-08-19 10:25:50
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Using CAF fcl: cafmaker_atmos_dune10kt_1x2x6_runreco-nuenergy-nuangular_geov5.fcl
Using custom sources from /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/localProducts_larsoft__e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/localProducts_larsoft__e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/localProducts_larsoft__e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/localProducts_larsoft__e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Justin processors: 1
Will use justin-get-file
No files available from this stage
Could not get file
No files available from this stage
Could not get file
No files available from this stage
Could not get file
No files available from this stage
Could not get file
No files available from this stage
Could not get file
No files available from this stage
Could not get file
now run lar on these files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/dd/99/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_466_20231208T041157Z_gen_g4_detsim_hitreco__20240510T050636Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/06/ec/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_775_20231207T000741Z_gen_g4_detsim_hitreco__20240510T032038Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bb/7d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_839_20231202T172615Z_gen_g4_detsim_hitreco__20240507T210509Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c2/41/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_183_20231122T082855Z_gen_g4_detsim_hitreco__20240503T064932Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_466_20231208T041157Z_gen_g4_detsim_hitreco__20240510T050636Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_775_20231207T000741Z_gen_g4_detsim_hitreco__20240510T032038Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_839_20231202T172615Z_gen_g4_detsim_hitreco__20240507T210509Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_183_20231122T082855Z_gen_g4_detsim_hitreco__20240503T064932Z_reco2.root
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/77bde15fa509d0f93131802e0d65821efa1ff0ff/cafmaker_tests/localProducts_larsoft__e26_prof/dunecore/v10_08_02d00/gdml/dune10kt_v6_refactored_1x2x6.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1149
Info in <TGeoManager::CloseGeometry>: 67545 nodes/ 5439 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
lar exit code 0
=== Start last 100 lines of lar log file ===
tail: cannot open '_reco_20250819T101634Z.log' for reading: No such file or directory
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/dd/99/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74576447_466_20231208T041157Z_gen_g4_detsim_hitreco__20240510T050636Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/06/ec/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_775_20231207T000741Z_gen_g4_detsim_hitreco__20240510T032038Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bb/7d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_839_20231202T172615Z_gen_g4_detsim_hitreco__20240507T210509Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c2/41/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6220954_183_20231122T082855Z_gen_g4_detsim_hitreco__20240503T064932Z_reco2.root
.:
total 21348
-rw-r--r--. 1 dune012 dune 6882392 Aug 19 12:25 flatcaf.root
-rw-r--r--. 1 dune012 dune 5656679 Aug 19 12:25 caf.root
-rw-r--r--. 1 dune012 dune 5656679 Aug 19 12:25 caf_fd_hd_atmo_646_20250819T101634Z.root
-rw-r--r--. 1 dune012 dune 3606487 Aug 19 12:25 caf_20250819T101634Z.log
-rw-r--r--. 1 dune012 dune    5655 Aug 19 12:25 jobscript.log
-rw-r--r--. 1 dune012 dune     760 Aug 19 12:25 caf_20250819T101634Z.file
-rw-r--r--. 1 dune012 dune     760 Aug 19 12:25 caf_20250819T101634Z.pfns
-rw-r--r--. 1 dune012 dune     760 Aug 19 12:16 file.list
-rw-r--r--. 1 dune012 dune     760 Aug 19 12:25 justin-processed-pfns.txt
-rw-r--r--. 1 dune012 dune     550 Aug 19 12:16 all-input-dids.txt
-rw-r--r--. 1 dune012 dune     550 Aug 19 12:25 caf_20250819T101634Z.did
-rw-r--r--. 1 dune012 dune     550 Aug 19 12:16 did.list
justIN time: 2025-11-05 09:43:18 UTC       justIN version: 01.05.01