Jobsub ID 46706.31@dunegpschedd01.fnal.gov
Jobscript log (last 10,000 characters)
AngularReco 0.000190977 0.00203831 0.168621 0.000576293 0.00732677 800
reco:anglereconumu:NuAngularReco 6.621e-05 0.00164306 0.142343 0.000269471 0.00656752 800
reco:anglereconuepfps:NuAngularReco 0.000113781 0.00168557 0.0828083 0.000413397 0.0051108 800
reco:anglereconumupfps:NuAngularReco 8.6049e-05 0.00161556 0.0665662 0.000337924 0.00497473 800
reco:anglerecohits:NuAngularReco 0.000102955 0.00244237 0.0786454 0.00091728 0.00594364 800
[art]:TriggerResults:TriggerResultInserter 9.975e-06 1.82595e-05 0.000178569 1.6079e-05 9.80912e-06 800
end_path:cafmaker:CAFMaker 0.00134807 0.025212 1.26894 0.00708201 0.0745067 800
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3997.89 MB
Peak resident set size usage (VmHWM): 2269.51 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 15-Sep-2025 17:11:57 UTC ModuleEndJob
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Socket timeout
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/93/46/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_708_20231203T082050Z_gen_g4_detsim_hitreco__20240508T060539Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
lar exit code 1
=== Start last 100 lines of lar log file ===
minwire 0: 2331
minwire 1: 116
minwire 2: 2005
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.00994208, 0.823217, 0.166841,
Output 1: 0.451522, 0.458345, 0.0644854, 0.0256482,
Output 2: 0.166369, 0.50395, 0.23622, 0.0934605,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2150!
Begin processing the 792nd record. run: 50602933 subRun: 1 event: 74692 at 15-Sep-2025 17:04:22 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 805!
Begin processing the 793rd record. run: 50602933 subRun: 1 event: 74693 at 15-Sep-2025 17:04:24 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 278!
Begin processing the 794th record. run: 50602933 subRun: 1 event: 74694 at 15-Sep-2025 17:04:25 UTC
Boundary wire vector sizes: 225, 219, 163
minwire 0: 1965
minwire 1: 1185
minwire 2: 1748
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.215319, 0.380909, 0.403773,
Output 1: 0.0254876, 0.937058, 0.0361669, 0.00128736,
Output 2: 0.279653, 0.698653, 0.0214462, 0.000247337,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2468!
Begin processing the 795th record. run: 50602933 subRun: 1 event: 74695 at 15-Sep-2025 17:04:28 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 725!
Begin processing the 796th record. run: 50602933 subRun: 1 event: 74696 at 15-Sep-2025 17:04:30 UTC
Boundary wire vector sizes: 351, 331, 309
minwire 0: 2302
minwire 1: 129
minwire 2: 2470
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.00398855, 0.944633, 0.051378,
Output 1: 0.016536, 0.946909, 0.0356737, 0.000881159,
Output 2: 0.986332, 0.013451, 0.000212658, 4.42454e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1058!
Begin processing the 797th record. run: 50602933 subRun: 1 event: 74697 at 15-Sep-2025 17:04:33 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 745!
Begin processing the 798th record. run: 50602933 subRun: 1 event: 74698 at 15-Sep-2025 17:04:35 UTC
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 61!
Begin processing the 799th record. run: 50602933 subRun: 1 event: 74699 at 15-Sep-2025 17:04:35 UTC
Boundary wire vector sizes: 59, 63, 49
minwire 0: 1338
minwire 1: 448
minwire 2: 1466
Used alternate method to get min and max wires due to vertex determination failure: 625, 924
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 728, 1027
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0355644, 0.0125754, 0.95186,
Output 1: 0.692752, 0.276666, 0.0285136, 0.0020682,
Output 2: 0.986664, 0.0129437, 0.00037084, 2.15776e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1108!
Begin processing the 800th record. run: 50602933 subRun: 1 event: 74700 at 15-Sep-2025 17:04:37 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 685!
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
processed files
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/c0/d9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_401_20231122T124933Z_gen_g4_detsim_hitreco__20240507T181914Z_reco2.root
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/89/93/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_395_20231204T065540Z_gen_g4_detsim_hitreco__20240509T214943Z_reco2.root
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/8f/55/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50546594_562_20231202T075816Z_gen_g4_detsim_hitreco__20240507T195516Z_reco2.root
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/49/87/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_310_20231202T102617Z_gen_g4_detsim_hitreco__20240507T210752Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/8d/d1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419113_195_20231203T143909Z_gen_g4_detsim_hitreco__20240508T052700Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/58/a6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50609765_766_20231207T075258Z_gen_g4_detsim_hitreco__20240510T024406Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d1/53/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50602933_746_20231205T110936Z_gen_g4_detsim_hitreco__20240510T042321Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/93/46/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_708_20231203T082050Z_gen_g4_detsim_hitreco__20240508T060539Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/18/66/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_232_20231202T180405Z_gen_g4_detsim_hitreco__20240508T063333Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/59/cc/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50604274_488_20231207T025259Z_gen_g4_detsim_hitreco__20240510T050752Z_reco2.root
.:
total 37936
-rw-r--r-- 1 dune001 dune 14267542 Sep 15 17:11 flatcaf.root
-rw-r--r-- 1 dune001 dune 11932737 Sep 15 17:11 caf.root
-rw-r--r-- 1 dune001 dune 11932737 Sep 15 17:11 caf_fd_hd_atmo_2501_20250915T154535Z.root
-rw-r--r-- 1 dune001 dune 476803 Sep 15 17:11 caf_20250915T154535Z.log
-rw-r--r-- 1 dune001 dune 180542 Sep 15 17:11 jobscript.log
-rw-r--r-- 1 dune001 dune 1990 Sep 15 17:11 caf_20250915T154535Z.file
-rw-r--r-- 1 dune001 dune 1990 Sep 15 17:11 caf_20250915T154535Z.pfns
-rw-r--r-- 1 dune001 dune 1990 Sep 15 15:45 file.list
-rw-r--r-- 1 dune001 dune 1990 Sep 15 17:11 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune 1376 Sep 15 15:45 all-input-dids.txt
-rw-r--r-- 1 dune001 dune 1376 Sep 15 17:11 caf_20250915T154535Z.did
-rw-r--r-- 1 dune001 dune 1376 Sep 15 15:45 did.list
-rw-r--r-- 1 dune001 dune 0 Sep 15 15:45 debugprod.log 