Jobsub ID 40095.187@dunegpschedd02.fnal.gov
Jobscript log (last 10,000 characters)
0.00013453 0.00179184 0.123483 0.000322151 0.009048 200
reco:anglereconumu:NuAngularReco 6.8111e-05 0.00163964 0.121397 0.000175889 0.00890449 200
reco:anglereconuepfps:NuAngularReco 0.00010332 0.00173258 0.123072 0.000257405 0.00901781 200
reco:anglereconumupfps:NuAngularReco 9.1542e-05 0.00171844 0.124969 0.00023691 0.00914908 200
reco:anglerecohits:NuAngularReco 7.7078e-05 0.00220471 0.134893 0.00054956 0.00989641 200
[art]:TriggerResults:TriggerResultInserter 9.392e-06 1.19013e-05 5.7792e-05 1.1172e-05 3.92736e-06 200
end_path:cafmaker:CAFMaker 0.00130991 0.0225832 1.41135 0.00453594 0.104366 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3678.52 MB
Peak resident set size usage (VmHWM): 2001.71 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 15-Sep-2025 16:52:37 BST ModuleEndJob
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Socket timeout
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/be/af/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422149_408_20231204T120421Z_gen_g4_detsim_hitreco__20240509T220900Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
lar exit code 1
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 451, 415, 429
minwire 0: 2101
minwire 1: 183
minwire 2: 2486
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0015296, 0.972694, 0.0257766,
Output 1: 0.00459506, 0.149662, 0.828826, 0.0169164,
Output 2: 0.978459, 0.0205682, 0.000948367, 2.42058e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1180!
Begin processing the 195th record. run: 74506302 subRun: 1 event: 80895 at 15-Sep-2025 16:44:52 BST
Boundary wire vector sizes: 67, 72, 65
minwire 0: 1830
minwire 1: 907
minwire 2: 1320
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.035774, 0.005312, 0.958914,
Output 1: 0.072504, 0.535002, 0.377425, 0.0150695,
Output 2: 0.917139, 0.0803558, 0.00245819, 4.68536e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1972!
Begin processing the 196th record. run: 74506302 subRun: 1 event: 80896 at 15-Sep-2025 16:44:55 BST
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 409!
Begin processing the 197th record. run: 74506302 subRun: 1 event: 80897 at 15-Sep-2025 16:44:57 BST
Boundary wire vector sizes: 153, 110, 105
minwire 0: 2880
minwire 1: 478
minwire 2: 2740
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.810326, 0.136524, 0.0531507,
Output 1: 0.020121, 0.28385, 0.639841, 0.0561878,
Output 2: 0.773445, 0.21575, 0.010594, 0.000211191,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1121!
Begin processing the 198th record. run: 74506302 subRun: 1 event: 80898 at 15-Sep-2025 16:44:59 BST
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
PcaShowerParticleBuildingAlgorithm::OpeningAngle - principal eigenvalue less than or equal to 0.
Boundary wire vector sizes: 1575, 483, 933
minwire 0: 9
minwire 1: 2390
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.993767, 0.00144941, 0.00478381,
Output 1: 0.0784391, 0.234196, 0.229325, 0.45804,
Output 2: 0.434575, 0.411854, 0.124098, 0.0294736,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2513!
Begin processing the 199th record. run: 74506302 subRun: 1 event: 80899 at 15-Sep-2025 16:45:03 BST
Boundary wire vector sizes: 893, 1029, 928
minwire 0: 1918
minwire 1: 509
minwire 2: 1968
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.000920436, 0.994954, 0.00412533,
Output 1: 0.845779, 0.147379, 0.00631963, 0.000522962,
Output 2: 0.107257, 0.870512, 0.021846, 0.00038564,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2208!
Begin processing the 200th record. run: 74506302 subRun: 1 event: 80900 at 15-Sep-2025 16:45:06 BST
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 6!
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
processed files
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/80/73/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50649927_561_20231208T052343Z_gen_g4_detsim_hitreco__20240510T065405Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/3e/79/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_808_20231202T182018Z_gen_g4_detsim_hitreco__20240508T065538Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/be/af/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422149_408_20231204T120421Z_gen_g4_detsim_hitreco__20240509T220900Z_reco2.root
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/78/ce/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_124_20231207T161745Z_gen_g4_detsim_hitreco__20240510T064035Z_reco2.root
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/b1/39/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_491_20231203T053213Z_gen_g4_detsim_hitreco__20240508T075107Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/f9/ef/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6420853_86_20231204T032444Z_gen_g4_detsim_hitreco__20240509T211605Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/3a/0c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6422672_271_20231204T081307Z_gen_g4_detsim_hitreco__20240509T213904Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/16/d6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_146_20231203T032955Z_gen_g4_detsim_hitreco__20240508T071556Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/4e/0e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74551443_740_20231207T113334Z_gen_g4_detsim_hitreco__20240510T054253Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/19/1c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50572032_825_20231203T070053Z_gen_g4_detsim_hitreco__20240508T080306Z_reco2.root
.:
total 9408
-rw-r--r-- 1 pltdune002 pltdune 3596095 Sep 15 16:52 flatcaf.root
-rw-r--r-- 1 pltdune002 pltdune 2872743 Sep 15 16:52 caf.root
-rw-r--r-- 1 pltdune002 pltdune 2872743 Sep 15 16:52 caf_fd_hd_atmo_2501_20250915T153045Z.root
-rw-r--r-- 1 pltdune002 pltdune 126602 Sep 15 16:52 caf_20250915T153045Z.log
-rw-r--r-- 1 pltdune002 pltdune 66578 Sep 15 16:52 jobscript.log
-rw-r--r-- 1 pltdune002 pltdune 1965 Sep 15 16:52 caf_20250915T153045Z.file
-rw-r--r-- 1 pltdune002 pltdune 1965 Sep 15 16:52 caf_20250915T153045Z.pfns
-rw-r--r-- 1 pltdune002 pltdune 1965 Sep 15 16:30 file.list
-rw-r--r-- 1 pltdune002 pltdune 1965 Sep 15 16:52 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune002 pltdune 1375 Sep 15 16:30 all-input-dids.txt
-rw-r--r-- 1 pltdune002 pltdune 1375 Sep 15 16:52 caf_20250915T153045Z.did
-rw-r--r-- 1 pltdune002 pltdune 1375 Sep 15 16:30 did.list
-rw-r--r-- 1 pltdune002 pltdune 0 Sep 15 16:30 debugprod.log 