Jobsub ID 102435.80@dunegpschedd01.fnal.gov
Jobscript log (last 10,000 characters)
9e-05 0.37984 1.60402 0.306427 0.300864 500
reco:rns:RandomNumberSaver 2.1593e-05 8.51087e-05 0.00129135 5.7923e-05 0.000136471 500
reco:anglereconue:NuAngularReco 0.000178158 0.0032838 0.123061 0.000766382 0.00951401 500
reco:anglereconumu:NuAngularReco 0.000107776 0.002853 0.121627 0.000453104 0.0092793 500
reco:anglereconuepfps:NuAngularReco 0.000152419 0.00306369 0.119073 0.000646573 0.00920763 500
reco:anglereconumupfps:NuAngularReco 0.000136363 0.00299671 0.12054 0.00059795 0.00924719 500
reco:anglerecohits:NuAngularReco 0.000127028 0.00407749 0.134231 0.00134479 0.0104104 500
[art]:TriggerResults:TriggerResultInserter 1.1652e-05 2.49247e-05 0.000606791 2.28095e-05 2.67073e-05 500
end_path:cafmaker:CAFMaker 0.00170667 0.0431344 1.69178 0.012476 0.118573 500
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3717.2 MB
Peak resident set size usage (VmHWM): 2050.65 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 23-Sep-2025 08:21:39 -03 ModuleEndJob
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Connection error
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/a5/d7/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50579427_12_20231204T042214Z_gen_g4_detsim_hitreco__20240509T203427Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
lar exit code 1
=== Start last 100 lines of lar log file ===
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 0!
Begin processing the 494th record. run: 6402377 subRun: 1 event: 91694 at 23-Sep-2025 08:17:30 -03
Boundary wire vector sizes: 39, 53, 47
minwire 0: 1853
minwire 1: 1318
minwire 2: 1757
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.957958, 0.00135999, 0.0406818,
Output 1: 0.00236047, 0.995111, 0.00243998, 8.84118e-05,
Output 2: 0.957311, 0.0417191, 0.00095295, 1.67841e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1115!
Begin processing the 495th record. run: 6402377 subRun: 1 event: 91695 at 23-Sep-2025 08:17:40 -03
Boundary wire vector sizes: 297, 358, 331
minwire 0: 465
minwire 1: 1543
minwire 2: 497
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0473811, 0.0149717, 0.937647,
Output 1: 0.345299, 0.544291, 0.0855051, 0.0249053,
Output 2: 0.613575, 0.348421, 0.0347687, 0.00323467,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1647!
Begin processing the 496th record. run: 6402377 subRun: 1 event: 91696 at 23-Sep-2025 08:17:51 -03
Boundary wire vector sizes: 59, 79, 66
minwire 0: 2305
minwire 1: 897
minwire 2: 1987
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.857355, 0.0938349, 0.0488097,
Output 1: 0.123977, 0.831719, 0.0425026, 0.0018015,
Output 2: 0.274973, 0.719714, 0.00528073, 3.17863e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1300!
Begin processing the 497th record. run: 6402377 subRun: 1 event: 91697 at 23-Sep-2025 08:18:02 -03
Boundary wire vector sizes: 5047, 3388, 2457
minwire 0: 1349
minwire 1: 1027
minwire 2: 1165
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0339615, 0.953117, 0.0129215,
Output 1: 0.390773, 0.450611, 0.0930268, 0.0655888,
Output 2: 0.0747204, 0.280331, 0.356871, 0.288078,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3335!
Begin processing the 498th record. run: 6402377 subRun: 1 event: 91698 at 23-Sep-2025 08:18:39 -03
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 532!
Begin processing the 499th record. run: 6402377 subRun: 1 event: 91699 at 23-Sep-2025 08:18:49 -03
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1000!
Begin processing the 500th record. run: 6402377 subRun: 1 event: 91700 at 23-Sep-2025 08:18:57 -03
Boundary wire vector sizes: 954, 1073, 849
minwire 0: 589
minwire 1: 1882
minwire 2: 99
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0248292, 0.446127, 0.529044,
Output 1: 0.0178296, 0.0688615, 0.139077, 0.774232,
Output 2: 0.392164, 0.458492, 0.116361, 0.0329843,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2735!
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bf/b6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_793_20231208T050858Z_gen_g4_detsim_hitreco__20240510T042531Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ad/51/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6432497_771_20231207T000738Z_gen_g4_detsim_hitreco__20240510T043030Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/86/20/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6407968_1007_20231203T064857Z_gen_g4_detsim_hitreco__20240508T040942Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/57/38/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74531892_334_20231207T033425Z_gen_g4_detsim_hitreco__20240510T043738Z_reco2.root
root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/32/36/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_916_20231202T113759Z_gen_g4_detsim_hitreco__20240507T211325Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/a5/d7/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50579427_12_20231204T042214Z_gen_g4_detsim_hitreco__20240509T203427Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/5d/fd/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472706_352_20231208T081435Z_gen_g4_detsim_hitreco__20240510T053905Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/89/ba/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517585_652_20231204T072321Z_gen_g4_detsim_hitreco__20240509T210723Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/1c/ec/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_742_20231203T044715Z_gen_g4_detsim_hitreco__20240508T064758Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/f3/09/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6409602_701_20231202T151133Z_gen_g4_detsim_hitreco__20240508T031206Z_reco2.root
.:
total 26448
-rw-r--r-- 1 nobody nobody 9993571 Sep 23 08:21 flatcaf.root
-rw-r--r-- 1 nobody nobody 8300033 Sep 23 08:21 caf.root
-rw-r--r-- 1 nobody nobody 8300033 Sep 23 08:21 caf_fd_hd_atmo_3301_20250923T090828Z.root
-rw-r--r-- 1 nobody nobody 311857 Sep 23 08:21 caf_20250923T090828Z.log
-rw-r--r-- 1 nobody nobody 125947 Sep 23 08:21 jobscript.log
-rw-r--r-- 1 nobody nobody 1915 Sep 23 08:21 caf_20250923T090828Z.file
-rw-r--r-- 1 nobody nobody 1915 Sep 23 08:21 caf_20250923T090828Z.pfns
-rw-r--r-- 1 nobody nobody 1915 Sep 23 06:08 file.list
-rw-r--r-- 1 nobody nobody 1915 Sep 23 08:21 justin-processed-pfns.txt
-rw-r--r-- 1 nobody nobody 1373 Sep 23 06:08 all-input-dids.txt
-rw-r--r-- 1 nobody nobody 1373 Sep 23 08:21 caf_20250923T090828Z.did
-rw-r--r-- 1 nobody nobody 1373 Sep 23 06:08 did.list
-rw-r--r-- 1 nobody nobody 0 Sep 23 06:08 debugprod.log 