Jobsub ID 96131.18@dunegpschedd02.fnal.gov
Jobscript log (last 10,000 characters)
sh:OpFlashFinder 0.000164417 0.00170397 0.00546517 0.00164063 0.00084823 300
reco:opslicer:OpSlicer 6.4963e-05 0.379638 1.79925 0.307045 0.326494 300
reco:rns:RandomNumberSaver 2.4671e-05 3.94408e-05 0.0004473 3.64615e-05 2.554e-05 300
reco:anglereconue:NuAngularReco 0.000216298 0.0028957 0.073669 0.000495696 0.00965352 300
reco:anglereconumu:NuAngularReco 0.000122055 0.0027377 0.0723838 0.000347585 0.00959481 300
reco:anglereconuepfps:NuAngularReco 0.000183608 0.00284376 0.0736606 0.000441099 0.0095855 300
reco:anglereconumupfps:NuAngularReco 0.000163468 0.00280903 0.0729405 0.000433604 0.00956233 300
reco:anglerecohits:NuAngularReco 0.000138785 0.00370751 0.0784919 0.000930466 0.0107055 300
[art]:TriggerResults:TriggerResultInserter 1.1191e-05 1.86029e-05 0.000195278 1.70055e-05 1.32137e-05 300
end_path:cafmaker:CAFMaker 0.00158771 0.0367521 1.01972 0.0053253 0.122603 300
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3702.98 MB
Peak resident set size usage (VmHWM): 2002.79 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 23-Sep-2025 07:23:04 -03 ModuleEndJob
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Connection error
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/1d/73/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_486_20231205T031859Z_gen_g4_detsim_hitreco__20240509T221315Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
lar exit code 1
=== Start last 100 lines of lar log file ===
Output 0: 0.00493454, 0.00200687, 0.993059,
Output 1: 0.936087, 0.0632664, 0.000484348, 0.000161905,
Output 2: 0.99921, 0.000766925, 1.94997e-05, 3.68214e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 979!
Begin processing the 294th record. run: 6483936 subRun: 1 event: 6094 at 23-Sep-2025 07:15:27 -03
Boundary wire vector sizes: 63, 38, 35
minwire 0: 1928
minwire 1: 915
minwire 2: 1990
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.00344873, 0.00254842, 0.994003,
Output 1: 0.176831, 0.813211, 0.00951242, 0.000445237,
Output 2: 0.997051, 0.00282432, 0.000117922, 6.48903e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1361!
Begin processing the 295th record. run: 6483936 subRun: 1 event: 6095 at 23-Sep-2025 07:15:37 -03
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 609!
Begin processing the 296th record. run: 6483936 subRun: 1 event: 6096 at 23-Sep-2025 07:15:46 -03
Boundary wire vector sizes: 10309, 11998, 8357
minwire 0: 50
minwire 1: 1399
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0526926, 0.895622, 0.0516854,
Output 1: 0.328243, 0.530723, 0.0755691, 0.0654645,
Output 2: 0.0177408, 0.074646, 0.229016, 0.678597,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3459!
Begin processing the 297th record. run: 6483936 subRun: 1 event: 6097 at 23-Sep-2025 07:18:17 -03
Boundary wire vector sizes: 312, 426, 208
minwire 0: 919
minwire 1: 1322
minwire 2: 978
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.967873, 0.0031211, 0.0290056,
Output 1: 0.890783, 0.107959, 0.00120072, 5.74522e-05,
Output 2: 0.217962, 0.772281, 0.00968545, 7.11698e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1599!
Begin processing the 298th record. run: 6483936 subRun: 1 event: 6098 at 23-Sep-2025 07:18:29 -03
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1434!
Begin processing the 299th record. run: 6483936 subRun: 1 event: 6099 at 23-Sep-2025 07:18:39 -03
Boundary wire vector sizes: 1046, 1007, 602
minwire 0: 1119
minwire 1: 1430
minwire 2: 1001
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.000920366, 0.989149, 0.00993107,
Output 1: 0.994166, 0.00561925, 0.000139966, 7.48827e-05,
Output 2: 0.997241, 0.00267484, 6.37907e-05, 1.99752e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2508!
Begin processing the 300th record. run: 6483936 subRun: 1 event: 6100 at 23-Sep-2025 07:18:51 -03
Boundary wire vector sizes: 39, 39, 33
minwire 0: 1625
minwire 1: 477
minwire 2: 1566
Used alternate method to get min and max wires due to vertex determination failure: 257, 556
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 74, 373
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 592, 891
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.004954, 0.00229351, 0.992753,
Output 1: 0.926738, 0.0686529, 0.00401727, 0.000591861,
Output 2: 0.996774, 0.00311288, 0.000101961, 1.11008e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 807!
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
processed files
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/1e/c1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_357_20231027T175740Z_gen_g4_detsim_hitreco__20240503T042012Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/e7/ad/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_60_20231207T082551Z_gen_g4_detsim_hitreco__20240510T040357Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/1d/73/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_486_20231205T031859Z_gen_g4_detsim_hitreco__20240509T221315Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/17/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50577091_877_20231204T004156Z_gen_g4_detsim_hitreco__20240509T211609Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/69/32/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6494524_600_20231208T071753Z_gen_g4_detsim_hitreco__20240510T055812Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/4f/f9/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_433_20231203T043235Z_gen_g4_detsim_hitreco__20240508T064309Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/4f/c0/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50568178_1107_20231202T173927Z_gen_g4_detsim_hitreco__20240507T201908Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/61/19/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6416859_583_20231203T081524Z_gen_g4_detsim_hitreco__20240509T210104Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/77/ce/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6435715_493_20231207T045114Z_gen_g4_detsim_hitreco__20240510T051342Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/42/bb/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_355_20231201T212810Z_gen_g4_detsim_hitreco__20240508T044740Z_reco2.root
.:
total 16892
-rw-r--r-- 1 nobody nobody 6395895 Sep 23 07:23 flatcaf.root
-rw-r--r-- 1 nobody nobody 5271257 Sep 23 07:23 caf.root
-rw-r--r-- 1 nobody nobody 5271257 Sep 23 07:23 caf_fd_hd_atmo_3301_20250923T085905Z.root
-rw-r--r-- 1 nobody nobody 192245 Sep 23 07:23 caf_20250923T085905Z.log
-rw-r--r-- 1 nobody nobody 75649 Sep 23 07:23 jobscript.log
-rw-r--r-- 1 nobody nobody 2081 Sep 23 07:23 caf_20250923T085905Z.file
-rw-r--r-- 1 nobody nobody 2081 Sep 23 07:23 caf_20250923T085905Z.pfns
-rw-r--r-- 1 nobody nobody 2081 Sep 23 05:59 file.list
-rw-r--r-- 1 nobody nobody 2081 Sep 23 07:23 justin-processed-pfns.txt
-rw-r--r-- 1 nobody nobody 1371 Sep 23 05:59 all-input-dids.txt
-rw-r--r-- 1 nobody nobody 1371 Sep 23 07:23 caf_20250923T085905Z.did
-rw-r--r-- 1 nobody nobody 1371 Sep 23 05:59 did.list
-rw-r--r-- 1 nobody nobody 0 Sep 23 05:59 debugprod.log 