Jobsub ID 38120.170@dunegpschedd02.fnal.gov
Jobscript log (last 10,000 characters)
:OpHitFinderDeco 0.908279 0.959411 1.02385 0.969056 0.0442758 200
reco:opflash:OpFlashFinder 0.00030706 0.00110419 0.00325718 0.000989541 0.000556948 200
reco:opslicer:OpSlicer 0.000590775 0.17391 0.577166 0.147874 0.136733 200
reco:rns:RandomNumberSaver 1.8665e-05 5.70588e-05 0.00119707 3.715e-05 0.000104771 200
reco:anglereconue:NuAngularReco 0.000145194 0.001053 0.0172399 0.000333289 0.00235307 200
reco:anglereconumu:NuAngularReco 6.4351e-05 0.000837908 0.0175291 0.000178175 0.00228516 200
reco:anglereconuepfps:NuAngularReco 9.7905e-05 0.000975511 0.0169006 0.000264148 0.00230669 200
reco:anglereconumupfps:NuAngularReco 8.8847e-05 0.000872997 0.0173804 0.000220085 0.00229614 200
reco:anglerecohits:NuAngularReco 0.000282774 0.00149446 0.0207112 0.000574805 0.00278523 200
[art]:TriggerResults:TriggerResultInserter 8.216e-06 1.20937e-05 6.8118e-05 1.046e-05 7.07194e-06 200
end_path:cafmaker:CAFMaker 0.00107051 0.0152893 0.264091 0.00530028 0.0332973 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3649.21 MB
Peak resident set size usage (VmHWM): 1976.71 MB
====================================================================================================
%MSG-s ArtException: PostEndJob 11-Sep-2025 05:59:31 CDT ModuleEndJob
---- FileOpenError BEGIN
---- FatalRootError BEGIN
Fatal Root Error: TNetXNGFile::Open
[FATAL] Socket timeout
ROOT severity: 3000
---- FatalRootError END
RootInputFileSequence::initFile(): Input file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/ba/53/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50581281_709_20231204T052355Z_gen_g4_detsim_hitreco__20240509T213601Z_reco2.root was not found or could not be opened.
---- FileOpenError END
%MSG
lar exit code 1
=== Start last 100 lines of lar log file ===
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 363!
Begin processing the 193rd record. run: 50624901 subRun: 1 event: 29193 at 11-Sep-2025 05:51:16 CDT
Boundary wire vector sizes: 56, 70, 59
minwire 0: 1681
minwire 1: 932
minwire 2: 1799
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0103274, 0.00236549, 0.987307,
Output 1: 0.0528052, 0.615516, 0.324033, 0.00764563,
Output 2: 0.966692, 0.0320713, 0.00121273, 2.38282e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1524!
Begin processing the 194th record. run: 50624901 subRun: 1 event: 29194 at 11-Sep-2025 05:51:22 CDT
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 779!
Begin processing the 195th record. run: 50624901 subRun: 1 event: 29195 at 11-Sep-2025 05:51:27 CDT
Boundary wire vector sizes: 63, 64, 53
minwire 0: 727
minwire 1: 1375
minwire 2: 364
Used alternate method to get min and max wires due to vertex determination failure: 182, 481
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 904, 1203
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 803, 1102
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0109408, 0.0030159, 0.986043,
Output 1: 0.857477, 0.137041, 0.00498258, 0.000498911,
Output 2: 0.994975, 0.00489822, 0.000118054, 8.77576e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2140!
Begin processing the 196th record. run: 50624901 subRun: 1 event: 29196 at 11-Sep-2025 05:51:32 CDT
Boundary wire vector sizes: 107, 81, 96
minwire 0: 2014
minwire 1: 1409
minwire 2: 1691
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.94153, 0.0341286, 0.024341,
Output 1: 0.0268759, 0.739517, 0.221717, 0.0118902,
Output 2: 0.30382, 0.683861, 0.0122382, 8.13893e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1686!
Begin processing the 197th record. run: 50624901 subRun: 1 event: 29197 at 11-Sep-2025 05:51:38 CDT
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 405!
Begin processing the 198th record. run: 50624901 subRun: 1 event: 29198 at 11-Sep-2025 05:51:42 CDT
Boundary wire vector sizes: 755, 1231, 739
minwire 0: 1182
minwire 1: 822
minwire 2: 1138
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.538421, 0.199292, 0.262287,
Output 1: 0.1411, 0.239356, 0.192696, 0.426848,
Output 2: 0.164293, 0.245142, 0.29536, 0.295205,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2325!
Begin processing the 199th record. run: 50624901 subRun: 1 event: 29199 at 11-Sep-2025 05:51:49 CDT
Boundary wire vector sizes: 273, 340, 286
minwire 0: 440
minwire 1: 2331
minwire 2: 61
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.122457, 0.0100249, 0.867518,
Output 1: 0.236049, 0.743618, 0.0193616, 0.000970475,
Output 2: 0.309477, 0.665282, 0.0248006, 0.000440641,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1866!
Begin processing the 200th record. run: 50624901 subRun: 1 event: 29200 at 11-Sep-2025 05:51:55 CDT
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1622!
Art has completed and will exit with status 1.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b8/15/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2.root
root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/49/49/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50624901_291_20231207T124313Z_gen_g4_detsim_hitreco__20240510T043350Z_reco2.root
root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/ba/53/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50581281_709_20231204T052355Z_gen_g4_detsim_hitreco__20240509T213601Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/20/18/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505548_454_20231202T142329Z_gen_g4_detsim_hitreco__20240508T055048Z_reco2.root
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/ab/2f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_882_20231202T104135Z_gen_g4_detsim_hitreco__20240507T214527Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/2c/f5/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6467222_54_20231207T035225Z_gen_g4_detsim_hitreco__20240510T045300Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/c7/39/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65840069_468_20231122T063338Z_gen_g4_detsim_hitreco__20240507T172215Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/05/b1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74517726_82_20231204T100354Z_gen_g4_detsim_hitreco__20240509T200342Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/5d/3e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_781_20231204T004917Z_gen_g4_detsim_hitreco__20240509T211027Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b7/52/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419766_257_20231203T173737Z_gen_g4_detsim_hitreco__20240509T214917Z_reco2.root
.:
total 10732
-rw-r--r-- 1 slot1_8 slot1_8 4105541 Sep 11 05:59 flatcaf.root
-rw-r--r-- 1 slot1_8 slot1_8 3323982 Sep 11 05:59 caf.root
-rw-r--r-- 1 slot1_8 slot1_8 3323982 Sep 11 05:59 caf_fd_hd_atmo_2501_20250911T102830Z.root
-rw-r--r-- 1 slot1_8 slot1_8 130763 Sep 11 05:59 caf_20250911T102830Z.log
-rw-r--r-- 1 slot1_8 slot1_8 54394 Sep 11 05:59 jobscript.log
-rw-r--r-- 1 slot1_8 slot1_8 1972 Sep 11 05:59 caf_20250911T102830Z.file
-rw-r--r-- 1 slot1_8 slot1_8 1972 Sep 11 05:59 caf_20250911T102830Z.pfns
-rw-r--r-- 1 slot1_8 slot1_8 1972 Sep 11 05:28 file.list
-rw-r--r-- 1 slot1_8 slot1_8 1972 Sep 11 05:59 justin-processed-pfns.txt
-rw-r--r-- 1 slot1_8 slot1_8 1374 Sep 11 05:28 all-input-dids.txt
-rw-r--r-- 1 slot1_8 slot1_8 1374 Sep 11 05:59 caf_20250911T102830Z.did
-rw-r--r-- 1 slot1_8 slot1_8 1374 Sep 11 05:28 did.list
-rw-r--r-- 1 slot1_8 slot1_8 0 Sep 11 05:28 debugprod.log 