justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41415.11@dunegpschedd02.fnal.gov

Jobsub ID41415.11@dunegpschedd02.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 01:15:51
SiteUS_Wisconsin
EntryHCCHTPC_US_Wisconsin_osg01_rhel7
Last heartbeat2025-09-18 02:27:26
From worker nodeHostnamee4065
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit82800 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 01:17:01
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_476_20231205T031855Z_gen_g4_detsim_hitreco__20240509T221311Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50546594_921_20231202T084411Z_gen_g4_detsim_hitreco__20240507T200139Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50538503_684_20231201T174207Z_gen_g4_detsim_hitreco__20240507T191831Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74574126_201_20231208T001510Z_gen_g4_detsim_hitreco__20240510T034250Z_reco2.root
fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515154_411_20231201T082914Z_gen_g4_detsim_hitreco__20240507T184026Z_reco2.root
JobscriptExit code0
Real time1h (4207s)
CPU time30m (1833s = 43%)
Max RSS bytes2037059584 (1942 MiB)
Outputting started2025-09-18 02:27:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250918T011711Z.root
Finished2025-09-18 02:27:26
Saved logsjustin-logs:41415.11-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0.142335        500    
reco:pandorapid:Chi2ParticleID                        3.0026e-05    0.000217221   0.00313986    0.000140773   0.000271643      500    
reco:cvnmap:CVNMapper                                 1.4106e-05     0.0294797     0.196824      0.0218916     0.0295976       500    
reco:cvneva:CVNEvaluator                              1.3946e-05     0.332927       2.18708       0.39172      0.233695        500    
reco:energyrecnumu:EnergyReco                         0.000206888    0.205695       1.29631      0.0147791     0.288537        500    
reco:energyrecnue:EnergyReco                          0.000112571   0.00213202     0.0940626    0.000574075    0.0060069       500    
reco:energyrecnc:EnergyReco                            0.0012854     0.508953       1.37215      0.661437      0.338866        500    
reco:energyrecnumurange:EnergyReco                    0.000214292    0.0247023      1.20939     0.00203515     0.0930705       500    
reco:energyrecnumumcs:EnergyReco                      0.000718926    0.0259801      1.42492     0.00293946     0.101748        500    
reco:energyrecnumumcsllhd:EnergyReco                  0.000451695    0.0248995      1.1362      0.00277163     0.0904722       500    
reco:opdec:Deconvolution                               0.329974      0.574416       1.44235      0.568108      0.115047        500    
reco:ophitspe:OpHitFinderDeco                          0.976096      0.996847       1.2421       0.991226      0.0205534       500    
reco:opflash:OpFlashFinder                            0.000191899   0.00178911     0.0117717    0.00145654    0.00142406       500    
reco:opslicer:OpSlicer                                4.1087e-05     0.199589      0.841706      0.152644      0.168614        500    
reco:rns:RandomNumberSaver                            1.9406e-05    9.87037e-05   0.00110433    6.6434e-05    0.000108848      500    
reco:anglereconue:NuAngularReco                       0.000136275   0.00162179     0.0603933    0.000591112    0.0037483       500    
reco:anglereconumu:NuAngularReco                      6.7626e-05    0.00117768     0.0566257    0.000270176   0.00374458       500    
reco:anglereconuepfps:NuAngularReco                   0.000100689   0.00125801     0.0526337    0.000415413    0.0032027       500    
reco:anglereconumupfps:NuAngularReco                  8.0491e-05     0.0011031     0.0575823    0.000300272   0.00317297       500    
reco:anglerecohits:NuAngularReco                      7.7575e-05    0.00180286     0.0627514    0.000773403   0.00364873       500    
[art]:TriggerResults:TriggerResultInserter             7.725e-06    2.06208e-05   0.000610914   1.3681e-05    3.15544e-05      500    
end_path:cafmaker:CAFMaker                            0.00119829     0.0231092     0.932159      0.0096563     0.0526618       500    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3714.42 MB
  Peak resident set size usage (VmHWM): 2037.06 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 2: 1275
Used alternate method to get min and max wires due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 238, 537
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max wires due to vertex determination failure: 151, 450
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0457443, 0.00290955, 0.951346, 
Output 1: 0.0522516, 0.20062, 0.604667, 0.142461, 
Output 2: 0.984448, 0.0148491, 0.00066628, 3.6316e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1893!
Begin processing the 495th record. run: 50515154 subRun: 1 event: 41195 at 17-Sep-2025 21:25:43 CDT
Boundary wire vector sizes: 196, 253, 160
minwire 0: 1291
minwire 1: 2224
minwire 2: 634
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00034483, 0.997598, 0.00205706, 
Output 1: 0.0199516, 0.953076, 0.0260091, 0.000963349, 
Output 2: 0.992059, 0.00771927, 0.00021618, 6.04903e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2160!
Begin processing the 496th record. run: 50515154 subRun: 1 event: 41196 at 17-Sep-2025 21:25:50 CDT
Boundary wire vector sizes: 73, 138, 95
minwire 0: 195
minwire 1: 2089
minwire 2: 76
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.989301, 0.00372621, 0.00697261, 
Output 1: 0.0184599, 0.979055, 0.00240985, 7.49197e-05, 
Output 2: 0.989538, 0.0101496, 0.000305158, 7.65436e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1025!
Begin processing the 497th record. run: 50515154 subRun: 1 event: 41197 at 17-Sep-2025 21:25:56 CDT
Boundary wire vector sizes: 260, 410, 336
minwire 0: 2715
minwire 1: 613
minwire 2: 2604
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000867968, 0.996348, 0.00278358, 
Output 1: 0.00263587, 0.052835, 0.912921, 0.0316082, 
Output 2: 0.980797, 0.0181424, 0.00104041, 1.97073e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1677!
Begin processing the 498th record. run: 50515154 subRun: 1 event: 41198 at 17-Sep-2025 21:26:03 CDT
Boundary wire vector sizes: 77, 91, 81
minwire 0: 1115
minwire 1: 1220
minwire 2: 873
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.148827, 0.0075851, 0.843588, 
Output 1: 0.0329045, 0.206702, 0.659174, 0.101219, 
Output 2: 0.394523, 0.586034, 0.019218, 0.000224927, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1952!
Begin processing the 499th record. run: 50515154 subRun: 1 event: 41199 at 17-Sep-2025 21:26:09 CDT
Boundary wire vector sizes: 8651, 8657, 5945
minwire 0: 139
minwire 1: 1563
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.358389, 0.368562, 0.273049, 
Output 1: 0.198472, 0.393582, 0.221773, 0.186172, 
Output 2: 0.0131619, 0.0449398, 0.189629, 0.75227, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3194!
Begin processing the 500th record. run: 50515154 subRun: 1 event: 41200 at 17-Sep-2025 21:27:02 CDT

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 358!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/48/be/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74521450_476_20231205T031855Z_gen_g4_detsim_hitreco__20240509T221311Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/e5/0e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50546594_921_20231202T084411Z_gen_g4_detsim_hitreco__20240507T200139Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c7/8b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50538503_684_20231201T174207Z_gen_g4_detsim_hitreco__20240507T191831Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/b7/2c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74574126_201_20231208T001510Z_gen_g4_detsim_hitreco__20240510T034250Z_reco2.root
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/b2/00/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515154_411_20231201T082914Z_gen_g4_detsim_hitreco__20240507T184026Z_reco2.root
.:
total 21360
-rw-r--r-- 1 slot1_38 slot1_38 8073800 Sep 17 21:27 flatcaf.root
-rw-r--r-- 1 slot1_38 slot1_38 6656845 Sep 17 21:27 caf.root
-rw-r--r-- 1 slot1_38 slot1_38 6656845 Sep 17 21:27 caf_fd_hd_atmo_2501_20250918T011711Z.root
-rw-r--r-- 1 slot1_38 slot1_38  307283 Sep 17 21:27 caf_20250918T011711Z.log
-rw-r--r-- 1 slot1_38 slot1_38  118932 Sep 17 21:27 jobscript.log
-rw-r--r-- 1 slot1_38 slot1_38    1055 Sep 17 21:27 caf_20250918T011711Z.file
-rw-r--r-- 1 slot1_38 slot1_38    1055 Sep 17 21:27 caf_20250918T011711Z.pfns
-rw-r--r-- 1 slot1_38 slot1_38    1055 Sep 17 20:17 file.list
-rw-r--r-- 1 slot1_38 slot1_38    1055 Sep 17 21:27 justin-processed-pfns.txt
-rw-r--r-- 1 slot1_38 slot1_38     690 Sep 17 20:17 all-input-dids.txt
-rw-r--r-- 1 slot1_38 slot1_38     690 Sep 17 21:27 caf_20250918T011711Z.did
-rw-r--r-- 1 slot1_38 slot1_38     690 Sep 17 20:17 did.list
-rw-r--r-- 1 slot1_38 slot1_38       0 Sep 17 20:17 debugprod.log
justIN time: 2025-09-18 23:45:01 UTC       justIN version: 01.05.00