justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41725.9@dunegpschedd02.fnal.gov

Jobsub ID41725.9@dunegpschedd02.fnal.gov
Workflow ID2501
Stage ID1
User namepgranger@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 07:28:10
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2025-09-18 07:50:33
From worker nodeHostnamemh-7763-15.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 07:31:03
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481123_48_20231201T135809Z_gen_g4_detsim_hitreco__20240507T203513Z_reco2.root
JobscriptExit code0
Real time18m (1137s)
CPU time7m (438s = 38%)
Max RSS bytes1862447104 (1776 MiB)
Outputting started2025-09-18 07:50:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250918T073111Z.root
Finished2025-09-18 07:50:33
Saved logsjustin-logs:41725.9-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

==============================================================================================================================
Full event                                              3.26048       7.25809       51.9264       5.67317       7.64737        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.143738      0.226638      0.307595      0.165173      0.0721156       100    
reco:pandora:StandardPandora                           0.865628       3.53537       43.1128       2.14441       6.33534        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000123888   0.00660426     0.130515     0.00112472     0.0178423       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000154308    0.0140136     0.194872     0.00218185     0.0298894       100    
reco:pandoracalo:Calorimetry                          9.9698e-05     0.0291699     0.742389     0.000847747    0.118009        100    
reco:pandorapid:Chi2ParticleID                        3.9439e-05    0.000180755   0.00174522    0.000110493   0.000216522      100    
reco:cvnmap:CVNMapper                                  2.378e-05    0.00929288     0.0567354    0.00884966     0.0102534       100    
reco:cvneva:CVNEvaluator                               1.588e-05      0.30831       2.92996      0.407727      0.339532        100    
reco:energyrecnumu:EnergyReco                         0.000227917     0.25522       2.32111      0.0332585     0.352924        100    
reco:energyrecnue:EnergyReco                          0.000137408   0.00229527     0.0329213    0.000575816   0.00508036       100    
reco:energyrecnc:EnergyReco                           0.00169108     0.444924       1.64523      0.581733       0.36206        100    
reco:energyrecnumurange:EnergyReco                    0.000297255    0.0428589      2.11001     0.00321042     0.225098        100    
reco:energyrecnumumcs:EnergyReco                      0.000990335    0.0438973      2.22517     0.00345665     0.233051        100    
reco:energyrecnumumcsllhd:EnergyReco                  0.000860998    0.0441922      2.15686     0.00343991     0.229443        100    
reco:opdec:Deconvolution                               0.429939      0.739952       1.19226       0.72582      0.160616        100    
reco:ophitspe:OpHitFinderDeco                           1.2827        1.29309       1.63191       1.28962      0.034264        100    
reco:opflash:OpFlashFinder                            0.000123138    0.001337     0.00542136    0.00121731    0.000698645      100    
reco:opslicer:OpSlicer                                4.7219e-05     0.229944       1.01109      0.176902      0.196673        100    
reco:rns:RandomNumberSaver                             1.955e-05    5.22398e-05   0.000423344   4.6774e-05     4.092e-05       100    
reco:anglereconue:NuAngularReco                       0.000147517   0.00139134     0.022634     0.000391044   0.00346727       100    
reco:anglereconumu:NuAngularReco                      7.0119e-05    0.00131221     0.032121     0.000200432    0.0041522       100    
reco:anglereconuepfps:NuAngularReco                   0.000106319   0.00136155     0.0248446    0.000330745   0.00349591       100    
reco:anglereconumupfps:NuAngularReco                  0.000102258   0.00141234     0.032337      0.0002869    0.00402701       100    
reco:anglerecohits:NuAngularReco                      7.8339e-05    0.00208295     0.0306877    0.000776069   0.00463406       100    
[art]:TriggerResults:TriggerResultInserter             9.43e-06     1.5942e-05    7.3059e-05     1.423e-05    9.67325e-06      100    
end_path:cafmaker:CAFMaker                            0.00125872     0.0217697     0.328882     0.00602644     0.0523593       100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3658.16 MB
  Peak resident set size usage (VmHWM): 1862.45 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000622276, 0.997662, 0.00171576, 
Output 1: 0.00947941, 0.153937, 0.796706, 0.0398776, 
Output 2: 0.988817, 0.010599, 0.000570326, 1.36361e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1119!
Begin processing the 94th record. run: 74481123 subRun: 1 event: 4894 at 18-Sep-2025 00:49:04 PDT
Boundary wire vector sizes: 182, 254, 225
minwire 0: 2457
minwire 1: 509
minwire 2: 2320
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000282935, 0.998579, 0.00113778, 
Output 1: 0.00268426, 0.993605, 0.00358311, 0.000127507, 
Output 2: 0.996924, 0.00299559, 7.84953e-05, 2.23555e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1654!
Begin processing the 95th record. run: 74481123 subRun: 1 event: 4895 at 18-Sep-2025 00:49:12 PDT
Boundary wire vector sizes: 488, 665, 445
minwire 0: 2395
minwire 1: 80
minwire 2: 2413
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.61751, 0.303784, 0.0787063, 
Output 1: 0.238943, 0.496425, 0.217904, 0.0467268, 
Output 2: 0.0465805, 0.211524, 0.447636, 0.294259, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1440!
Begin processing the 96th record. run: 74481123 subRun: 1 event: 4896 at 18-Sep-2025 00:49:21 PDT

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 618!
Begin processing the 97th record. run: 74481123 subRun: 1 event: 4897 at 18-Sep-2025 00:49:27 PDT
Boundary wire vector sizes: 324, 398, 331
minwire 0: 369
minwire 1: 1133
minwire 2: 52
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.0276818, 0.0159289, 0.956389, 
Output 1: 0.0240088, 0.0793893, 0.120836, 0.775766, 
Output 2: 0.283411, 0.392121, 0.222841, 0.101627, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2065!
Begin processing the 98th record. run: 74481123 subRun: 1 event: 4898 at 18-Sep-2025 00:49:37 PDT
Boundary wire vector sizes: 102, 104, 79
minwire 0: 1364
minwire 1: 2192
minwire 2: 818
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.000212572, 0.998759, 0.00102824, 
Output 1: 0.76973, 0.228883, 0.0011649, 0.000222183, 
Output 2: 0.999193, 0.00079357, 1.29018e-05, 1.07504e-06, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1641!
Begin processing the 99th record. run: 74481123 subRun: 1 event: 4899 at 18-Sep-2025 00:49:45 PDT
Boundary wire vector sizes: 203, 158, 134
minwire 0: 134
minwire 1: 2464
minwire 2: 22
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary: 
Output 0: 0.00384138, 0.718139, 0.27802, 
Output 1: 0.986399, 0.0129281, 0.000501522, 0.000171331, 
Output 2: 0.992278, 0.00743798, 0.000216062, 6.79026e-05, 


Running Ophitfinder with InputDigiType = 'recob'
Found hits: 779!
Begin processing the 100th record. run: 74481123 subRun: 1 event: 4900 at 18-Sep-2025 00:49:52 PDT

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 853!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/df/df/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74481123_48_20231201T135809Z_gen_g4_detsim_hitreco__20240507T203513Z_reco2.root
.:
total 5012
-rw-r--r-- 1 cuser cuser 1973876 Sep 18 00:49 flatcaf.root
-rw-r--r-- 1 cuser cuser 1509481 Sep 18 00:49 caf.root
-rw-r--r-- 1 cuser cuser 1509481 Sep 18 00:50 caf_fd_hd_atmo_2501_20250918T073111Z.root
-rw-r--r-- 1 cuser cuser   70031 Sep 18 00:50 caf_20250918T073111Z.log
-rw-r--r-- 1 cuser cuser   28833 Sep 18 00:50 jobscript.log
-rw-r--r-- 1 cuser cuser     216 Sep 18 00:50 caf_20250918T073111Z.file
-rw-r--r-- 1 cuser cuser     216 Sep 18 00:50 caf_20250918T073111Z.pfns
-rw-r--r-- 1 cuser cuser     216 Sep 18 00:31 file.list
-rw-r--r-- 1 cuser cuser     216 Sep 18 00:50 justin-processed-pfns.txt
-rw-r--r-- 1 cuser cuser     137 Sep 18 00:31 all-input-dids.txt
-rw-r--r-- 1 cuser cuser     137 Sep 18 00:50 caf_20250918T073111Z.did
-rw-r--r-- 1 cuser cuser     137 Sep 18 00:31 did.list
-rw-r--r-- 1 cuser cuser       0 Sep 18 00:31 debugprod.log
justIN time: 2025-09-18 18:00:24 UTC       justIN version: 01.05.00