justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261325.1@dunegpschedd02.fnal.gov

Jobsub ID261325.1@dunegpschedd02.fnal.gov
Workflow ID11117
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 22:42:40
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 02:56:04
From worker nodeHostnamedunegli-8033269-0-fnpc19105.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 22:43:52
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003033Z_gen_002412_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (15116s)
CPU time4h (14949s = 98%)
Max RSS bytes2768506880 (2640 MiB)
Outputting started2025-12-12 02:55:49
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11117/1/001/analysisLowE_20251211T224353Z.root
Finished2025-12-12 02:56:04
Saved logsjustin-logs:261325.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

108, 
Output 2: 0.101542, 0.264648, 0.59428, 0.0395303, 
Output 3: 0.974932, 0.0103305, 0.00201711, 0.0127206, 
Output 4: 0.570466, 0.388588, 0.0360772, 0.00486896, 
Output 5: 0.570911, 0.384965, 0.0374481, 0.00667639, 
Output 6: 0.0108342, 0.0329691, 0.0510744, 0.905122, 

Begin processing the 10th record. run: 5027 subRun: 0 event: 24120 at 12-Dec-2025 02:26:08 UTC
Boundary wire vector sizes: 6528, 6470, 6665
minwire 0: 60
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1699, 2198
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.158721, 
Output 1: 0.0929756, 0.0990827, 0.171303, 0.636639, 
Output 2: 0.0379145, 0.157815, 0.776289, 0.0279813, 
Output 3: 0.977834, 0.0135353, 0.00233101, 0.00629948, 
Output 4: 0.437145, 0.499817, 0.0569832, 0.00605513, 
Output 5: 0.319874, 0.35365, 0.241083, 0.0853932, 
Output 6: 0.0445279, 0.101115, 0.0988443, 0.755513, 

12-Dec-2025 02:55:05 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 02:55:05 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/ab/6b/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T003033Z_gen_002412_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1383.25       1499.01       1733.83       1460.97       92.0442        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000941487   0.00561578     0.0169542    0.00363477    0.00496533       10     
reco:pandora:StandardPandora                            1380.2        1497.56       1732.5        1459.7        92.2526        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00188547     0.0177952     0.0713638    0.00688951     0.0209065       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000288442   0.00404655     0.0130176    0.00382457    0.00338432       10     
reco:pandoracalo:Calorimetry                          0.000292912   0.00825543     0.0235588     0.0013706     0.0100237       10     
reco:pandorapid:Chi2ParticleID                         6.137e-05    0.000215355   0.000915847   9.9746e-05    0.000249193      10     
reco:cvnmap:CVNMapper                                  0.0231834     0.0292119     0.0430966     0.0272808    0.00579622       10     
reco:cvneva:CVNEvaluator                               0.808857       1.00199       2.43284      0.840883      0.477576        10     
reco:energyrecnumu:EnergyReco                         0.00754367     0.0428669     0.0804901     0.0473354     0.0310742       10     
reco:energyrecnue:EnergyReco                          0.000851937   0.00326954    0.00682939    0.00114528     0.0027877       10     
reco:energyrecnc:EnergyReco                           0.00629185     0.0404806     0.104401      0.0342348     0.0355112       10     
[art]:TriggerResults:TriggerResultInserter             1.443e-05    2.45464e-05   8.7321e-05     1.685e-05    2.12119e-05      10     
end_path:out1:RootOutput                               6.03e-06     1.0263e-05     3.03e-05      6.885e-06    7.5174e-06       10     
end_path:out1:RootOutput(write)                        0.280397      0.295451      0.340653      0.288272      0.0175419       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21684.3 MB
  Peak resident set size usage (VmHWM): 2768.51 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 187.322 | hit correction : 1.22639
hit initial : 182.144 | hit correction : 1.22671
hit initial : 169.831 | hit correction : 1.22699
hit initial : 234.819 | hit correction : 1.22736
hit initial : 218.118 | hit correction : 1.22773
hit initial : 234.847 | hit correction : 1.22803
hit initial : 290.098 | hit correction : 1.22833
hit initial : 51.3416 | hit correction : 1.22836
 *** Total Charge : 3851.64
 Looking at view: 2 | Number of hits in view: 19
Begin processing the 8th record. run: 5027 subRun: 0 event: 24118 at 12-Dec-2025 02:55:40 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 941.463
True Electron Energy: 25.5816
Completed MCParticle Loop
True Neutrino Energy: 42.8624
 Size of all hits: 0
Begin processing the 9th record. run: 5027 subRun: 0 event: 24119 at 12-Dec-2025 02:55:43 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Neutron Energy: 940.371
True Electron Energy: 22.9024
Completed MCParticle Loop
True Neutrino Energy: 28.2808
 Size of all hits: 0
Begin processing the 10th record. run: 5027 subRun: 0 event: 24120 at 12-Dec-2025 02:55:46 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  2
This pfp has size:  42
Added hit list length: 42 | Max was set to: -3.40282e+38
True Neutron Energy: 939.572
True Electron Energy: 39.8772
Completed MCParticle Loop
True Neutrino Energy: 61.91
 Size of all hits: 42
hit initial : 17.0697 | hit correction : 1.32022
hit initial : 26.5903 | hit correction : 1.32023
hit initial : 69.4082 | hit correction : 1.32431
hit initial : 24.362 | hit correction : 1.32432
hit initial : 84.3762 | hit correction : 1.31434
hit initial : 60.2259 | hit correction : 1.31433
hit initial : 71.525 | hit correction : 1.30927
hit initial : 75.0064 | hit correction : 1.32023
hit initial : 233.071 | hit correction : 1.3202
hit initial : 344.614 | hit correction : 1.32004
hit initial : 307.026 | hit correction : 1.3195
hit initial : 273.588 | hit correction : 1.31881
hit initial : 260.449 | hit correction : 1.31828
hit initial : 304.213 | hit correction : 1.31752
hit initial : 289.943 | hit correction : 1.31681
hit initial : 314.712 | hit correction : 1.31624
hit initial : 346.226 | hit correction : 1.31574
hit initial : 285.58 | hit correction : 1.31541
hit initial : 250.822 | hit correction : 1.31487
hit initial : 254.122 | hit correction : 1.31439
 *** Total Charge : 5128.28
 Looking at view: 0 | Number of hits in view: 20
hit initial : 161.541 | hit correction : 1.32027
hit initial : 155.847 | hit correction : 1.32022
hit initial : 117.759 | hit correction : 1.32018
hit initial : 172.484 | hit correction : 1.31991
hit initial : 312.649 | hit correction : 1.31941
hit initial : 172.981 | hit correction : 1.31888
hit initial : 137.722 | hit correction : 1.31853
hit initial : 166.018 | hit correction : 1.31819
hit initial : 165.532 | hit correction : 1.31779
hit initial : 149.642 | hit correction : 1.31734
hit initial : 148.717 | hit correction : 1.31693
hit initial : 134.093 | hit correction : 1.31668
hit initial : 107.831 | hit correction : 1.31649
hit initial : 108.77 | hit correction : 1.31628
hit initial : 121.583 | hit correction : 1.31605
hit initial : 156.548 | hit correction : 1.31578
hit initial : 243.291 | hit correction : 1.3156
hit initial : 145.41 | hit correction : 1.31536
hit initial : 141.343 | hit correction : 1.31504
hit initial : 161.626 | hit correction : 1.31472
hit initial : 307.896 | hit correction : 1.31435
hit initial : 108.485 | hit correction : 1.31431
 *** Total Charge : 4739.01
 Looking at view: 2 | Number of hits in view: 22
12-Dec-2025 02:55:48 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0651785     0.0802932      0.10507      0.0827766     0.0121431       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000419023   0.000725558    0.0013712    0.000525209   0.000345187      10     
end_path:ana:AnalyzeEvents         0.0645817     0.079389      0.103137      0.0817887     0.0117865       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19477 MB
  Peak resident set size usage (VmHWM): 1538.47 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 14:59:41 UTC       justIN version: 01.06.00