justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267776.1@dunegpschedd01.fnal.gov

Jobsub ID267776.1@dunegpschedd01.fnal.gov
Workflow ID11177
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-13 05:32:16
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-13 17:28:01
From worker nodeHostnamedunegli-7890610-0-fnpc9038.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 05:33:46
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232726Z_gen_000123_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time11h (42833s)
CPU time11h (41665s = 97%)
Max RSS bytes2800193536 (2670 MiB)
Outputting started2025-12-13 17:27:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11177/1/001/analysisLowE_20251213T053347Z.root
Finished2025-12-13 17:28:01
Saved logsjustin-logs:267776.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ata product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:24:43 UTC run: 5027 subRun: 0 event: 1230
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  13-Dec-2025 17:24:43 UTC run: 5027 subRun: 0 event: 1230
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6892, 6796, 6870
minwire 0: 48
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 583, 1082
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.111049, 
Output 1: 0.0555659, 0.0602451, 0.141861, 0.742328, 
Output 2: 0.0647672, 0.198909, 0.709896, 0.0264284, 
Output 3: 0.965482, 0.0265912, 0.00275844, 0.00516797, 
Output 4: 0.343951, 0.580911, 0.0689307, 0.00620665, 
Output 5: 0.585578, 0.334811, 0.0648377, 0.0147738, 
Output 6: 0.0208589, 0.0553782, 0.0675198, 0.856243, 

13-Dec-2025 17:25:48 UTC  Closed output file "matt_reco2.root"
13-Dec-2025 17:25:48 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/72/48/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T232726Z_gen_000123_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3874.22       4245.04       4416.19       4303.72       166.576        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00166569    0.00569802     0.0239613    0.00206897    0.00743576       10     
reco:pandora:StandardPandora                            3804.18       4172.69       4343.89       4232.89       168.469        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000900801    0.125904      0.226002      0.134951      0.0788749       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000845368    0.0285932     0.166667      0.010983      0.0475945       10     
reco:pandoracalo:Calorimetry                          0.000603076    0.0026928     0.0148139    0.00116789    0.00409052       10     
reco:pandorapid:Chi2ParticleID                        0.000171793   0.000887972   0.00642093    0.000279243   0.00184608       10     
reco:cvnmap:CVNMapper                                   0.1313       0.185956      0.269402      0.181991      0.0355739       10     
reco:cvneva:CVNEvaluator                                1.65141       2.92896       9.79781       2.26056       2.30832        10     
reco:energyrecnumu:EnergyReco                           60.2736       68.3234       74.3621       69.8207       4.30875        10     
reco:energyrecnue:EnergyReco                          0.00317768    0.00584834     0.0187044    0.00402077    0.00444348       10     
reco:energyrecnc:EnergyReco                            0.0156576     0.0178125     0.0220388     0.0175548    0.00168121       10     
[art]:TriggerResults:TriggerResultInserter            4.0063e-05    6.60774e-05   0.000221425   4.98845e-05   5.19641e-05      10     
end_path:out1:RootOutput                              1.3804e-05    2.09116e-05   6.4255e-05    1.58755e-05   1.4584e-05       10     
end_path:out1:RootOutput(write)                         0.64302      0.720586       1.02986      0.675064       0.11103        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11906.9 MB
  Peak resident set size usage (VmHWM): 2800.19 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
 Looking at view: 0 | Number of hits in view: 1
hit initial : 75.5429 | hit correction : 1.08716
hit initial : 101.247 | hit correction : 1.08713
 *** Total Charge : 192.197
 Looking at view: 1 | Number of hits in view: 2
hit initial : 34.5105 | hit correction : 1.0878
hit initial : 201.218 | hit correction : 1.08713
 *** Total Charge : 256.291
 Looking at view: 2 | Number of hits in view: 2
Begin processing the 9th record. run: 5027 subRun: 0 event: 1229 at 13-Dec-2025 17:27:25 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  12
Added hit list length: 12 | Max was set to: -3.40282e+38
This pfp has size:  7
This pfp has size:  5
This pfp has size:  4
True Electron Energy: 4.14534
Completed MCParticle Loop
True Neutrino Energy: 9.64815
 Size of all hits: 12
hit initial : 23.7738 | hit correction : 1.12722
hit initial : 42.8468 | hit correction : 1.12704
hit initial : 39.5119 | hit correction : 1.12679
hit initial : 54.705 | hit correction : 1.12679
 *** Total Charge : 181.251
 Looking at view: 0 | Number of hits in view: 4
hit initial : 23.1082 | hit correction : 1.12715
hit initial : 24.6616 | hit correction : 1.12715
hit initial : 28.5373 | hit correction : 1.12681
hit initial : 63.1287 | hit correction : 1.12676
hit initial : 28.3158 | hit correction : 1.12677
 *** Total Charge : 189.036
 Looking at view: 1 | Number of hits in view: 5
hit initial : 42.1561 | hit correction : 1.12715
hit initial : 26.3194 | hit correction : 1.12675
hit initial : 91.3581 | hit correction : 1.12676
 *** Total Charge : 180.111
 Looking at view: 2 | Number of hits in view: 3
Begin processing the 10th record. run: 5027 subRun: 0 event: 1230 at 13-Dec-2025 17:27:31 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  6
This pfp has size:  25
Added hit list length: 25 | Max was set to: -3.40282e+38
This pfp has size:  9
This pfp has size:  6
True Neutron Energy: 939.567
True Electron Energy: 16.4918
Completed MCParticle Loop
True Neutrino Energy: 33.7308
 Size of all hits: 25
hit initial : 132.895 | hit correction : 1.04254
hit initial : 50.328 | hit correction : 1.04253
hit initial : 62.4916 | hit correction : 1.04273
hit initial : 25.1727 | hit correction : 1.04273
hit initial : 165.892 | hit correction : 1.04271
hit initial : 217.536 | hit correction : 1.04212
hit initial : 406.772 | hit correction : 1.04283
hit initial : 439.326 | hit correction : 1.04206
hit initial : 243.862 | hit correction : 1.04304
 *** Total Charge : 1818.46
 Looking at view: 0 | Number of hits in view: 9
hit initial : 66.3055 | hit correction : 1.04253
hit initial : 129.479 | hit correction : 1.04254
hit initial : 62.2242 | hit correction : 1.04272
hit initial : 151.093 | hit correction : 1.04265
hit initial : 625.553 | hit correction : 1.04265
hit initial : 144.049 | hit correction : 1.04199
hit initial : 61.6014 | hit correction : 1.04206
hit initial : 163.655 | hit correction : 1.04305
 *** Total Charge : 1463.76
 Looking at view: 1 | Number of hits in view: 8
hit initial : 243.602 | hit correction : 1.04254
hit initial : 230.594 | hit correction : 1.04249
hit initial : 76.7985 | hit correction : 1.04272
hit initial : 38.4042 | hit correction : 1.04269
hit initial : 560.206 | hit correction : 1.0428
hit initial : 271.558 | hit correction : 1.04214
hit initial : 263.492 | hit correction : 1.04189
hit initial : 213.169 | hit correction : 1.04312
 *** Total Charge : 1978.56
 Looking at view: 2 | Number of hits in view: 8
13-Dec-2025 17:27:38 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.173829      0.212334      0.319923      0.205747      0.0377975       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000980421    0.0040362     0.0189621     0.0012459     0.005854        10     
end_path:ana:AnalyzeEvents         0.172445      0.207883      0.299735      0.203218      0.0329639       10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9809.67 MB
  Peak resident set size usage (VmHWM): 1534.44 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 12:12:13 UTC       justIN version: 01.06.00