justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267040.0@dunegpschedd01.fnal.gov

Jobsub ID267040.0@dunegpschedd01.fnal.gov
Workflow ID11114
Stage ID1
User nameosbiston@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-12-11 22:02:38
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-12-12 02:13:45
From worker nodeHostnamedunegli-7877334-0-fnpc19111.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-11 22:03:31
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T200658Z_gen_002540_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (14999s)
CPU time4h (14877s = 99%)
Max RSS bytes2918469632 (2783 MiB)
Outputting started2025-12-12 02:13:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/osbiston/fnal/11114/1/001/analysisLowE_20251211T220332Z.root
Finished2025-12-12 02:13:45
Saved logsjustin-logs:267040.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

wer@BeginModule  12-Dec-2025 02:12:48 UTC run: 5026 subRun: 0 event: 25400
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  12-Dec-2025 02:12:48 UTC run: 5026 subRun: 0 event: 25400
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 6767, 6729, 6605
minwire 0: 41
minwire 1: 1
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 61, 560
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0936039, 
Output 1: 0.0493293, 0.11581, 0.138673, 0.696188, 
Output 2: 0.0562332, 0.220787, 0.681988, 0.0409919, 
Output 3: 0.932112, 0.0488124, 0.0090032, 0.0100724, 
Output 4: 0.628923, 0.316495, 0.048175, 0.00640782, 
Output 5: 0.557201, 0.286557, 0.120838, 0.0354039, 
Output 6: 0.053322, 0.0883635, 0.083714, 0.774601, 

12-Dec-2025 02:12:50 UTC  Closed output file "matt_reco2.root"
12-Dec-2025 02:12:50 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/82/02/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T200658Z_gen_002540_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1398.88       1487.17       1605.09       1479.04       51.7813        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00187068    0.00597606     0.0167148    0.00412673    0.00473376       10     
reco:pandora:StandardPandora                            1397.48       1485.69       1603.69       1477.81       51.9105        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000569336    0.0156833     0.0820858    0.00833587     0.0231268       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00114626    0.00370763    0.00766079    0.00423976    0.00192873       10     
reco:pandoracalo:Calorimetry                          0.000222743   0.00304479     0.024529     0.000392529   0.00717514       10     
reco:pandorapid:Chi2ParticleID                        5.8391e-05    0.000183188   0.000572386   8.9221e-05    0.000177462      10     
reco:cvnmap:CVNMapper                                  0.0229051     0.0323353     0.0745318     0.0271044     0.0148173       10     
reco:cvneva:CVNEvaluator                                0.80024       1.01037       2.6691       0.822029      0.553265        10     
reco:energyrecnumu:EnergyReco                         0.00246674     0.0497056      0.09868      0.0718991     0.0370176       10     
reco:energyrecnue:EnergyReco                          0.000836478   0.00284762    0.00695611    0.00126747    0.00263337       10     
reco:energyrecnc:EnergyReco                           0.00651798     0.0491374     0.231898      0.006783      0.067817        10     
[art]:TriggerResults:TriggerResultInserter             1.342e-05    2.05351e-05    6.074e-05    1.46405e-05   1.39702e-05      10     
end_path:out1:RootOutput                               5.26e-06     8.6611e-06     2.293e-05    6.9255e-06    4.99236e-06      10     
end_path:out1:RootOutput(write)                        0.274415       0.30192      0.372131      0.287829      0.0322945       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21679.8 MB
  Peak resident set size usage (VmHWM): 2918.47 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Start last 100 lines of lar log file ===
hit initial : 315.598 | hit correction : 1.31092
hit initial : 140.877 | hit correction : 1.31015
hit initial : 116.935 | hit correction : 1.31004
hit initial : 115.571 | hit correction : 1.31009
hit initial : 137.726 | hit correction : 1.31017
hit initial : 47.3261 | hit correction : 1.30877
hit initial : 75.1798 | hit correction : 1.31024
hit initial : 162.084 | hit correction : 1.31026
hit initial : 129.373 | hit correction : 1.31036
hit initial : 117.038 | hit correction : 1.31046
hit initial : 122.614 | hit correction : 1.31049
hit initial : 150.599 | hit correction : 1.31051
hit initial : 121.927 | hit correction : 1.31056
hit initial : 137.359 | hit correction : 1.3107
 *** Total Charge : 2610.41
 Looking at view: 2 | Number of hits in view: 15
Begin processing the 8th record. run: 5026 subRun: 0 event: 25398 at 12-Dec-2025 02:13:22 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 6.44437
Completed MCParticle Loop
True Neutrino Energy: 10.1696
 Size of all hits: 0
Begin processing the 9th record. run: 5026 subRun: 0 event: 25399 at 12-Dec-2025 02:13:24 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  0
True Electron Energy: 61.9852
Completed MCParticle Loop
True Neutrino Energy: 67.5389
 Size of all hits: 0
Begin processing the 10th record. run: 5026 subRun: 0 event: 25400 at 12-Dec-2025 02:13:27 UTC
*--------------------------- Running LowE Analysis --------------------------------------*
Number of reconstructed particles:  3
This pfp has size:  33
Added hit list length: 33 | Max was set to: -3.40282e+38
This pfp has size:  13
True Neutron Energy: 939.574
True Neutron Energy: 943.534
True Neutron Energy: 940.844
True Electron Energy: 36.6931
Completed MCParticle Loop
True Neutrino Energy: 53.9634
 Size of all hits: 33
hit initial : 55.3951 | hit correction : 1.18065
hit initial : 25.3213 | hit correction : 1.18066
hit initial : 51.7674 | hit correction : 1.18544
hit initial : 71.572 | hit correction : 1.18874
hit initial : 174.214 | hit correction : 1.18549
hit initial : 80.5919 | hit correction : 1.1802
hit initial : 127.282 | hit correction : 1.18876
hit initial : 71.0191 | hit correction : 1.1887
hit initial : 249.276 | hit correction : 1.18582
hit initial : 487.078 | hit correction : 1.18604
hit initial : 818.387 | hit correction : 1.18645
hit initial : 352.69 | hit correction : 1.18666
hit initial : 260.273 | hit correction : 1.18562
 *** Total Charge : 3350.49
 Looking at view: 0 | Number of hits in view: 13
hit initial : 68.7799 | hit correction : 1.18725
hit initial : 213.536 | hit correction : 1.18728
hit initial : 298.024 | hit correction : 1.1873
hit initial : 224.384 | hit correction : 1.18727
hit initial : 203.258 | hit correction : 1.18722
hit initial : 227.135 | hit correction : 1.18711
hit initial : 260.708 | hit correction : 1.18677
hit initial : 371.966 | hit correction : 1.18648
hit initial : 402.866 | hit correction : 1.18625
hit initial : 268.699 | hit correction : 1.18595
hit initial : 205.106 | hit correction : 1.18582
hit initial : 258.52 | hit correction : 1.18555
hit initial : 75.4713 | hit correction : 1.18548
hit initial : 31.0729 | hit correction : 1.18721
hit initial : 28.4603 | hit correction : 1.18721
hit initial : 297.997 | hit correction : 1.18572
 *** Total Charge : 4076.88
 Looking at view: 1 | Number of hits in view: 16
hit initial : 189.158 | hit correction : 1.18549
hit initial : 92.6981 | hit correction : 1.1856
hit initial : 117.419 | hit correction : 1.18585
hit initial : 646.815 | hit correction : 1.18574
 *** Total Charge : 1240.34
 Looking at view: 2 | Number of hits in view: 4
12-Dec-2025 02:13:30 UTC  Closed input file "matt_reco2.root"

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                         0.0659151     0.0860103     0.123755      0.0834445     0.0154925       10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000351494   0.000675121   0.00152806    0.000603076   0.000348954      10     
end_path:ana:AnalyzeEvents         0.0654222     0.0851671     0.123247      0.0825324     0.015643        10     
====================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19475.9 MB
  Peak resident set size usage (VmHWM): 1536.83 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-07 15:00:05 UTC       justIN version: 01.06.00