justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233774.4@dunegpschedd02.fnal.gov

Jobsub ID233774.4@dunegpschedd02.fnal.gov
Workflow ID9413
Stage ID1
User namedrivera@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-28 15:32:19
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-10-28 18:26:42
From worker nodeHostnamedunegli-7387337-0-fnpc22012.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes7864320000 (7500 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-28 15:33:50
Input fileshd-protodune:np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000.hdf5
JobscriptExit code0
Real time2h (10354s)
CPU time2h (9321s = 90%)
Max RSS bytes2621235200 (2499 MiB)
Outputting started2025-10-28 18:26:25
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/drivera/lar1010/JustinTest/fnal/09413/1/001/np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.root
Finished2025-10-28 18:26:42
Saved logsjustin-logs:233774.4-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r channel 8692
wclsFrameSaver: no samples within desired window for channel 8692
wclsFrameSaver: no samples within desired window for channel 8693
wclsFrameSaver: no samples within desired window for channel 8693
wclsFrameSaver: no samples within desired window for channel 8694
wclsFrameSaver: no samples within desired window for channel 8694
wclsFrameSaver: no samples within desired window for channel 8694
wclsFrameSaver: no samples within desired window for channel 8695
wclsFrameSaver: no samples within desired window for channel 8695
wclsFrameSaver: no samples within desired window for channel 8695
wclsFrameSaver: no samples within desired window for channel 8696
wclsFrameSaver: no samples within desired window for channel 8696
wclsFrameSaver: no samples within desired window for channel 8697
wclsFrameSaver: no samples within desired window for channel 8697
wclsFrameSaver: no samples within desired window for channel 8698
wclsFrameSaver: no samples within desired window for channel 8698
wclsFrameSaver: no samples within desired window for channel 8699
wclsFrameSaver: no samples within desired window for channel 8701
wclsFrameSaver: no samples within desired window for channel 8702
wclsFrameSaver: no samples within desired window for channel 8703
wclsFrameSaver: no samples within desired window for channel 8930
wclsFrameSaver: no samples within desired window for channel 8931
wclsFrameSaver: no samples within desired window for channel 8931
wclsFrameSaver: no samples within desired window for channel 8932
wclsFrameSaver: no samples within desired window for channel 8932
wclsFrameSaver: no samples within desired window for channel 8933
wclsFrameSaver: no samples within desired window for channel 8934
wclsFrameSaver: no samples within desired window for channel 8934
wclsFrameSaver: no samples within desired window for channel 8935
wclsFrameSaver: no samples within desired window for channel 8935
wclsFrameSaver: no samples within desired window for channel 8936
wclsFrameSaver: no samples within desired window for channel 8936
wclsFrameSaver: no samples within desired window for channel 8936
wclsFrameSaver: no samples within desired window for channel 8937
wclsFrameSaver: no samples within desired window for channel 8937
wclsFrameSaver: no samples within desired window for channel 8938
wclsFrameSaver: no samples within desired window for channel 8939
wclsFrameSaver: no samples within desired window for channel 8939
wclsFrameSaver: no samples within desired window for channel 8939
wclsFrameSaver: no samples within desired window for channel 8940
wclsFrameSaver: no samples within desired window for channel 8940
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9476
wclsFrameSaver: no samples within desired window for channel 9476
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9514
wclsFrameSaver: no samples within desired window for channel 9515
wclsFrameSaver: no samples within desired window for channel 9515
wclsFrameSaver: no samples within desired window for channel 9516
wclsFrameSaver: no samples within desired window for channel 9516
wclsFrameSaver: no samples within desired window for channel 9517
wclsFrameSaver: no samples within desired window for channel 9517
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9523
wclsFrameSaver: no samples within desired window for channel 9524
wclsFrameSaver: no samples within desired window for channel 9524
wclsFrameSaver: no samples within desired window for channel 9525
wclsFrameSaver: no samples within desired window for channel 9526
wclsFrameSaver: no samples within desired window for channel 9526
wclsFrameSaver: no samples within desired window for channel 9527
wclsFrameSaver: no samples within desired window for channel 9528
wclsFrameSaver: no samples within desired window for channel 9528
FrameSaver: q=876385 n=621414 tag=wiener
28-Oct-2025 18:26:24 UTC  Closed output file "np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          73.3492       263.598       3033.35       122.482       476.086        39     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        3.6228e-05    5.29741e-05   7.5723e-05    5.4193e-05    1.0072e-05       39     
produce:tpcrawdecoder:PDHDTPCReader                 21.8824       25.3352       30.7166       24.7665       3.30315        39     
produce:fembfilter:PDHDFEMBFilter                 4.7601e-05    6.21591e-05   0.000226219   5.6698e-05    2.71499e-05      39     
produce:wclsdatahd:WireCellToolkit                  42.9087       45.3091       47.3675       45.2496       0.99582        39     
produce:gaushit:GausHitFinder                      0.218254      0.637597       1.35079      0.611929      0.277344        39     
produce:nhitsfilter:NumberOfHitsFilter            0.000132801   0.000365194   0.00075984    0.00034317    0.000136665      39     
produce:reco3d:SpacePointSolver                    0.112161       15.8607       97.2346       5.80608       20.5892        39     
produce:hitpdune:DisambigFromSpacePoints           0.0895573      2.50176       9.91307       1.9674        2.50204        39     
produce:pandora:StandardPandora                     2.21741       167.78        2836.72       31.4287       456.269        39     
produce:pandoraWriter:StandardPandora              0.0426637     0.143651      0.445789      0.125834      0.0788972       39     
produce:pandoraTrack:LArPandoraTrackCreation       0.257607       1.51851       6.06924       1.25314       1.1253         39     
produce:pandoraShower:LArPandoraShowerCreation     0.293274       1.71466       4.6623        1.54395       1.02386        39     
produce:pandoracalo:Calorimetry                     0.17074      0.944372       3.85166      0.743492      0.726094        39     
produce:pandoracalonosce:Calorimetry                0.15609       0.90493       3.71348       0.71728      0.708477        39     
[art]:TriggerResults:TriggerResultInserter         1.055e-05    2.07879e-05   0.000106372   1.8746e-05    1.41483e-05      39     
end_path:out1:RootOutput                           5.761e-06    7.89049e-06   3.6379e-05     6.853e-06    4.90979e-06      39     
end_path:out1:RootOutput(write)                    0.301115      0.839816       2.09184      0.690411      0.403421        39     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4518.72 MB
  Peak resident set size usage (VmHWM): 2621.24 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.root', 'start_time': 1761665635.013923, 'end_time': 1761675984.9457746}
lsing
total 986M
-rw-r--r-- 1 dunegli fnalgrid 701M Oct 28 18:26 np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.root
-rw-r--r-- 1 dunegli fnalgrid 264M Oct 28 18:26 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 6.7M Oct 28 18:26 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 6.5M Oct 28 18:26 np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.err
-rw-r--r-- 1 dunegli fnalgrid 276K Oct 28 18:26 mem.db
-rw-r--r-- 1 dunegli fnalgrid  69K Oct 28 18:26 np04hd_raw_run032933_0000_dataflow5_datawriter_0_20241203T231000_1_233774_4_1761665635_deco_reco.log
-rw-r--r-- 1 dunegli fnalgrid  56K Oct 28 18:26 time.db
-rw-r--r-- 1 dunegli fnalgrid  19K Oct 28 18:26 pdhd_keepup_decoder.root
-rw-r--r-- 1 dunegli fnalgrid 3.5K Oct 28 18:26 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid   83 Oct 28 15:33 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid    0 Oct 28 15:34 debugprod.log
disk usage
986M	.
justIN time: 2025-11-05 17:30:04 UTC       justIN version: 01.05.01