justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302846.3@dunegpschedd01.fnal.gov

Jobsub ID302846.3@dunegpschedd01.fnal.gov
Workflow ID12331
Stage ID1
User namechasnip@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-02-02 15:07:52
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2026-02-03 11:50:26
From worker nodeHostnameb9p35p0059.cern.ch
cpuinfoAMD EPYC 9655P 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-02 15:46:27
Input fileshd-protodune:np04hd_raw_run029917_0225_dataflow1_datawriter_0_20241011T193601.hdf5
JobscriptExit code0
Real time20h (72172s)
CPU time19h (71305s = 98%)
Max RSS bytes3766513664 (3592 MiB)
Outputting started2026-02-03 11:49:21
Output filesusertests:proc_bsmtrigger_protodunehd_1_302846_3_1770047211_reco.root
Finished2026-02-03 11:50:26
Saved logsjustin-logs:302846.3-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

sired window for channel 10186
wclsFrameSaver: no samples within desired window for channel 10186
wclsFrameSaver: no samples within desired window for channel 10187
wclsFrameSaver: no samples within desired window for channel 10187
wclsFrameSaver: no samples within desired window for channel 10188
wclsFrameSaver: no samples within desired window for channel 10188
wclsFrameSaver: no samples within desired window for channel 10189
wclsFrameSaver: no samples within desired window for channel 10190
wclsFrameSaver: no samples within desired window for channel 10191
wclsFrameSaver: no samples within desired window for channel 10191
wclsFrameSaver: no samples within desired window for channel 10192
wclsFrameSaver: no samples within desired window for channel 10192
wclsFrameSaver: no samples within desired window for channel 10193
wclsFrameSaver: no samples within desired window for channel 10194
wclsFrameSaver: no samples within desired window for channel 10194
wclsFrameSaver: no samples within desired window for channel 10195
wclsFrameSaver: no samples within desired window for channel 10196
wclsFrameSaver: no samples within desired window for channel 10196
wclsFrameSaver: no samples within desired window for channel 10197
wclsFrameSaver: no samples within desired window for channel 10197
wclsFrameSaver: no samples within desired window for channel 10198
wclsFrameSaver: no samples within desired window for channel 10198
wclsFrameSaver: no samples within desired window for channel 10199
wclsFrameSaver: no samples within desired window for channel 10199
wclsFrameSaver: no samples within desired window for channel 10200
wclsFrameSaver: no samples within desired window for channel 10200
wclsFrameSaver: no samples within desired window for channel 10201
wclsFrameSaver: no samples within desired window for channel 10201
wclsFrameSaver: no samples within desired window for channel 10202
wclsFrameSaver: no samples within desired window for channel 10202
wclsFrameSaver: no samples within desired window for channel 10203
wclsFrameSaver: no samples within desired window for channel 10203
wclsFrameSaver: no samples within desired window for channel 10204
wclsFrameSaver: no samples within desired window for channel 10204
wclsFrameSaver: no samples within desired window for channel 10205
wclsFrameSaver: no samples within desired window for channel 10205
wclsFrameSaver: no samples within desired window for channel 10206
wclsFrameSaver: no samples within desired window for channel 10206
wclsFrameSaver: no samples within desired window for channel 10207
wclsFrameSaver: no samples within desired window for channel 10207
wclsFrameSaver: no samples within desired window for channel 10207
wclsFrameSaver: no samples within desired window for channel 10208
wclsFrameSaver: no samples within desired window for channel 10208
wclsFrameSaver: no samples within desired window for channel 10208
wclsFrameSaver: no samples within desired window for channel 10209
wclsFrameSaver: no samples within desired window for channel 10209
wclsFrameSaver: no samples within desired window for channel 10210
wclsFrameSaver: no samples within desired window for channel 10210
wclsFrameSaver: no samples within desired window for channel 10211
wclsFrameSaver: no samples within desired window for channel 10211
wclsFrameSaver: no samples within desired window for channel 10212
wclsFrameSaver: no samples within desired window for channel 10213
wclsFrameSaver: no samples within desired window for channel 10213
wclsFrameSaver: no samples within desired window for channel 10214
wclsFrameSaver: no samples within desired window for channel 10215
wclsFrameSaver: no samples within desired window for channel 10216
wclsFrameSaver: no samples within desired window for channel 10217
wclsFrameSaver: no samples within desired window for channel 10218
wclsFrameSaver: no samples within desired window for channel 10219
wclsFrameSaver: no samples within desired window for channel 10220
wclsFrameSaver: no samples within desired window for channel 10221
wclsFrameSaver: no samples within desired window for channel 10222
wclsFrameSaver: no samples within desired window for channel 10223
wclsFrameSaver: no samples within desired window for channel 10224
wclsFrameSaver: no samples within desired window for channel 10225
wclsFrameSaver: no samples within desired window for channel 10226
wclsFrameSaver: no samples within desired window for channel 10227
wclsFrameSaver: no samples within desired window for channel 10228
wclsFrameSaver: no samples within desired window for channel 10229
wclsFrameSaver: no samples within desired window for channel 10230
wclsFrameSaver: no samples within desired window for channel 10231
wclsFrameSaver: no samples within desired window for channel 10232
wclsFrameSaver: no samples within desired window for channel 10233
wclsFrameSaver: no samples within desired window for channel 10234
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10237
wclsFrameSaver: no samples within desired window for channel 10238
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
FrameSaver: q=4.85242e+06 n=1670729 tag=wiener
03-Feb-2026 12:49:10 CET  Closed output file "proc_bsmtrigger_protodunehd_1_302846_3_1770047211_reco.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            197.913       3601.33       30848.4       566.952       7792.11        20     
--------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                           5.279e-05    7.13649e-05   0.000150661    6.851e-05    2.0239e-05       20     
produce:spillflag:PDHDSPSSpillFlagProducer           3.537e-05     6.28e-05     0.000181432   5.6951e-05    2.9945e-05       20     
produce:triggerrawdecoder:PDHDTriggerReader3         0.0661147      0.10446      0.150131      0.101145      0.0211247       20     
produce:triggertypefilter:PDHDTriggerTypeFilter      4.322e-05    7.36048e-05   0.000325903   6.1336e-05    5.83911e-05      20     
produce:tpcrawdecoder:PDHDTPCReader                   3.48798       4.82635       7.64586       4.89192      0.829104        20     
produce:timingrawdecoder:PDHDTimingRawDecoder       0.000578884   0.000777939   0.00129257    0.000743165   0.000168482      20     
produce:fembfilter:PDHDFEMBFilter                   6.0511e-05    9.46083e-05   0.000329842   8.0621e-05    5.52923e-05      20     
produce:wclsdatahd:WireCellToolkit                    96.7356       123.277       164.561       118.073       19.1148        20     
produce:gaushit:GausHitFinder                         0.73324       1.15101       2.01831       1.01403      0.362352        20     
produce:reco3d:SpacePointSolver                       1.62298       26.1559       145.081       5.07909       43.6242        20     
produce:hitpdune:DisambigFromSpacePoints              1.04692       3.96538       16.8954       2.34497       4.41834        20     
produce:nhitsfilter:NumberOfHitsFilter              0.000247962   0.000339896   0.000558973   0.000328542   8.05631e-05      20     
produce:pandora:StandardPandora                       57.929        3408.01       30529.4       376.017       7750.22        20     
produce:pandoraWriter:StandardPandora                0.263099      0.537092       1.21399      0.485579      0.251863        20     
produce:pandoraTrack:LArPandoraTrackCreation          2.47943       7.62116       32.0764       5.56399       6.22298        20     
produce:pandoraShower:LArPandoraShowerCreation        1.95078       5.76347       15.6421       4.68741       3.06007        20     
produce:pandoracalo:Calorimetry                       1.0458        2.83294       10.098        2.26154       1.82373        20     
produce:pandoracalonosce:Calorimetry                 0.951917       2.61726       7.71155       2.03317       1.41844        20     
produce:pandorapid:Chi2ParticleID                   0.00178181    0.00335192    0.00699237     0.0033074    0.00107614       20     
produce:pandoraShowercalo:ShowerCalorimetry           1.33414       4.09436       7.3976        4.13831       1.4997         20     
produce:pandoraShowercalonosce:ShowerCalorimetry      1.28121       4.01142       8.30507       4.00211       1.57057        20     
[art]:TriggerResults:TriggerResultInserter           3.249e-05    4.85739e-05    9.413e-05    4.58155e-05   1.31185e-05      20     
end_path:out1:RootOutput                             1.973e-05    4.61922e-05   0.000287902   3.32305e-05   5.57924e-05      20     
end_path:out1:RootOutput(write)                       3.70793       5.79703       7.71037       5.86174       1.14217        20     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5719.29 MB
  Peak resident set size usage (VmHWM): 3766.51 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'proc_bsmtrigger_protodunehd_1_302846_3_1770047211_reco.root', 'start_time': 1770047211.2817903, 'end_time': 1770119350.7068942}
justIN time: 2026-02-04 11:47:27 UTC       justIN version: 01.06.00