justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294936.1@dunegpschedd02.fnal.gov

Jobsub ID294936.1@dunegpschedd02.fnal.gov
Workflow ID12243
Stage ID1
User namechasnip@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-25 11:08:45
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2026-01-26 09:03:58
From worker nodeHostnameb9p07p9496.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-25 11:09:38
Input fileshd-protodune:np04hd_raw_run029917_0149_dataflow1_datawriter_0_20241011T181724.hdf5
JobscriptExit code0
Real time21h (78789s)
CPU time21h (77105s = 97%)
Max RSS bytes3716538368 (3544 MiB)
Outputting started2026-01-26 09:02:48
Output filesusertests:proc_bsmtrigger_protodunehd_1_294936_1_1769339389_reco.root
Finished2026-01-26 09:03:58
Saved logsjustin-logs:294936.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

wclsFrameSaver: no samples within desired window for channel 9731
wclsFrameSaver: no samples within desired window for channel 9731
wclsFrameSaver: no samples within desired window for channel 9731
wclsFrameSaver: no samples within desired window for channel 9732
wclsFrameSaver: no samples within desired window for channel 9732
wclsFrameSaver: no samples within desired window for channel 9732
wclsFrameSaver: no samples within desired window for channel 9733
wclsFrameSaver: no samples within desired window for channel 9733
wclsFrameSaver: no samples within desired window for channel 9733
wclsFrameSaver: no samples within desired window for channel 9733
wclsFrameSaver: no samples within desired window for channel 9733
wclsFrameSaver: no samples within desired window for channel 9734
wclsFrameSaver: no samples within desired window for channel 9734
wclsFrameSaver: no samples within desired window for channel 9735
wclsFrameSaver: no samples within desired window for channel 9736
wclsFrameSaver: no samples within desired window for channel 9737
wclsFrameSaver: no samples within desired window for channel 9738
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9740
wclsFrameSaver: no samples within desired window for channel 9741
wclsFrameSaver: no samples within desired window for channel 9742
wclsFrameSaver: no samples within desired window for channel 9743
wclsFrameSaver: no samples within desired window for channel 9743
wclsFrameSaver: no samples within desired window for channel 9744
wclsFrameSaver: no samples within desired window for channel 9749
wclsFrameSaver: no samples within desired window for channel 9752
wclsFrameSaver: no samples within desired window for channel 9752
wclsFrameSaver: no samples within desired window for channel 9756
wclsFrameSaver: no samples within desired window for channel 9759
wclsFrameSaver: no samples within desired window for channel 9788
wclsFrameSaver: no samples within desired window for channel 9850
wclsFrameSaver: no samples within desired window for channel 9850
wclsFrameSaver: no samples within desired window for channel 9851
wclsFrameSaver: no samples within desired window for channel 9852
wclsFrameSaver: no samples within desired window for channel 10047
wclsFrameSaver: no samples within desired window for channel 10047
wclsFrameSaver: no samples within desired window for channel 10048
wclsFrameSaver: no samples within desired window for channel 10048
wclsFrameSaver: no samples within desired window for channel 10049
wclsFrameSaver: no samples within desired window for channel 10049
wclsFrameSaver: no samples within desired window for channel 10050
wclsFrameSaver: no samples within desired window for channel 10050
wclsFrameSaver: no samples within desired window for channel 10192
wclsFrameSaver: no samples within desired window for channel 10232
wclsFrameSaver: no samples within desired window for channel 10233
wclsFrameSaver: no samples within desired window for channel 10233
wclsFrameSaver: no samples within desired window for channel 10234
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10235
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10236
wclsFrameSaver: no samples within desired window for channel 10237
wclsFrameSaver: no samples within desired window for channel 10237
wclsFrameSaver: no samples within desired window for channel 10237
wclsFrameSaver: no samples within desired window for channel 10237
wclsFrameSaver: no samples within desired window for channel 10238
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
FrameSaver: q=3.83263e+06 n=1476445 tag=wiener
26-Jan-2026 10:02:41 CET  Closed output file "proc_bsmtrigger_protodunehd_1_294936_1_1769339389_reco.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            151.089       3936.66        53926        349.844       11756.7        20     
--------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                          4.9371e-05    0.000109018   0.00019223    9.9056e-05    4.28759e-05      20     
produce:spillflag:PDHDSPSSpillFlagProducer           2.627e-05    6.32018e-05   0.000173881   4.89305e-05   3.84144e-05      20     
produce:triggerrawdecoder:PDHDTriggerReader3         0.0450171     0.0989367     0.154533      0.101496      0.0296326       20     
produce:triggertypefilter:PDHDTriggerTypeFilter      3.024e-05    7.26855e-05   0.000293253   5.58305e-05   5.42595e-05      20     
produce:tpcrawdecoder:PDHDTPCReader                   5.03201       6.43677       15.7556       5.80192       2.42786        20     
produce:timingrawdecoder:PDHDTimingRawDecoder       0.000576384   0.000804304   0.00114627    0.000792694   0.00013413       20     
produce:fembfilter:PDHDFEMBFilter                   5.8411e-05    0.000104892   0.000357353   9.1125e-05    6.32728e-05      20     
produce:wclsdatahd:WireCellToolkit                    100.447       117.52        131.33        119.184       10.661         20     
produce:gaushit:GausHitFinder                        0.631506       1.23098       2.83246       1.17936      0.498218        20     
produce:reco3d:SpacePointSolver                       1.68626       25.5232       120.975       5.86691       37.3123        20     
produce:hitpdune:DisambigFromSpacePoints             0.854366       3.65826       12.6978       2.64045       3.20581        20     
produce:nhitsfilter:NumberOfHitsFilter              0.000315681   0.000546314    0.001241     0.000494513   0.000223798      20     
produce:pandora:StandardPandora                       22.2992       3754.31       53565.1       190.723       11709.1        20     
produce:pandoraWriter:StandardPandora                0.157241      0.284681      0.567118      0.246508      0.110946        20     
produce:pandoraTrack:LArPandoraTrackCreation          1.60311       5.48938       26.8283       3.39206       5.68208        20     
produce:pandoraShower:LArPandoraShowerCreation        1.36751       4.32376       14.921        3.53374       3.01738        20     
produce:pandoracalo:Calorimetry                      0.770427       2.50363       9.15109       1.7046        1.96778        20     
produce:pandoracalonosce:Calorimetry                 0.720837       2.48892       9.5858        1.83158       2.0439         20     
produce:pandorapid:Chi2ParticleID                   0.00198135    0.00296107    0.00474535    0.00280383    0.000765053      20     
produce:pandoraShowercalo:ShowerCalorimetry           1.14942       3.83595       9.96597       3.38318       2.14909        20     
produce:pandoraShowercalonosce:ShowerCalorimetry      1.17425       3.68073       9.35706       3.42094       1.95251        20     
[art]:TriggerResults:TriggerResultInserter           2.152e-05    3.51353e-05   0.000142282    2.971e-05    2.52667e-05      20     
end_path:out1:RootOutput                             1.591e-05    4.02493e-05   0.000356373   2.3375e-05    7.2654e-05       20     
end_path:out1:RootOutput(write)                       3.66881       5.00193       6.9422        4.86268      0.839315        20     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5632.74 MB
  Peak resident set size usage (VmHWM): 3716.54 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'proc_bsmtrigger_protodunehd_1_294936_1_1769339389_reco.root', 'start_time': 1769339389.5628793, 'end_time': 1769418163.077801}
justIN time: 2026-02-05 09:54:02 UTC       justIN version: 01.06.00