justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 14359.4@dunegpschedd02.fnal.gov

Jobsub ID14359.4@dunegpschedd02.fnal.gov
Workflow ID330
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-04 16:22:02
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2025-08-04 22:33:38
From worker nodeHostnameheplnc166.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-04 16:25:28
Input fileshd-protodune:np04hd_raw_run027087_0037_dataflow1_datawriter_0_20240614T150505.hdf5
JobscriptExit code0
Real time6h (22029s)
CPU time5h (21090s = 95%)
Max RSS bytes4580511744 (4368 MiB)
Outputting started2025-08-04 22:32:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00330/1/001/pdhd_processing_1_14359_4_1754324741_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00330/1/001/pdhd_processing_1_14359_4_1754324741_stage1_stage2_tfile.root
Finished2025-08-04 22:33:38
Saved logsjustin-logs:14359.4-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

q=2.49348e+06 n=1025088 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 87983 traces, 87983 summary
Retagger: tagging trace set: gauss with 75721 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75721 traces tagged "gauss"
FrameSaver: q=2.04128e+06 n=994971 tag=gauss
wclsFrameSaver: saving 87983 traces tagged "wiener"
FrameSaver: q=2.34474e+06 n=1046825 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 93185 traces, 93185 summary
Retagger: tagging trace set: gauss with 82079 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 82079 traces tagged "gauss"
FrameSaver: q=2.38376e+06 n=1175682 tag=gauss
wclsFrameSaver: saving 93185 traces tagged "wiener"
FrameSaver: q=2.50752e+06 n=1203556 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 80179 traces, 80179 summary
Retagger: tagging trace set: gauss with 71203 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71203 traces tagged "gauss"
FrameSaver: q=1.96344e+06 n=933017 tag=gauss
wclsFrameSaver: saving 80179 traces tagged "wiener"
FrameSaver: q=2.11915e+06 n=1007172 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 91014 traces, 91014 summary
Retagger: tagging trace set: gauss with 76709 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 76709 traces tagged "gauss"
FrameSaver: q=2.37743e+06 n=1068283 tag=gauss
wclsFrameSaver: saving 91014 traces tagged "wiener"
FrameSaver: q=2.56426e+06 n=1113789 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83820 traces, 83820 summary
Retagger: tagging trace set: gauss with 75202 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75202 traces tagged "gauss"
FrameSaver: q=1.99149e+06 n=1056622 tag=gauss
wclsFrameSaver: saving 83820 traces tagged "wiener"
FrameSaver: q=2.15754e+06 n=1104003 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 90775 traces, 90775 summary
Retagger: tagging trace set: gauss with 80163 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 80163 traces tagged "gauss"
FrameSaver: q=2.40465e+06 n=1174335 tag=gauss
wclsFrameSaver: saving 90775 traces tagged "wiener"
FrameSaver: q=2.53475e+06 n=1199058 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83239 traces, 83239 summary
Retagger: tagging trace set: gauss with 70281 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 70281 traces tagged "gauss"
FrameSaver: q=1.99422e+06 n=958340 tag=gauss
wclsFrameSaver: saving 83239 traces tagged "wiener"
FrameSaver: q=2.23175e+06 n=997254 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 91463 traces, 91463 summary
Retagger: tagging trace set: gauss with 79042 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 79042 traces tagged "gauss"
FrameSaver: q=2.50488e+06 n=1104799 tag=gauss
wclsFrameSaver: saving 91463 traces tagged "wiener"
FrameSaver: q=2.70324e+06 n=1135839 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70039 traces, 70039 summary
Retagger: tagging trace set: gauss with 62995 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62995 traces tagged "gauss"
FrameSaver: q=1.47732e+06 n=816148 tag=gauss
wclsFrameSaver: saving 70039 traces tagged "wiener"
FrameSaver: q=1.58447e+06 n=844960 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 96330 traces, 96330 summary
Retagger: tagging trace set: gauss with 83197 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 83197 traces tagged "gauss"
FrameSaver: q=2.51317e+06 n=1207277 tag=gauss
wclsFrameSaver: saving 96330 traces tagged "wiener"
FrameSaver: q=2.62463e+06 n=1231862 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78420 traces, 78420 summary
Retagger: tagging trace set: gauss with 68759 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68759 traces tagged "gauss"
FrameSaver: q=1.87436e+06 n=1012194 tag=gauss
wclsFrameSaver: saving 78420 traces tagged "wiener"
FrameSaver: q=1.9739e+06 n=1024876 tag=wiener
04-Aug-2025 23:32:34 BST  Closed output file "pdhd_processing_1_14359_4_1754324741_stage1_stage2.root"
04-Aug-2025 23:32:34 BST  Closed input file "pdhd_processing_1_14359_4_1754324741_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              84.1018       514.373       975.165       496.051       190.82         40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000545749   0.00582728     0.0275935    0.00360007    0.00538564       40     
produce:fembfilter:PDHDFEMBFilter                      0.736074       1.05508       1.27154       1.10642      0.137895        40     
produce:wclsdatahd:WireCellToolkit                      39.6888       58.401        75.3362       62.1719       9.31955        40     
produce:gaushit:GausHitFinder                          0.522764       1.35092       5.05378       1.34418      0.692865        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000285003   0.000547592   0.00248018    0.000504207   0.000330097      40     
produce:reco3d:SpacePointSolver                        0.250445       1.39748       4.46764       1.32243       0.9124         39     
produce:hitpdune:DisambigFromSpacePoints               0.214002       1.12364       2.37126       1.14762      0.550792        39     
produce:pandora:StandardPandora                         7.26998       28.3588       60.6598       27.5486       13.6891        39     
produce:pandoraWriter:StandardPandora                  0.107799      0.284089      0.467002      0.284571      0.0897657       39     
produce:pandoraTrack:LArPandoraTrackCreation           0.973572        5.699        18.3202       4.62158       4.2139         39     
produce:pandoraShower:LArPandoraShowerCreation         0.709652       8.04031       27.7409       5.92796       6.71744        39     
produce:pandoracalo:Calorimetry                        0.457187       2.14755       6.61606       1.72012       1.43149        39     
produce:pandoracalonosce:Calorimetry                   0.408086       1.96597       5.54606       1.52195       1.31946        39     
produce:pandorapid:Chi2ParticleID                     0.00141641     0.0043934     0.0109088    0.00339033    0.00254432       39     
produce:pandoraShowercalo:ShowerCalorimetry            0.667466       5.76024       16.5729       5.07436       4.29325        39     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.623223       5.66327       15.2212       4.65738       4.11201        39     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0800047     0.250493      0.414225      0.249228      0.0970337       39     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0310301     0.0876354     0.150268      0.0895358     0.0324331       39     
produce:emtrkmichelid:EmTrackMichelId                   22.8609       42.0326       59.9069       44.2734       9.78436        39     
[art]:TriggerResults:TriggerResultInserter             3.191e-05    8.45284e-05   0.000149659   8.6817e-05    3.12096e-05      40     
end_path:out1:RootOutput                               5.57e-06     2.54944e-05   7.8066e-05    2.5252e-05    1.34116e-05      40     
end_path:michelremoving:michelremoving                7.5542e-05     0.218303      0.792887      0.133804      0.213254        40     
end_path:michelremovinggnocchi:michelremoving          6.432e-05     0.210834      0.773182      0.151215      0.204829        40     
end_path:calibanagnocchi:CalibAnaTree                 6.8127e-05      350.687       728.671       324.985       149.95         40     
end_path:out1:RootOutput(write)                         0.58358       1.4226        2.2178        1.46454      0.371206        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7090.11 MB
  Peak resident set size usage (VmHWM): 4580.51 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_14359_4_1754324741_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_14359_4_1754324741_stage1_stage2_tfile.root', 'start_time': 1754326102.845171, 'end_time': 1754346757.5835187}
justIN time: 2025-08-05 01:19:02 UTC       justIN version: 01.04.00