justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20914.152@dunegpschedd01.fnal.gov

Jobsub ID20914.152@dunegpschedd01.fnal.gov
Workflow ID404
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-05 02:58:41
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-08-05 08:45:54
From worker nodeHostnamewn2204240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-05 03:12:26
Input fileshd-protodune:np04hd_raw_run027087_0061_dataflow0_datawriter_0_20240614T151802.hdf5
JobscriptExit code0
Real time5h (19954s)
CPU time5h (19117s = 95%)
Max RSS bytes3405385728 (3247 MiB)
Outputting started2025-08-05 08:45:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20914_152_1754363561_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20914_152_1754363561_stage1_stage2_tfile.root
Finished2025-08-05 08:45:54
Saved logsjustin-logs:20914.152-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Saver: q=1.81276e+06 n=867016 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74286 traces, 74286 summary
Retagger: tagging trace set: gauss with 66070 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66070 traces tagged "gauss"
FrameSaver: q=1.73835e+06 n=848076 tag=gauss
wclsFrameSaver: saving 74286 traces tagged "wiener"
FrameSaver: q=1.82582e+06 n=887301 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 84683 traces, 84683 summary
Retagger: tagging trace set: gauss with 72896 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 72896 traces tagged "gauss"
FrameSaver: q=2.11895e+06 n=1005796 tag=gauss
wclsFrameSaver: saving 84683 traces tagged "wiener"
FrameSaver: q=2.28591e+06 n=1057431 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 75808 traces, 75808 summary
Retagger: tagging trace set: gauss with 64118 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 64118 traces tagged "gauss"
FrameSaver: q=1.71638e+06 n=892470 tag=gauss
wclsFrameSaver: saving 75808 traces tagged "wiener"
FrameSaver: q=1.83364e+06 n=895201 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79152 traces, 79152 summary
Retagger: tagging trace set: gauss with 66160 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66160 traces tagged "gauss"
FrameSaver: q=1.63804e+06 n=861049 tag=gauss
wclsFrameSaver: saving 79152 traces tagged "wiener"
FrameSaver: q=1.86233e+06 n=894196 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78808 traces, 78808 summary
Retagger: tagging trace set: gauss with 68523 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68523 traces tagged "gauss"
FrameSaver: q=1.89871e+06 n=926728 tag=gauss
wclsFrameSaver: saving 78808 traces tagged "wiener"
FrameSaver: q=1.99813e+06 n=969410 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 87496 traces, 87496 summary
Retagger: tagging trace set: gauss with 75277 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75277 traces tagged "gauss"
FrameSaver: q=2.22263e+06 n=1080421 tag=gauss
wclsFrameSaver: saving 87496 traces tagged "wiener"
FrameSaver: q=2.37668e+06 n=1106859 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 96393 traces, 96393 summary
Retagger: tagging trace set: gauss with 81715 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 81715 traces tagged "gauss"
FrameSaver: q=2.29821e+06 n=1106343 tag=gauss
wclsFrameSaver: saving 96393 traces tagged "wiener"
FrameSaver: q=2.50744e+06 n=1146120 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54435 traces, 54435 summary
Retagger: tagging trace set: gauss with 50198 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50198 traces tagged "gauss"
FrameSaver: q=995081 n=651805 tag=gauss
wclsFrameSaver: saving 54435 traces tagged "wiener"
FrameSaver: q=1.01309e+06 n=671246 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71851 traces, 71851 summary
Retagger: tagging trace set: gauss with 61486 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 61486 traces tagged "gauss"
FrameSaver: q=1.41194e+06 n=845318 tag=gauss
wclsFrameSaver: saving 71851 traces tagged "wiener"
FrameSaver: q=1.54106e+06 n=826920 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60942 traces, 60942 summary
Retagger: tagging trace set: gauss with 56080 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56080 traces tagged "gauss"
FrameSaver: q=1.18277e+06 n=714827 tag=gauss
wclsFrameSaver: saving 60942 traces tagged "wiener"
FrameSaver: q=1.20147e+06 n=739897 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82704 traces, 82704 summary
Retagger: tagging trace set: gauss with 71243 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71243 traces tagged "gauss"
FrameSaver: q=2.05991e+06 n=970281 tag=gauss
wclsFrameSaver: saving 82704 traces tagged "wiener"
FrameSaver: q=2.36691e+06 n=1025684 tag=wiener
05-Aug-2025 09:44:59 BST  Closed output file "pdhd_processing_1_20914_152_1754363561_stage1_stage2.root"
05-Aug-2025 09:44:59 BST  Closed input file "pdhd_processing_1_20914_152_1754363561_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              159.629       464.221       826.294       437.412       156.248        40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00079232     0.0049392     0.0116413    0.00524317     0.0028649       40     
produce:fembfilter:PDHDFEMBFilter                       0.64141      0.982107       1.2719        0.99964      0.129178        40     
produce:wclsdatahd:WireCellToolkit                      39.6154       54.7101       83.672        51.0523       9.60442        40     
produce:gaushit:GausHitFinder                          0.374698       1.08499       2.01199       1.05056      0.364089        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000304733   0.000656684   0.00411128    0.000434316   0.000638722      40     
produce:reco3d:SpacePointSolver                        0.157523       1.22602       6.92435      0.828306       1.2022         40     
produce:hitpdune:DisambigFromSpacePoints               0.165733       0.92448       3.70542      0.810643      0.643648        40     
produce:pandora:StandardPandora                         6.09705       22.6504       111.587       18.7382       16.3617        40     
produce:pandoraWriter:StandardPandora                   0.09639      0.272706       0.57013       0.24633      0.112781        40     
produce:pandoraTrack:LArPandoraTrackCreation           0.788713       6.13626       22.1628       4.75952       5.05508        40     
produce:pandoraShower:LArPandoraShowerCreation         0.671634       7.35239       34.6004       4.73643       7.00397        40     
produce:pandoracalo:Calorimetry                        0.304325       2.15837       8.39592       1.55543       1.72545        40     
produce:pandoracalonosce:Calorimetry                   0.258465       2.03389       9.58555       1.42084       1.76914        40     
produce:pandorapid:Chi2ParticleID                     0.00134594    0.00369761    0.00756465    0.00324535    0.00143169       40     
produce:pandoraShowercalo:ShowerCalorimetry            0.537558       5.77944       30.0775       3.9285        5.6605         40     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.590602       5.37449       28.5917       3.44112       5.31234        40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0591402     0.218411       0.40454      0.216874      0.0767228       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0318554     0.0844673     0.249557      0.0780337     0.0417777       40     
produce:emtrkmichelid:EmTrackMichelId                   20.7684       37.4843       67.9442       35.1046       11.1735        40     
[art]:TriggerResults:TriggerResultInserter            3.8632e-05    9.71219e-05   0.000201368    8.781e-05    3.51981e-05      40     
end_path:out1:RootOutput                               1.044e-05    2.73647e-05   8.9087e-05    2.3114e-05    1.56975e-05      40     
end_path:michelremoving:michelremoving                 0.0373301     0.191611      0.843053      0.153254      0.158151        40     
end_path:michelremovinggnocchi:michelremoving          0.0349595      0.19354      0.950785      0.143629      0.187524        40     
end_path:calibanagnocchi:CalibAnaTree                   84.9558       313.393       684.233       284.148       124.788        40     
end_path:out1:RootOutput(write)                        0.451504       1.15295       1.91227       1.14398      0.303122        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5841.2 MB
  Peak resident set size usage (VmHWM): 3405.39 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_20914_152_1754363561_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_20914_152_1754363561_stage1_stage2_tfile.root', 'start_time': 1754364843.6313722, 'end_time': 1754383501.1930296}
justIN time: 2025-08-05 23:17:28 UTC       justIN version: 01.04.00