justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20914.120@dunegpschedd01.fnal.gov

Jobsub ID20914.120@dunegpschedd01.fnal.gov
Workflow ID404
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-05 02:58:41
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-08-05 08:47:10
From worker nodeHostnamemahagon03.farm.particle.cz
cpuinfoAMD EPYC 7301 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-05 03:07:17
Input fileshd-protodune:np04hd_raw_run027087_0043_dataflow0_datawriter_0_20240614T150826.hdf5
JobscriptExit code0
Real time5h (20354s)
CPU time5h (19473s = 95%)
Max RSS bytes4000411648 (3815 MiB)
Outputting started2025-08-05 08:46:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20914_120_1754363261_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20914_120_1754363261_stage1_stage2_tfile.root
Finished2025-08-05 08:47:10
Saved logsjustin-logs:20914.120-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r: q=2.1369e+06 n=1059112 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 84148 traces, 84148 summary
Retagger: tagging trace set: gauss with 71226 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71226 traces tagged "gauss"
FrameSaver: q=2.1148e+06 n=972462 tag=gauss
wclsFrameSaver: saving 84148 traces tagged "wiener"
FrameSaver: q=2.28393e+06 n=1030315 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68207 traces, 68207 summary
Retagger: tagging trace set: gauss with 60747 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60747 traces tagged "gauss"
FrameSaver: q=1.53833e+06 n=816317 tag=gauss
wclsFrameSaver: saving 68207 traces tagged "wiener"
FrameSaver: q=1.58175e+06 n=845776 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 99465 traces, 99465 summary
Retagger: tagging trace set: gauss with 85556 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 85556 traces tagged "gauss"
FrameSaver: q=2.40458e+06 n=1155269 tag=gauss
wclsFrameSaver: saving 99465 traces tagged "wiener"
FrameSaver: q=2.65035e+06 n=1178197 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65063 traces, 65063 summary
Retagger: tagging trace set: gauss with 59187 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59187 traces tagged "gauss"
FrameSaver: q=1.30833e+06 n=763636 tag=gauss
wclsFrameSaver: saving 65063 traces tagged "wiener"
FrameSaver: q=1.36e+06 n=796716 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83778 traces, 83778 summary
Retagger: tagging trace set: gauss with 75681 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75681 traces tagged "gauss"
FrameSaver: q=2.25507e+06 n=1099854 tag=gauss
wclsFrameSaver: saving 83778 traces tagged "wiener"
FrameSaver: q=2.25271e+06 n=1135199 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67835 traces, 67835 summary
Retagger: tagging trace set: gauss with 59877 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59877 traces tagged "gauss"
FrameSaver: q=1.28755e+06 n=747062 tag=gauss
wclsFrameSaver: saving 67835 traces tagged "wiener"
FrameSaver: q=1.43804e+06 n=772786 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83092 traces, 83092 summary
Retagger: tagging trace set: gauss with 70368 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 70368 traces tagged "gauss"
FrameSaver: q=1.91885e+06 n=1060529 tag=gauss
wclsFrameSaver: saving 83092 traces tagged "wiener"
FrameSaver: q=2.13916e+06 n=1077524 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64746 traces, 64746 summary
Retagger: tagging trace set: gauss with 59998 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59998 traces tagged "gauss"
FrameSaver: q=1.37118e+06 n=778830 tag=gauss
wclsFrameSaver: saving 64746 traces tagged "wiener"
FrameSaver: q=1.43673e+06 n=827857 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70784 traces, 70784 summary
Retagger: tagging trace set: gauss with 64385 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 64385 traces tagged "gauss"
FrameSaver: q=1.7307e+06 n=910793 tag=gauss
wclsFrameSaver: saving 70784 traces tagged "wiener"
FrameSaver: q=1.79643e+06 n=939658 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85907 traces, 85907 summary
Retagger: tagging trace set: gauss with 73130 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 73130 traces tagged "gauss"
FrameSaver: q=1.92765e+06 n=995542 tag=gauss
wclsFrameSaver: saving 85907 traces tagged "wiener"
FrameSaver: q=2.11565e+06 n=1015419 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 75456 traces, 75456 summary
Retagger: tagging trace set: gauss with 66977 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66977 traces tagged "gauss"
FrameSaver: q=1.93461e+06 n=944641 tag=gauss
wclsFrameSaver: saving 75456 traces tagged "wiener"
FrameSaver: q=1.96751e+06 n=967091 tag=wiener
05-Aug-2025 10:46:26 CEST  Closed output file "pdhd_processing_1_20914_120_1754363261_stage1_stage2.root"
05-Aug-2025 10:46:26 CEST  Closed input file "pdhd_processing_1_20914_120_1754363261_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              184.808       465.481       1224.26       441.658       180.128        40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000779684   0.00629615     0.0678839    0.00367411     0.011075        40     
produce:fembfilter:PDHDFEMBFilter                      0.782124       1.15046       5.22111       1.0404       0.665205        40     
produce:wclsdatahd:WireCellToolkit                      48.889        59.9349       72.1378       59.9398       6.24098        40     
produce:gaushit:GausHitFinder                          0.642302       1.13922       1.78918       1.11101       0.27471        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000277354   0.000452683   0.00121749    0.000400256   0.000178371      40     
produce:reco3d:SpacePointSolver                         0.32112       1.17832       6.30142      0.835669      0.998597        40     
produce:hitpdune:DisambigFromSpacePoints               0.237848      0.991259       3.60756      0.772443      0.642351        40     
produce:pandora:StandardPandora                         6.18222       36.7111       405.298       24.5495       60.5103        40     
produce:pandoraWriter:StandardPandora                   0.14218      0.267328       0.45871      0.269365      0.0685265       40     
produce:pandoraTrack:LArPandoraTrackCreation            1.10724       5.61314       21.0161       5.24391       3.2129         40     
produce:pandoraShower:LArPandoraShowerCreation         0.695024       7.0737        20.9868       6.05596       4.19982        40     
produce:pandoracalo:Calorimetry                        0.382132       2.0433        6.17311       1.99807      0.969454        40     
produce:pandoracalonosce:Calorimetry                   0.372629       1.94107       5.61368       1.79586      0.913735        40     
produce:pandorapid:Chi2ParticleID                      0.001592     0.00343858    0.00912515    0.00321973    0.00133862       40     
produce:pandoraShowercalo:ShowerCalorimetry            0.574554       5.34227       14.0481       4.84046       2.84156        40     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.542763       5.23688       14.171        4.71009       2.87894        40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0957384     0.223244      0.348712      0.222529      0.0614809       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0343052     0.0819694     0.147796      0.0836456     0.0248361       40     
produce:emtrkmichelid:EmTrackMichelId                   29.3154       57.0957       89.0835       55.5912       14.4099        40     
[art]:TriggerResults:TriggerResultInserter            4.5947e-05    7.20991e-05   0.000115939   7.0508e-05    1.37349e-05      40     
end_path:out1:RootOutput                               1.618e-05    2.52378e-05   4.1148e-05    2.52225e-05   5.93855e-06      40     
end_path:michelremoving:michelremoving                 0.0338509     0.148419      0.546713      0.101477      0.113783        40     
end_path:michelremovinggnocchi:michelremoving          0.0274872     0.146125      0.535634      0.0846074     0.118939        40     
end_path:calibanagnocchi:CalibAnaTree                   79.9244       277.088       574.883       265.875       116.385        40     
end_path:out1:RootOutput(write)                        0.787361       1.41956       3.19022       1.33981      0.464461        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6389.79 MB
  Peak resident set size usage (VmHWM): 4000.41 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_20914_120_1754363261_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_20914_120_1754363261_stage1_stage2_tfile.root', 'start_time': 1754364869.8090587, 'end_time': 1754383591.3465633}
justIN time: 2025-08-05 23:26:13 UTC       justIN version: 01.04.00