justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20917.76@dunegpschedd01.fnal.gov

Jobsub ID20917.76@dunegpschedd01.fnal.gov
Workflow ID404
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-05 03:22:43
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-08-05 08:41:10
From worker nodeHostnamemilan15.farm.particle.cz
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-05 04:52:49
Input fileshd-protodune:np04hd_raw_run027087_0266_dataflow1_datawriter_0_20240614T170759.hdf5
JobscriptExit code0
Real time3h (13668s)
CPU time3h (13170s = 96%)
Max RSS bytes7806763008 (7445 MiB)
Outputting started2025-08-05 08:40:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20917_76_1754369580_stage1_stage2_tfile.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20917_76_1754369580_stage1_stage2.root
Finished2025-08-05 08:41:10
Saved logsjustin-logs:20917.76-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Saver: q=1.57821e+06 n=765715 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 62137 traces, 62137 summary
Retagger: tagging trace set: gauss with 57176 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57176 traces tagged "gauss"
FrameSaver: q=1.10808e+06 n=710358 tag=gauss
wclsFrameSaver: saving 62137 traces tagged "wiener"
FrameSaver: q=1.16098e+06 n=736759 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82742 traces, 82742 summary
Retagger: tagging trace set: gauss with 71134 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71134 traces tagged "gauss"
FrameSaver: q=1.93148e+06 n=992133 tag=gauss
wclsFrameSaver: saving 82742 traces tagged "wiener"
FrameSaver: q=2.1244e+06 n=998494 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 62554 traces, 62554 summary
Retagger: tagging trace set: gauss with 56097 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56097 traces tagged "gauss"
FrameSaver: q=1.27775e+06 n=752230 tag=gauss
wclsFrameSaver: saving 62554 traces tagged "wiener"
FrameSaver: q=1.33365e+06 n=764952 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82172 traces, 82172 summary
Retagger: tagging trace set: gauss with 73309 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 73309 traces tagged "gauss"
FrameSaver: q=2.25206e+06 n=1016503 tag=gauss
wclsFrameSaver: saving 82172 traces tagged "wiener"
FrameSaver: q=2.6247e+06 n=1070228 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67575 traces, 67575 summary
Retagger: tagging trace set: gauss with 63310 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 63310 traces tagged "gauss"
FrameSaver: q=1.61547e+06 n=937098 tag=gauss
wclsFrameSaver: saving 67575 traces tagged "wiener"
FrameSaver: q=1.57706e+06 n=945366 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 94708 traces, 94708 summary
Retagger: tagging trace set: gauss with 82900 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 82900 traces tagged "gauss"
FrameSaver: q=2.54535e+06 n=1182214 tag=gauss
wclsFrameSaver: saving 94708 traces tagged "wiener"
FrameSaver: q=2.73343e+06 n=1212602 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85861 traces, 85861 summary
Retagger: tagging trace set: gauss with 71351 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71351 traces tagged "gauss"
FrameSaver: q=1.94195e+06 n=984850 tag=gauss
wclsFrameSaver: saving 85861 traces tagged "wiener"
FrameSaver: q=2.04077e+06 n=974476 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 107416 traces, 107416 summary
Retagger: tagging trace set: gauss with 89799 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 89799 traces tagged "gauss"
FrameSaver: q=2.62059e+06 n=1204651 tag=gauss
wclsFrameSaver: saving 107416 traces tagged "wiener"
FrameSaver: q=3.0117e+06 n=1264505 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 76661 traces, 76661 summary
Retagger: tagging trace set: gauss with 67412 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67412 traces tagged "gauss"
FrameSaver: q=2.0157e+06 n=902201 tag=gauss
wclsFrameSaver: saving 76661 traces tagged "wiener"
FrameSaver: q=2.17149e+06 n=937369 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63803 traces, 63803 summary
Retagger: tagging trace set: gauss with 56350 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56350 traces tagged "gauss"
FrameSaver: q=1.30985e+06 n=710373 tag=gauss
wclsFrameSaver: saving 63803 traces tagged "wiener"
FrameSaver: q=1.40779e+06 n=738958 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 86765 traces, 86765 summary
Retagger: tagging trace set: gauss with 74112 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 74112 traces tagged "gauss"
FrameSaver: q=2.09476e+06 n=978905 tag=gauss
wclsFrameSaver: saving 86765 traces tagged "wiener"
FrameSaver: q=2.3141e+06 n=1039127 tag=wiener
05-Aug-2025 10:40:36 CEST  Closed output file "pdhd_processing_1_20917_76_1754369580_stage1_stage2.root"
05-Aug-2025 10:40:36 CEST  Closed input file "pdhd_processing_1_20917_76_1754369580_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              61.4583       311.273       609.849       296.293       110.372        40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000631958   0.00331668    0.00909453    0.00280886    0.00203566       40     
produce:fembfilter:PDHDFEMBFilter                      0.560234      0.747669       1.04508      0.716081      0.0905839       40     
produce:wclsdatahd:WireCellToolkit                      34.0491       41.6349       58.4828       39.3185       5.69475        40     
produce:gaushit:GausHitFinder                          0.316822      0.730692       1.72193      0.678483      0.269663        40     
produce:nhitsfilter:NumberOfHitsFilter                0.00024801    0.000627743   0.00393605    0.000436199   0.00074188       40     
produce:reco3d:SpacePointSolver                        0.113299      0.724256       2.52344      0.566751      0.533128        39     
produce:hitpdune:DisambigFromSpacePoints               0.129088      0.539855       1.41349      0.463803       0.33421        39     
produce:pandora:StandardPandora                         4.34978       14.671        40.2943       12.3441       7.61921        39     
produce:pandoraWriter:StandardPandora                  0.101684      0.201172      0.393958      0.182066      0.0702475       39     
produce:pandoraTrack:LArPandoraTrackCreation           0.755743       2.28755       6.54539       2.06213       1.19891        39     
produce:pandoraShower:LArPandoraShowerCreation         0.724838       2.71941       8.37124       2.19244       1.66664        39     
produce:pandoracalo:Calorimetry                         0.31415      0.912329       2.43748      0.888468      0.411987        39     
produce:pandoracalonosce:Calorimetry                   0.287294       0.85714       2.29527      0.801124      0.396565        39     
produce:pandorapid:Chi2ParticleID                      0.0012919    0.00252567    0.00612328    0.00219298    0.00108817       39     
produce:pandoraShowercalo:ShowerCalorimetry            0.526341       2.19023       6.93253       1.89046       1.27867        39     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.558492       2.11421       7.01352       1.78897       1.27284        39     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0640162     0.125363       0.23909      0.113259      0.0386962       39     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0238295     0.0479626     0.176709      0.0417009     0.0244857       39     
produce:emtrkmichelid:EmTrackMichelId                   13.8645       26.8809       43.3293       25.8155       6.96237        39     
[art]:TriggerResults:TriggerResultInserter             1.855e-05    5.34779e-05   0.000142089    5.002e-05    2.83351e-05      40     
end_path:out1:RootOutput                               7.45e-06     2.1604e-05     5.076e-05    1.8175e-05    1.14156e-05      40     
end_path:michelremoving:michelremoving                 3.887e-05     0.0971258     0.376618      0.0568173     0.0895394       40     
end_path:michelremovinggnocchi:michelremoving          2.398e-05     0.0955536     0.357653      0.0466696     0.095283        40     
end_path:calibanagnocchi:CalibAnaTree                  1.346e-05      213.656       486.244       198.297       94.0915        40     
end_path:out1:RootOutput(write)                         0.49312      0.877659       1.53155      0.862287      0.230587        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10372.4 MB
  Peak resident set size usage (VmHWM): 7806.76 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_20917_76_1754369580_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_20917_76_1754369580_stage1_stage2_tfile.root', 'start_time': 1754370732.545736, 'end_time': 1754383238.641268}
justIN time: 2025-08-05 23:17:47 UTC       justIN version: 01.04.00