justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20916.142@dunegpschedd01.fnal.gov

Jobsub ID20916.142@dunegpschedd01.fnal.gov
Workflow ID404
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-05 03:20:43
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-08-05 08:45:45
From worker nodeHostnameheplnc148.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-05 03:59:56
Input fileshd-protodune:np04hd_raw_run027087_0116_dataflow2_datawriter_0_20240614T154745.hdf5
JobscriptExit code0
Real time4h (17097s)
CPU time4h (16421s = 96%)
Max RSS bytes3640324096 (3471 MiB)
Outputting started2025-08-05 08:44:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20916_142_1754366408_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20916_142_1754366408_stage1_stage2_tfile.root
Finished2025-08-05 08:45:45
Saved logsjustin-logs:20916.142-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

2.3572e+06 n=1067220 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68800 traces, 68800 summary
Retagger: tagging trace set: gauss with 59847 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59847 traces tagged "gauss"
FrameSaver: q=1.58845e+06 n=851816 tag=gauss
wclsFrameSaver: saving 68800 traces tagged "wiener"
FrameSaver: q=1.61139e+06 n=847960 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69440 traces, 69440 summary
Retagger: tagging trace set: gauss with 58582 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58582 traces tagged "gauss"
FrameSaver: q=1.33508e+06 n=724480 tag=gauss
wclsFrameSaver: saving 69440 traces tagged "wiener"
FrameSaver: q=1.57554e+06 n=760372 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83876 traces, 83876 summary
Retagger: tagging trace set: gauss with 73519 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 73519 traces tagged "gauss"
FrameSaver: q=2.00024e+06 n=1040180 tag=gauss
wclsFrameSaver: saving 83876 traces tagged "wiener"
FrameSaver: q=2.10193e+06 n=1076595 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78939 traces, 78939 summary
Retagger: tagging trace set: gauss with 66885 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66885 traces tagged "gauss"
FrameSaver: q=1.69548e+06 n=900661 tag=gauss
wclsFrameSaver: saving 78939 traces tagged "wiener"
FrameSaver: q=1.90229e+06 n=920331 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83291 traces, 83291 summary
Retagger: tagging trace set: gauss with 74632 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 74632 traces tagged "gauss"
FrameSaver: q=2.21618e+06 n=1025369 tag=gauss
wclsFrameSaver: saving 83291 traces tagged "wiener"
FrameSaver: q=2.28816e+06 n=1074338 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78285 traces, 78285 summary
Retagger: tagging trace set: gauss with 69582 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 69582 traces tagged "gauss"
FrameSaver: q=1.92209e+06 n=960974 tag=gauss
wclsFrameSaver: saving 78285 traces tagged "wiener"
FrameSaver: q=1.96339e+06 n=991681 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 107861 traces, 107861 summary
Retagger: tagging trace set: gauss with 91652 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 91652 traces tagged "gauss"
FrameSaver: q=2.82131e+06 n=1325855 tag=gauss
wclsFrameSaver: saving 107861 traces tagged "wiener"
FrameSaver: q=2.9221e+06 n=1328026 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70127 traces, 70127 summary
Retagger: tagging trace set: gauss with 62157 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62157 traces tagged "gauss"
FrameSaver: q=1.29712e+06 n=762503 tag=gauss
wclsFrameSaver: saving 70127 traces tagged "wiener"
FrameSaver: q=1.48924e+06 n=803488 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79328 traces, 79328 summary
Retagger: tagging trace set: gauss with 69420 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 69420 traces tagged "gauss"
FrameSaver: q=1.86546e+06 n=1029741 tag=gauss
wclsFrameSaver: saving 79328 traces tagged "wiener"
FrameSaver: q=2.05332e+06 n=1032104 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74733 traces, 74733 summary
Retagger: tagging trace set: gauss with 66624 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66624 traces tagged "gauss"
FrameSaver: q=1.65582e+06 n=952133 tag=gauss
wclsFrameSaver: saving 74733 traces tagged "wiener"
FrameSaver: q=1.72237e+06 n=957078 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 89132 traces, 89132 summary
Retagger: tagging trace set: gauss with 75050 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75050 traces tagged "gauss"
FrameSaver: q=2.15263e+06 n=1004805 tag=gauss
wclsFrameSaver: saving 89132 traces tagged "wiener"
FrameSaver: q=2.31026e+06 n=1049599 tag=wiener
05-Aug-2025 09:44:53 BST  Closed output file "pdhd_processing_1_20916_142_1754366408_stage1_stage2.root"
05-Aug-2025 09:44:53 BST  Closed input file "pdhd_processing_1_20916_142_1754366408_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              183.605       394.979       629.465       382.57        100.03         40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00120324    0.00409591     0.0107048    0.00317892    0.00226276       40     
produce:fembfilter:PDHDFEMBFilter                      0.628162      0.843238      0.896125      0.859075      0.0521396       40     
produce:wclsdatahd:WireCellToolkit                      43.0614       51.5125       56.5783       52.0065       2.88158        40     
produce:gaushit:GausHitFinder                          0.561069      0.933539       1.27878      0.940009      0.181355        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000326446   0.000485213   0.000854683   0.000464551   0.000100631      40     
produce:reco3d:SpacePointSolver                        0.224708       1.06956       3.88666      0.914136      0.728772        40     
produce:hitpdune:DisambigFromSpacePoints               0.203647      0.727446       1.70016      0.689755      0.285587        40     
produce:pandora:StandardPandora                         10.3689       21.017        51.7234       19.3534       9.30577        40     
produce:pandoraWriter:StandardPandora                  0.117967      0.250147       0.36388      0.250084      0.0443124       40     
produce:pandoraTrack:LArPandoraTrackCreation           0.955755       5.21323       12.8478       4.87495       2.40504        40     
produce:pandoraShower:LArPandoraShowerCreation         0.950759       6.27779       25.0098       5.63422       4.26519        40     
produce:pandoracalo:Calorimetry                         0.42545       1.78007       4.88266       1.56885      0.890162        40     
produce:pandoracalonosce:Calorimetry                   0.390357       1.73189       4.62019       1.61241       0.86866        40     
produce:pandorapid:Chi2ParticleID                     0.00157382    0.00356962    0.00831082    0.00302099    0.00154915       40     
produce:pandoraShowercalo:ShowerCalorimetry            0.693901       4.80644       21.4963       4.16746       3.74556        40     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.617394       4.65051       21.3481       3.81598       3.58885        40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0936165     0.216228       0.36118      0.219485      0.0510183       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0326191     0.0766019     0.125368      0.0734145     0.0202416       40     
produce:emtrkmichelid:EmTrackMichelId                   19.9228       33.7103       51.8091       33.6943       6.08539        40     
[art]:TriggerResults:TriggerResultInserter            4.1378e-05    9.22991e-05   0.000130116    9.014e-05    1.6293e-05       40     
end_path:out1:RootOutput                              1.1803e-05    2.93403e-05   6.8259e-05    2.7792e-05    1.00992e-05      40     
end_path:michelremoving:michelremoving                 0.0339713      0.17913      0.651611      0.109012      0.151514        40     
end_path:michelremovinggnocchi:michelremoving          0.0314508     0.168588      0.645325      0.101471      0.148161        40     
end_path:calibanagnocchi:CalibAnaTree                   92.234        257.935       483.952       246.04        83.3876        40     
end_path:out1:RootOutput(write)                        0.669437       1.11988       1.53615       1.13172      0.223675        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6076.8 MB
  Peak resident set size usage (VmHWM): 3640.32 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_20916_142_1754366408_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_20916_142_1754366408_stage1_stage2_tfile.root', 'start_time': 1754367628.0752294, 'end_time': 1754383494.706664}
justIN time: 2025-08-05 23:17:47 UTC       justIN version: 01.04.00