justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20916.12@dunegpschedd01.fnal.gov

Jobsub ID20916.12@dunegpschedd01.fnal.gov
Workflow ID404
Stage ID1
User namemsiden@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-08-05 03:20:43
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2025-08-05 08:47:20
From worker nodeHostnameheplnc161.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-05 03:24:10
Input fileshd-protodune:np04hd_raw_run027087_0053_dataflow1_datawriter_0_20240614T151343.hdf5
JobscriptExit code0
Real time5h (19338s)
CPU time5h (18377s = 95%)
Max RSS bytes4397670400 (4193 MiB)
Outputting started2025-08-05 08:46:29
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20916_12_1754364269_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/msiden/channel_map_test/fnal/00404/1/001/pdhd_processing_1_20916_12_1754364269_stage1_stage2_tfile.root
Finished2025-08-05 08:47:20
Saved logsjustin-logs:20916.12-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er"
FrameSaver: q=1.90864e+06 n=969321 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 73544 traces, 73544 summary
Retagger: tagging trace set: gauss with 67466 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67466 traces tagged "gauss"
FrameSaver: q=1.82852e+06 n=951967 tag=gauss
wclsFrameSaver: saving 73544 traces tagged "wiener"
FrameSaver: q=2.08413e+06 n=990124 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79200 traces, 79200 summary
Retagger: tagging trace set: gauss with 67554 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67554 traces tagged "gauss"
FrameSaver: q=1.83309e+06 n=853678 tag=gauss
wclsFrameSaver: saving 79200 traces tagged "wiener"
FrameSaver: q=2.02129e+06 n=910047 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72298 traces, 72298 summary
Retagger: tagging trace set: gauss with 64383 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 64383 traces tagged "gauss"
FrameSaver: q=1.5856e+06 n=865477 tag=gauss
wclsFrameSaver: saving 72298 traces tagged "wiener"
FrameSaver: q=1.75899e+06 n=908044 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 84047 traces, 84047 summary
Retagger: tagging trace set: gauss with 73738 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 73738 traces tagged "gauss"
FrameSaver: q=2.09787e+06 n=1004568 tag=gauss
wclsFrameSaver: saving 84047 traces tagged "wiener"
FrameSaver: q=2.15269e+06 n=1034868 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 95466 traces, 95466 summary
Retagger: tagging trace set: gauss with 85004 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 85004 traces tagged "gauss"
FrameSaver: q=2.52565e+06 n=1234424 tag=gauss
wclsFrameSaver: saving 95466 traces tagged "wiener"
FrameSaver: q=2.56528e+06 n=1250763 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77487 traces, 77487 summary
Retagger: tagging trace set: gauss with 68257 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68257 traces tagged "gauss"
FrameSaver: q=1.90631e+06 n=1010362 tag=gauss
wclsFrameSaver: saving 77487 traces tagged "wiener"
FrameSaver: q=1.99175e+06 n=1011403 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56561 traces, 56561 summary
Retagger: tagging trace set: gauss with 51729 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51729 traces tagged "gauss"
FrameSaver: q=919741 n=624358 tag=gauss
wclsFrameSaver: saving 56561 traces tagged "wiener"
FrameSaver: q=983785 n=652585 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68078 traces, 68078 summary
Retagger: tagging trace set: gauss with 62085 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62085 traces tagged "gauss"
FrameSaver: q=1.48232e+06 n=841182 tag=gauss
wclsFrameSaver: saving 68078 traces tagged "wiener"
FrameSaver: q=1.51717e+06 n=863561 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 99487 traces, 99487 summary
Retagger: tagging trace set: gauss with 87218 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 87218 traces tagged "gauss"
FrameSaver: q=2.6296e+06 n=1299313 tag=gauss
wclsFrameSaver: saving 99487 traces tagged "wiener"
FrameSaver: q=2.91149e+06 n=1336792 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64060 traces, 64060 summary
Retagger: tagging trace set: gauss with 59588 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 59588 traces tagged "gauss"
FrameSaver: q=1.76861e+06 n=849146 tag=gauss
wclsFrameSaver: saving 64060 traces tagged "wiener"
FrameSaver: q=1.97593e+06 n=871941 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 73073 traces, 73073 summary
Retagger: tagging trace set: gauss with 63779 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 63779 traces tagged "gauss"
FrameSaver: q=1.66944e+06 n=855508 tag=gauss
wclsFrameSaver: saving 73073 traces tagged "wiener"
FrameSaver: q=1.82061e+06 n=898317 tag=wiener
05-Aug-2025 09:46:27 BST  Closed output file "pdhd_processing_1_20916_12_1754364269_stage1_stage2.root"
05-Aug-2025 09:46:27 BST  Closed input file "pdhd_processing_1_20916_12_1754364269_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              184.286       448.317       762.258       460.062       126.95         40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000918753   0.00484087      0.02684     0.00346216    0.00419399       40     
produce:fembfilter:PDHDFEMBFilter                      0.756496      0.913457       1.03403      0.920748      0.0427721       40     
produce:wclsdatahd:WireCellToolkit                      46.9952       52.3229       57.4699       52.7243       2.37348        40     
produce:gaushit:GausHitFinder                          0.612147       1.04541       1.65978       1.05707      0.235397        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000338084   0.000503899   0.000813105   0.000490284   9.34159e-05      40     
produce:reco3d:SpacePointSolver                        0.219716       1.1749        4.93247       0.96322      0.838062        40     
produce:hitpdune:DisambigFromSpacePoints               0.218073      0.873926       2.09443       0.76101      0.445602        40     
produce:pandora:StandardPandora                         8.03367       24.6755       89.5385       21.3055       15.0592        40     
produce:pandoraWriter:StandardPandora                  0.151815      0.249727       0.40335      0.255375      0.0524413       40     
produce:pandoraTrack:LArPandoraTrackCreation            1.06235       4.74879       16.6395       4.36082       2.99217        40     
produce:pandoraShower:LArPandoraShowerCreation         0.848426       5.94191       26.8115       4.9264        4.97849        40     
produce:pandoracalo:Calorimetry                         0.42676       1.7276        5.5275        1.49534      0.978976        40     
produce:pandoracalonosce:Calorimetry                   0.434967       1.63985       5.28581       1.40311       1.00986        40     
produce:pandorapid:Chi2ParticleID                     0.00190395    0.00400324     0.0103917    0.00337373    0.00175942       40     
produce:pandoraShowercalo:ShowerCalorimetry            0.617554       4.31089       20.8248       3.45946       3.66309        40     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.571162       4.00865       21.3825       3.01992       3.56488        40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0790419      0.21015      0.380293      0.210762      0.0672316       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0320698     0.0741984     0.142237      0.0716744     0.0238365       40     
produce:emtrkmichelid:EmTrackMichelId                   22.629        35.1515       54.9701       33.5957       7.13001        40     
[art]:TriggerResults:TriggerResultInserter            5.1566e-05    8.99868e-05   0.000150733   8.5886e-05    2.27642e-05      40     
end_path:out1:RootOutput                              1.5108e-05    2.64236e-05   4.4073e-05    2.6429e-05    4.86759e-06      40     
end_path:michelremoving:michelremoving                 0.0357665     0.195828      0.886964      0.0860935     0.199522        40     
end_path:michelremovinggnocchi:michelremoving          0.0285017     0.187257      0.877442      0.0794995     0.202128        40     
end_path:calibanagnocchi:CalibAnaTree                   94.0823       306.921       567.546       322.073       106.81         40     
end_path:out1:RootOutput(write)                        0.720077       1.19009       1.84973       1.17466      0.272543        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6939.05 MB
  Peak resident set size usage (VmHWM): 4397.67 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_20916_12_1754364269_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_20916_12_1754364269_stage1_stage2_tfile.root', 'start_time': 1754365589.4480479, 'end_time': 1754383589.1451783}
justIN time: 2025-08-05 23:26:15 UTC       justIN version: 01.04.00