justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 208123.0@dunegpschedd02.fnal.gov

Jobsub ID208123.0@dunegpschedd02.fnal.gov
Workflow ID8146
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-09-27 14:49:47
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-09-27 16:10:38
From worker nodeHostnamecn115.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-27 14:50:21
Input filesmonte-carlo-008146-009045
Outputting started2025-09-27 15:47:21
Output files
Finished2025-09-27 16:10:38
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: WireCell::Gen::IncoherentAddNoise : 0.43 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.43 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.42 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.4 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.4 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.4 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.38 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.37 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.37 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.35 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.32 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.29 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.28 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::DepoBagger : 0.27 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.27 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.27 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.26 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.25 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.25 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.22 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0.21 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.2 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::DepoSetFanout : 0.19 sec
[16:42:44.022] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.18 sec
[16:42:44.022] I [ timer  ] Timer: wcls::FrameSaver : 0.15 sec
[16:42:44.022] I [ timer  ] Timer: wcls::SimDepoSource : 0.06 sec
[16:42:44.022] I [ timer  ] Timer: Total node execution : 219.99000012502074 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
Begin processing the 10th record. run: 8146 subRun: 0 event: 90450 at 27-Sep-2025 16:42:58 BST
SimDepoSource got 138682 depos from art tag "InputTag: label = 'IonAndScint', instance = ''" returns: okay
SimDepoSource: ready with 138682 depos spanning: [-2245.66, 2245.36]us
Retagger: tagging trace set: daq with 30720 traces, 0 summary
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 21.55 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 20.64 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 20.53 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 19.26 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 18.91 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 18.1 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 17 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 16.24 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 15.26 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 14.74 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 14.62 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoTransform : 11.71 sec
[16:47:00.872] I [ timer  ] Timer: wcls::SimChannelSink : 2.11 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.78 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.78 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.76 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.76 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.75 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.74 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.74 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.73 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.73 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.72 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.71 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Digitizer : 0.69 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.52 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.5 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Drifter : 0.5 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.45 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.43 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.43 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.42 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.41 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.39 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.39 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.38 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.38 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::IncoherentAddNoise : 0.36 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.3 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.29 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.29 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.28 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.28 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.28 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.27 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.26 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoBagger : 0.25 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.25 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.25 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.23 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.23 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0.2 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Reframer : 0.2 sec
[16:47:00.872] I [ timer  ] Timer: wcls::FrameSaver : 0.19 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.18 sec
[16:47:00.872] I [ timer  ] Timer: WireCell::Gen::DepoSetFanout : 0.18 sec
[16:47:00.872] I [ timer  ] Timer: wcls::SimDepoSource : 0.1 sec
[16:47:00.872] I [ timer  ] Timer: Total node execution : 229.6299993172288 sec
wclsFrameSaver: saving raw::RawDigits tagged "daq"
27-Sep-2025 16:47:18 BST  Closed output file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim.root"
27-Sep-2025 16:47:18 BST  Closed input file "prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      238.091       255.151       312.326       248.083       20.5652        10     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00113993    0.00422566    0.00566528    0.00451139    0.00120082       10     
simulate:tpcrawdecoder:WireCellToolkit           227.5        244.685       301.861       237.368       20.6031        10     
simulate:opdigi:OpDetDigitizerDUNE              2.52583       2.66489       2.79233       2.67755      0.0876986       10     
simulate:rns:RandomNumberSaver                3.1635e-05    5.60887e-05   0.000235526   3.6623e-05     5.985e-05       10     
[art]:TriggerResults:TriggerResultInserter    1.4151e-05    1.83846e-05   4.4465e-05    1.54405e-05   8.74179e-06      10     
end_path:out1:RootOutput                       2.926e-06    5.9827e-06    1.9449e-05    4.8165e-06    4.55072e-06      10     
end_path:out1:RootOutput(write)                 7.49153       7.7968        7.9441        7.90248      0.182255        10     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5165.66 MB
  Peak resident set size usage (VmHWM): 3516.82 MB
====================================================================================================
Art has completed and will exit with status 0.
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045.root 21952983
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4.root 179696360
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim.root 2350633311
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim.root.ext.json 1561
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim.root.json 1827
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim.root.temp.json 1827
written output file: prodbackground_radiological_decay0_dune10kt_1x2x6_centralAPA_20250927T145025Z_gen_009045_supernova_g4_detsim_hist.root 519
justIN time: 2025-11-05 14:16:38 UTC       justIN version: 01.05.01