justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233301.4@dunegpschedd01.fnal.gov

Jobsub ID233301.4@dunegpschedd01.fnal.gov
Workflow ID8928
Stage ID1
User namejjo@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-10-13 13:53:54
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-10-13 18:33:55
From worker nodeHostnamewn026.hep
cpuinfoIntel(R) Xeon(R) Gold 5215 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-13 14:03:10
Input filesmonte-carlo-008928-000005
JobscriptExit code0
Real time4h (16119s)
CPU time4h (15846s = 98%)
Max RSS bytes7356915712 (7016 MiB)
Outputting started2025-10-13 18:31:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/jjo/fnal/08928/1/001/monte-carlo-008928-000005_233301_4_1_20251013T140314Z_reco.root
Finished2025-10-13 18:33:55
Saved logsjustin-logs:233301.4-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

ning [607,977] "wiener" 
[19:31:44.269] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 save plane index: 0, Qtot=10744819 Qloss=-954795, 358 indices spanning [978,1335] "gauss" 
[19:31:44.559] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 save plane index: 1, Qtot=13062122 Qloss=-1226990, 856 indices spanning [1336,2191] "wiener" 
[19:31:44.772] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 save plane index: 1, Qtot=10973017 Qloss=-652171, 817 indices spanning [2192,3008] "gauss" 
[19:31:45.100] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 save plane index: 2, Qtot=8991714 Qloss=-675702, 443 indices spanning [3009,3451] "wiener" 
[19:31:45.422] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 save plane index: 2, Qtot=8490301 Qloss=-171490, 350 indices spanning [3452,3801] "gauss" 
[19:31:45.422] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 produce 3802 traces: 1670 wiener0, 0 decon_charge0, 1525 gauss0, frame tag: sigproc 
[19:31:45.422] D [sigproc ] <OmnibusSigProc:anode0sigproc0> call=38 output frame: frame: ident=20 time=14 tick=500 with 3802 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss0":1525 [0] "mp2_roi0":457 [0] "mp3_roi0":150 [0] "wiener0":1670 [1670] ] cmm:[ ] 
[19:31:45.440] D [  glue  ] <FrameFanin:nfsp> call=38 input 0: frame: ident=20 time=14 tick=500 with 3802 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss0":1525 [0] "mp2_roi0":457 [0] "mp3_roi0":150 [0] "wiener0":1670 [1670] ] cmm:[ ] input 1: frame: ident=20 time=14 tick=500 with 20140 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss1":5439 [0] "mp2_roi1":6339 [0] "mp3_roi1":1687 [0] "wiener1":6675 [6675] ] cmm:[ ] input 2: frame: ident=20 time=14 tick=500 with 8176 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss2":2676 [0] "mp2_roi2":1633 [0] "mp3_roi2":451 [0] "wiener2":3416 [3416] ] cmm:[ ] input 3: frame: ident=20 time=14 tick=500 with 14920 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss3":4604 [0] "mp2_roi3":3690 [0] "mp3_roi3":1190 [0] "wiener3":5436 [5436] ] cmm:[ ] input 4: frame: ident=20 time=14 tick=500 with 4059 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss4":1485 [0] "mp2_roi4":544 [0] "mp3_roi4":343 [0] "wiener4":1687 [1687] ] cmm:[ ] input 5: frame: ident=20 time=14 tick=500 with 19524 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss5":5882 [0] "mp2_roi5":5189 [0] "mp3_roi5":1506 [0] "wiener5":6947 [6947] ] cmm:[ ] input 6: frame: ident=20 time=14 tick=500 with 6856 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss6":2233 [0] "mp2_roi6":1277 [0] "mp3_roi6":585 [0] "wiener6":2761 [2761] ] cmm:[ ] input 7: frame: ident=20 time=14 tick=500 with 9822 traces.  frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss7":3251 [0] "mp2_roi7":1715 [0] "mp3_roi7":871 [0] "wiener7":3985 [3985] ] cmm:[ ] output: frame: ident=20 time=14 tick=500 with 87299 traces.  frame tags:[ "framefanin" ] 16 tagged trace sets:[ "gauss0":1525 [0] "gauss1":5439 [0] "gauss2":2676 [0] "gauss3":4604 [0] "gauss4":1485 [0] "gauss5":5882 [0] "gauss6":2233 [0] "gauss7":3251 [0] "wiener0":1670 [1670] "wiener1":6675 [6675] "wiener2":3416 [3416] "wiener3":5436 [5436] "wiener4":1687 [1687] "wiener5":6947 [6947] "wiener6":2761 [2761] "wiener7":3985 [3985] ] cmm:[ ] 
Retagger: tagging trace set: wiener with 32577 traces, 32577 summary
Retagger: tagging trace set: gauss with 27095 traces, 0 summary
[19:31:45.451] D [  glue  ] sink frame: #20 @1.4e-05 with 87299 traces, frame tags:[retagger], trace tags:[gauss, wiener]
[19:31:45.451] D [  glue  ] <FrameFanout:nfsp> call=39: see EOS 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel7> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode7sigproc7> EOS at call=39 anode=7 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel6> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode6sigproc6> EOS at call=39 anode=6 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel5> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode5sigproc5> EOS at call=39 anode=5 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel4> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode4sigproc4> EOS at call=39 anode=4 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel3> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode3sigproc3> EOS at call=39 anode=3 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel2> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode2sigproc2> EOS at call=39 anode=2 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel1> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode1sigproc1> EOS at call=39 anode=1 
[19:31:45.451] D [  glue  ] <ChannelSelector:chsel0> see EOS at call=19 
[19:31:45.451] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=39 anode=0 
[19:31:45.451] D [  glue  ] <FrameFanin:nfsp> EOS at call=39 with 8 
[19:31:45.451] D [  glue  ] frame sink sees EOS
[19:31:45.451] D [ pgraph ] <Pgrapher:> graph execution complete 
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.25 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.36 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.19 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.12 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.97 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.94 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.71 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.65 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:31:45.451] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[19:31:45.451] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[19:31:45.451] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[19:31:45.451] I [ timer  ] Timer: Total node execution : 65.23999910242856 sec
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 27095 traces tagged "gauss"
FrameSaver: q=5.87821e+06 n=473709 tag=gauss
wclsFrameSaver: saving 32577 traces tagged "wiener"
FrameSaver: q=6.13503e+06 n=449712 tag=wiener
13-Oct-2025 19:31:48 BST  Closed output file "monte-carlo-008928-000005_233301_4_1_20251013T140314Z_reco.root"
13-Oct-2025 19:31:48 BST  Closed input file "detsim.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      62.5831       70.6748       77.2345       70.859        3.76972        20     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000816579   0.00184084    0.00319023    0.00178075    0.000589452      20     
reco:rns:RandomNumberSaver                     3.884e-05    5.85571e-05   0.000288648   4.57785e-05   5.31252e-05      20     
reco:wclsdatavd:WireCellToolkit                 59.9516       68.6355       74.7137       69.1339       3.73802        20     
reco:gaushit:GausHitFinder                      1.15784       1.59338       2.21694       1.48898      0.361464        20     
[art]:TriggerResults:TriggerResultInserter    2.6993e-05    3.45391e-05   8.9796e-05    3.15835e-05    1.301e-05       20     
end_path:out1:RootOutput                       3.973e-06    6.85875e-06   2.7824e-05    5.1095e-06    5.03438e-06      20     
end_path:out1:RootOutput(write)                0.304544      0.440674      0.636789      0.408707      0.0950443       20     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5545.12 MB
  Peak resident set size usage (VmHWM): 3841.57 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Job completed successfully.
-rw-r--r--. 1 dune004 dune 3.5G Oct 13 19:04 detsim.root
-rw-r--r--. 1 dune004 dune  519 Oct 13 19:04 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune  519 Oct 13 15:37 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune 4.8G Oct 13 15:25 g4_stage1.root
-rw-r--r--. 1 dune004 dune  11G Oct 13 15:37 g4_stage2.root
-rw-r--r--. 1 dune004 dune 971K Oct 13 15:05 gen.root
-rw-r--r--. 1 dune004 dune  519 Oct 13 15:05 gen_protodunevd_cosmics_hist.root
-rw-r--r--. 1 dune004 dune 3.6G Oct 13 19:31 monte-carlo-008928-000005_233301_4_1_20251013T140314Z_reco.root
-rw-r--r--. 1 dune004 dune  519 Oct 13 19:31 reco_protoDUNE_hist.root
justIN time: 2025-11-04 15:55:15 UTC       justIN version: 01.05.01