justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302096.111@dunegpschedd01.fnal.gov

Jobsub ID302096.111@dunegpschedd01.fnal.gov
Workflow ID12511
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-29 10:37:51
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2026-01-29 10:48:50
From worker nodeHostnamenode2b04.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 10:39:09
Input filesfardet-hd:nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root
JobscriptExit code0
Real time9m (561s)
CPU time6m (419s = 74%)
Max RSS bytes1362583552 (1299 MiB)
Outputting started2026-01-29 10:48:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/kalmanAlg_CHECK_nu_0/fnal/12511/1/001/ClusterValidation_WithAlg_nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root
Finished2026-01-29 10:48:50
Saved logsjustin-logs:302096.111-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ize: 139609) ) with status: [SUCCESS] .
[2026-01-29 10:48:26.994787 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x21977950.
[2026-01-29 10:48:27.011042 +0000][Debug  ][File              ][ 1260] [0xaec2560@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root?xrdcl.requuid=0e4f8681-be41-43fe-a797-9eec22061573] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:48:27.011098 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170400, size: 132) ).
[2026-01-29 10:48:27.011159 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170400, size: 132) ) from out-queu to in-queue.
[2026-01-29 10:48:27.025691 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x26de2ed0] Assigned MsgHandler: 0x14d008c0.
[2026-01-29 10:48:27.025707 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x14d008c0] Removed MsgHandler: 0x14d008c0 from the in-queue.
[2026-01-29 10:48:27.025760 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170400, size: 132) ) with status: [SUCCESS] .
[2026-01-29 10:48:27.025797 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x14d008c0.
[2026-01-29 10:48:27.025843 +0000][Debug  ][File              ][ 1260] [0xaec2560@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root?xrdcl.requuid=0e4f8681-be41-43fe-a797-9eec22061573] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:48:27.025868 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170532, size: 138) ).
[2026-01-29 10:48:27.025908 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170532, size: 138) ) from out-queu to in-queue.
[2026-01-29 10:48:27.040613 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x2199a1d0] Assigned MsgHandler: 0x14d008c0.
[2026-01-29 10:48:27.040659 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x14d008c0] Removed MsgHandler: 0x14d008c0 from the in-queue.
[2026-01-29 10:48:27.040700 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 170532, size: 138) ) with status: [SUCCESS] .
[2026-01-29 10:48:27.040725 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x14d008c0.
[2026-01-29 10:48:27.040773 +0000][Debug  ][File              ][ 1260] [0xaec2560@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root?xrdcl.requuid=0e4f8681-be41-43fe-a797-9eec22061573] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:48:27.040799 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 161652, size: 2342) ).
[2026-01-29 10:48:27.040855 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 161652, size: 2342) ) from out-queu to in-queue.
[2026-01-29 10:48:27.055277 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x2199a1d0] Assigned MsgHandler: 0x14d008c0.
[2026-01-29 10:48:27.055305 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x14d008c0] Removed MsgHandler: 0x14d008c0 from the in-queue.
[2026-01-29 10:48:27.055461 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x14d008c0 (message: kXR_read (handle: 0x00000000, offset: 161652, size: 2342) ) with status: [SUCCESS] .
[2026-01-29 10:48:27.055502 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x14d008c0.
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
29-Jan-2026 10:48:29 GMT  Closed output file "nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2_reco2.root"
[2026-01-29 10:48:29.608493 +0000][Debug  ][File              ][ 1260] [0xaec2560@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root?xrdcl.requuid=0e4f8681-be41-43fe-a797-9eec22061573] Sending a close command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:48:29.608653 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x14701500 (message: kXR_close (handle: 0x00000000) ).
[2026-01-29 10:48:29.608714 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x14701500 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2026-01-29 10:48:29.623791 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x13b68ac0] Assigned MsgHandler: 0x14701500.
[2026-01-29 10:48:29.623819 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x14701500] Removed MsgHandler: 0x14701500 from the in-queue.
[2026-01-29 10:48:29.623872 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x14701500 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2026-01-29 10:48:29.623900 +0000][Debug  ][File              ][ 1260] [0xaec2560@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root?xrdcl.requuid=0e4f8681-be41-43fe-a797-9eec22061573] Close returned from heplns147.pp.rl.ac.uk:51847 with: [SUCCESS] 
[2026-01-29 10:48:29.623918 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x14701500.
29-Jan-2026 10:48:29 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/9a/98/nu_dune10kt_1x2x6_1410_207_20230826T123322Z_gen_g4_detsim_hitreco__20240229T171522Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.126465       3.94158       79.6047       2.16231       10.0354        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0298367     0.0406326     0.067081      0.0326056     0.0100639       100    
reco:pandora:StandardPandora                   0.0949281      3.89066       79.5036       2.12287       10.0247        100    
[art]:TriggerResults:TriggerResultInserter    1.0925e-05    1.81639e-05   5.4573e-05    1.5827e-05    8.46782e-06      100    
end_path:out1:RootOutput                       2.015e-06    2.57233e-06    9.576e-06     2.462e-06    7.71964e-07      100    
end_path:out1:RootOutput(write)               0.00069017     0.0101251     0.0797512    0.00764919     0.0121787       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2344.44 MB
  Peak resident set size usage (VmHWM): 1362.58 MB
====================================================================================================
Art has completed and will exit with status 0.
[2026-01-29 10:48:30.029421 +0000][Debug  ][JobMgr            ][ 1260] Stopping the job manager...
[2026-01-29 10:48:30.030382 +0000][Debug  ][JobMgr            ][ 1260] Job manager stopped
[2026-01-29 10:48:30.030818 +0000][Debug  ][TaskMgr           ][ 1260] Stopping the task manager...
[2026-01-29 10:48:30.030957 +0000][Debug  ][TaskMgr           ][ 1260] Task manager stopped
[2026-01-29 10:48:30.031002 +0000][Debug  ][Poller            ][ 1260] Stopping the poller...
[2026-01-29 10:48:30.031124 +0000][Debug  ][AsyncSock         ][ 1260] [heplns147.pp.rl.ac.uk:51847.0] Closing the socket
[2026-01-29 10:48:30.031284 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.37]:40314><--><[::ffff:130.246.47.147]:51847> Removing socket from the poller
[2026-01-29 10:48:30.031349 +0000][Debug  ][PostMaster        ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying stream
[2026-01-29 10:48:30.031511 +0000][Debug  ][AsyncSock         ][ 1260] [heplns147.pp.rl.ac.uk:51847.0] Closing the socket
[2026-01-29 10:48:30.031535 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
[2026-01-29 10:48:30.031541 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.37]:39450><--><[::ffff:130.246.47.230]:1094> Removing socket from the poller
[2026-01-29 10:48:30.031737 +0000][Debug  ][PostMaster        ][ 1260] [mover.pp.rl.ac.uk:1094] Destroying stream
[2026-01-29 10:48:30.031961 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 04:26:58 UTC       justIN version: 01.06.00