justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302096.47@dunegpschedd01.fnal.gov

Jobsub ID302096.47@dunegpschedd01.fnal.gov
Workflow ID12511
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-29 10:37:51
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2026-01-29 10:48:45
From worker nodeHostnamenode2b06.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 10:39:41
Input filesfardet-hd:nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root
JobscriptExit code0
Real time8m (524s)
CPU time6m (376s = 71%)
Max RSS bytes1286184960 (1226 MiB)
Outputting started2026-01-29 10:48:26
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/kalmanAlg_CHECK_nu_0/fnal/12511/1/001/ClusterValidation_WithAlg_nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root
Finished2026-01-29 10:48:45
Saved logsjustin-logs:302096.47-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

x00000000, offset: 1449468456, size: 254019) ) with status: [SUCCESS] .
[2026-01-29 10:48:21.335040 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying MsgHandler: 0xae1fb20.
[2026-01-29 10:48:21.364455 +0000][Debug  ][File              ][ 1260] [0xd3a6a50@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root?xrdcl.requuid=403c857e-f62f-408a-b167-0c45ddd11aba] Sending a read command for handle 0x0 to heplns137.pp.rl.ac.uk:51988
[2026-01-29 10:48:21.364536 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] MsgHandler created: 0xdaa0190 (message: kXR_read (handle: 0x00000000, offset: 170013, size: 132) ).
[2026-01-29 10:48:21.364604 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Moving MsgHandler: 0xdaa0190 (message: kXR_read (handle: 0x00000000, offset: 170013, size: 132) ) from out-queu to in-queue.
[2026-01-29 10:48:21.378763 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0xdaa10e0] Assigned MsgHandler: 0xdaa0190.
[2026-01-29 10:48:21.378810 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0xdaa0190] Removed MsgHandler: 0xdaa0190 from the in-queue.
[2026-01-29 10:48:21.378849 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Calling MsgHandler: 0xdaa0190 (message: kXR_read (handle: 0x00000000, offset: 170013, size: 132) ) with status: [SUCCESS] .
[2026-01-29 10:48:21.378870 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying MsgHandler: 0xdaa0190.
[2026-01-29 10:48:21.378929 +0000][Debug  ][File              ][ 1260] [0xd3a6a50@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root?xrdcl.requuid=403c857e-f62f-408a-b167-0c45ddd11aba] Sending a read command for handle 0x0 to heplns137.pp.rl.ac.uk:51988
[2026-01-29 10:48:21.378952 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] MsgHandler created: 0xdaa0340 (message: kXR_read (handle: 0x00000000, offset: 170145, size: 138) ).
[2026-01-29 10:48:21.379004 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Moving MsgHandler: 0xdaa0340 (message: kXR_read (handle: 0x00000000, offset: 170145, size: 138) ) from out-queu to in-queue.
[2026-01-29 10:48:21.393124 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0xdaa11d0] Assigned MsgHandler: 0xdaa0340.
[2026-01-29 10:48:21.393141 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0xdaa0340] Removed MsgHandler: 0xdaa0340 from the in-queue.
[2026-01-29 10:48:21.393164 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Calling MsgHandler: 0xdaa0340 (message: kXR_read (handle: 0x00000000, offset: 170145, size: 138) ) with status: [SUCCESS] .
[2026-01-29 10:48:21.393177 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying MsgHandler: 0xdaa0340.
[2026-01-29 10:48:21.393210 +0000][Debug  ][File              ][ 1260] [0xd3a6a50@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root?xrdcl.requuid=403c857e-f62f-408a-b167-0c45ddd11aba] Sending a read command for handle 0x0 to heplns137.pp.rl.ac.uk:51988
[2026-01-29 10:48:21.393226 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] MsgHandler created: 0xdaa12f0 (message: kXR_read (handle: 0x00000000, offset: 160153, size: 3300) ).
[2026-01-29 10:48:21.393265 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Moving MsgHandler: 0xdaa12f0 (message: kXR_read (handle: 0x00000000, offset: 160153, size: 3300) ) from out-queu to in-queue.
[2026-01-29 10:48:21.407621 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0xdaa2330] Assigned MsgHandler: 0xdaa12f0.
[2026-01-29 10:48:21.407671 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0xdaa12f0] Removed MsgHandler: 0xdaa12f0 from the in-queue.
[2026-01-29 10:48:21.407710 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Calling MsgHandler: 0xdaa12f0 (message: kXR_read (handle: 0x00000000, offset: 160153, size: 3300) ) with status: [SUCCESS] .
[2026-01-29 10:48:21.407728 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying MsgHandler: 0xdaa12f0.
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
29-Jan-2026 10:48:25 GMT  Closed output file "nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2_reco2.root"
[2026-01-29 10:48:25.318055 +0000][Debug  ][File              ][ 1260] [0xd3a6a50@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root?xrdcl.requuid=403c857e-f62f-408a-b167-0c45ddd11aba] Sending a close command for handle 0x0 to heplns137.pp.rl.ac.uk:51988
[2026-01-29 10:48:25.318096 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] MsgHandler created: 0xaf10860 (message: kXR_close (handle: 0x00000000) ).
[2026-01-29 10:48:25.318155 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Moving MsgHandler: 0xaf10860 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2026-01-29 10:48:25.332595 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x1554c890] Assigned MsgHandler: 0xaf10860.
[2026-01-29 10:48:25.332607 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0xaf10860] Removed MsgHandler: 0xaf10860 from the in-queue.
[2026-01-29 10:48:25.332630 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Calling MsgHandler: 0xaf10860 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2026-01-29 10:48:25.332651 +0000][Debug  ][File              ][ 1260] [0xd3a6a50@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root?xrdcl.requuid=403c857e-f62f-408a-b167-0c45ddd11aba] Close returned from heplns137.pp.rl.ac.uk:51988 with: [SUCCESS] 
[2026-01-29 10:48:25.332669 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying MsgHandler: 0xaf10860.
29-Jan-2026 10:48:25 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/d4/9e/nu_dune10kt_1x2x6_1108_261_20230826T213346Z_gen_g4_detsim_hitreco__20240229T165526Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      0.12749       3.55923       25.8826       2.38253       3.98232        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0294833     0.0466347     0.105727      0.0490572     0.0121355       100    
reco:pandora:StandardPandora                   0.0957126      3.50143       25.7748       2.33098       3.96995        100    
[art]:TriggerResults:TriggerResultInserter     1.253e-05    3.93704e-05   0.00046044    1.9696e-05    7.71445e-05      100    
end_path:out1:RootOutput                       2.241e-06    2.8777e-06    1.1107e-05     2.685e-06    1.00812e-06      100    
end_path:out1:RootOutput(write)               0.000780593    0.0109708     0.0656563    0.00755927     0.0121475       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2269.56 MB
  Peak resident set size usage (VmHWM): 1286.18 MB
====================================================================================================
Art has completed and will exit with status 0.
[2026-01-29 10:48:25.795827 +0000][Debug  ][JobMgr            ][ 1260] Stopping the job manager...
[2026-01-29 10:48:25.796957 +0000][Debug  ][JobMgr            ][ 1260] Job manager stopped
[2026-01-29 10:48:25.797786 +0000][Debug  ][TaskMgr           ][ 1260] Stopping the task manager...
[2026-01-29 10:48:25.798390 +0000][Debug  ][TaskMgr           ][ 1260] Task manager stopped
[2026-01-29 10:48:25.798404 +0000][Debug  ][Poller            ][ 1260] Stopping the poller...
[2026-01-29 10:48:25.798705 +0000][Debug  ][AsyncSock         ][ 1260] [heplns137.pp.rl.ac.uk:51988.0] Closing the socket
[2026-01-29 10:48:25.798896 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.39]:37694><--><[::ffff:130.246.47.137]:51988> Removing socket from the poller
[2026-01-29 10:48:25.798964 +0000][Debug  ][PostMaster        ][ 1260] [heplns137.pp.rl.ac.uk:51988] Destroying stream
[2026-01-29 10:48:25.799187 +0000][Debug  ][AsyncSock         ][ 1260] [heplns137.pp.rl.ac.uk:51988.0] Closing the socket
[2026-01-29 10:48:25.799229 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
[2026-01-29 10:48:25.799307 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.39]:55980><--><[::ffff:130.246.47.231]:1094> Removing socket from the poller
[2026-01-29 10:48:25.799508 +0000][Debug  ][PostMaster        ][ 1260] [mover.pp.rl.ac.uk:1094] Destroying stream
[2026-01-29 10:48:25.799552 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 06:06:41 UTC       justIN version: 01.06.00