justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302096.138@dunegpschedd01.fnal.gov

Jobsub ID302096.138@dunegpschedd01.fnal.gov
Workflow ID12511
Stage ID1
User nameimawby@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-01-29 10:37:51
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2026-01-29 10:49:37
From worker nodeHostnamenode2b06.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 10:39:25
Input filesfardet-hd:nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root
JobscriptExit code0
Real time9m (592s)
CPU time7m (432s = 72%)
Max RSS bytes1235021824 (1177 MiB)
Outputting started2026-01-29 10:49:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/kalmanAlg_CHECK_nu_0/fnal/12511/1/001/ClusterValidation_WithAlg_nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root
Finished2026-01-29 10:49:37
Saved logsjustin-logs:302096.138-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

28406, size: 68156) ) with status: [SUCCESS] .
[2026-01-29 10:49:15.576961 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x1c784010.
[2026-01-29 10:49:15.588310 +0000][Debug  ][File              ][ 1260] [0xba77970@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root?xrdcl.requuid=e2b61088-bfa8-4341-95f8-65742c5fad24] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:49:15.588363 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166159, size: 132) ).
[2026-01-29 10:49:15.588430 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166159, size: 132) ) from out-queu to in-queue.
[2026-01-29 10:49:15.602727 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x16b325e0] Assigned MsgHandler: 0x179d6190.
[2026-01-29 10:49:15.602757 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x179d6190] Removed MsgHandler: 0x179d6190 from the in-queue.
[2026-01-29 10:49:15.602813 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166159, size: 132) ) with status: [SUCCESS] .
[2026-01-29 10:49:15.602852 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x179d6190.
[2026-01-29 10:49:15.602912 +0000][Debug  ][File              ][ 1260] [0xba77970@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root?xrdcl.requuid=e2b61088-bfa8-4341-95f8-65742c5fad24] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:49:15.602935 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166291, size: 138) ).
[2026-01-29 10:49:15.602978 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166291, size: 138) ) from out-queu to in-queue.
[2026-01-29 10:49:15.617303 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x16e0a0e0] Assigned MsgHandler: 0x179d6190.
[2026-01-29 10:49:15.617343 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x179d6190] Removed MsgHandler: 0x179d6190 from the in-queue.
[2026-01-29 10:49:15.617378 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 166291, size: 138) ) with status: [SUCCESS] .
[2026-01-29 10:49:15.617399 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x179d6190.
[2026-01-29 10:49:15.617427 +0000][Debug  ][File              ][ 1260] [0xba77970@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root?xrdcl.requuid=e2b61088-bfa8-4341-95f8-65742c5fad24] Sending a read command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:49:15.617440 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 157007, size: 2688) ).
[2026-01-29 10:49:15.617475 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 157007, size: 2688) ) from out-queu to in-queue.
[2026-01-29 10:49:15.631864 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x16e0a0e0] Assigned MsgHandler: 0x179d6190.
[2026-01-29 10:49:15.631899 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0x179d6190] Removed MsgHandler: 0x179d6190 from the in-queue.
[2026-01-29 10:49:15.631935 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0x179d6190 (message: kXR_read (handle: 0x00000000, offset: 157007, size: 2688) ) with status: [SUCCESS] .
[2026-01-29 10:49:15.631955 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0x179d6190.
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
29-Jan-2026 10:49:18 GMT  Closed output file "nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2_reco2.root"
[2026-01-29 10:49:18.222367 +0000][Debug  ][File              ][ 1260] [0xba77970@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root?xrdcl.requuid=e2b61088-bfa8-4341-95f8-65742c5fad24] Sending a close command for handle 0x0 to heplns147.pp.rl.ac.uk:51847
[2026-01-29 10:49:18.222507 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] MsgHandler created: 0xb629530 (message: kXR_close (handle: 0x00000000) ).
[2026-01-29 10:49:18.222579 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Moving MsgHandler: 0xb629530 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2026-01-29 10:49:18.237291 +0000][Debug  ][ExDbgMsg          ][ 1260] [msg: 0x12cddee0] Assigned MsgHandler: 0xb629530.
[2026-01-29 10:49:18.237302 +0000][Debug  ][ExDbgMsg          ][ 1260] [handler: 0xb629530] Removed MsgHandler: 0xb629530 from the in-queue.
[2026-01-29 10:49:18.237326 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Calling MsgHandler: 0xb629530 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2026-01-29 10:49:18.237346 +0000][Debug  ][File              ][ 1260] [0xba77970@root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root?xrdcl.requuid=e2b61088-bfa8-4341-95f8-65742c5fad24] Close returned from heplns147.pp.rl.ac.uk:51847 with: [SUCCESS] 
[2026-01-29 10:49:18.237364 +0000][Debug  ][ExDbgMsg          ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying MsgHandler: 0xb629530.
29-Jan-2026 10:49:18 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/55/49/nu_dune10kt_1x2x6_1109_168_20230827T003305Z_gen_g4_detsim_hitreco__20240229T190312Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.133741       4.15889       80.6569       2.42182       8.25355        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0298215     0.0409539     0.0766512     0.0330914     0.0101282       100    
reco:pandora:StandardPandora                   0.101967       4.10733       80.5449        2.386        8.24431        100    
[art]:TriggerResults:TriggerResultInserter    1.1807e-05    2.51475e-05   0.000217076   1.9293e-05    2.29643e-05      100    
end_path:out1:RootOutput                       2.288e-06    2.86102e-06    9.787e-06    2.7555e-06    7.79174e-07      100    
end_path:out1:RootOutput(write)               0.000852512    0.0104286     0.0658527    0.00641232     0.0119463       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2224.02 MB
  Peak resident set size usage (VmHWM): 1235.02 MB
====================================================================================================
Art has completed and will exit with status 0.
[2026-01-29 10:49:18.660775 +0000][Debug  ][JobMgr            ][ 1260] Stopping the job manager...
[2026-01-29 10:49:18.661760 +0000][Debug  ][JobMgr            ][ 1260] Job manager stopped
[2026-01-29 10:49:18.662209 +0000][Debug  ][TaskMgr           ][ 1260] Stopping the task manager...
[2026-01-29 10:49:18.662602 +0000][Debug  ][TaskMgr           ][ 1260] Task manager stopped
[2026-01-29 10:49:18.662774 +0000][Debug  ][Poller            ][ 1260] Stopping the poller...
[2026-01-29 10:49:18.663102 +0000][Debug  ][AsyncSock         ][ 1260] [heplns147.pp.rl.ac.uk:51847.0] Closing the socket
[2026-01-29 10:49:18.663207 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.39]:58484><--><[::ffff:130.246.47.147]:51847> Removing socket from the poller
[2026-01-29 10:49:18.663248 +0000][Debug  ][PostMaster        ][ 1260] [heplns147.pp.rl.ac.uk:51847] Destroying stream
[2026-01-29 10:49:18.663281 +0000][Debug  ][AsyncSock         ][ 1260] [heplns147.pp.rl.ac.uk:51847.0] Closing the socket
[2026-01-29 10:49:18.663742 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
[2026-01-29 10:49:18.663811 +0000][Debug  ][Poller            ][ 1260] <[::ffff:192.41.105.39]:55970><--><[::ffff:130.246.47.231]:1094> Removing socket from the poller
[2026-01-29 10:49:18.663975 +0000][Debug  ][PostMaster        ][ 1260] [mover.pp.rl.ac.uk:1094] Destroying stream
[2026-01-29 10:49:18.664052 +0000][Debug  ][AsyncSock         ][ 1260] [mover.pp.rl.ac.uk:1094.0] Closing the socket
=== End last 100 lines of lar log file ===
justIN time: 2026-02-04 07:45:05 UTC       justIN version: 01.06.00