justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 256238.51@dunegpschedd02.fnal.gov

Jobsub ID256238.51@dunegpschedd02.fnal.gov
Workflow ID10928
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes1048576000 (1000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2025-12-02 09:34:52
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2025-12-02 09:38:43
From worker nodeHostnamenode2b05.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes1048576000 (1000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-12-02 09:35:10
Input filesfardet-hd:nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-12-02 09:38:43
Saved logsjustin-logs:256238.51-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

38:25.085005 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x1ad5fb30 (message: kXR_read (handle: 0x00000000, offset: 85505320, size: 285942) ).
[2025-12-02 09:38:25.085069 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x1ad5fb30 (message: kXR_read (handle: 0x00000000, offset: 85505320, size: 285942) ) from out-queu to in-queue.
[2025-12-02 09:38:25.103521 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x1ad609d0] Assigned MsgHandler: 0x1ad5fb30.
[2025-12-02 09:38:25.103576 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x1ad5fb30] Removed MsgHandler: 0x1ad5fb30 from the in-queue.
[2025-12-02 09:38:25.105981 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x1ad5fb30 (message: kXR_read (handle: 0x00000000, offset: 85505320, size: 285942) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.106010 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x1ad5fb30.
[2025-12-02 09:38:25.112971 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.113017 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975491, size: 102) ).
[2025-12-02 09:38:25.113082 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975491, size: 102) ) from out-queu to in-queue.
[2025-12-02 09:38:25.131339 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x1afc9500] Assigned MsgHandler: 0x1ae3b8e0.
[2025-12-02 09:38:25.131381 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x1ae3b8e0] Removed MsgHandler: 0x1ae3b8e0 from the in-queue.
[2025-12-02 09:38:25.131421 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975491, size: 102) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.131442 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x1ae3b8e0.
[2025-12-02 09:38:25.131497 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.131521 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975593, size: 108) ).
[2025-12-02 09:38:25.131578 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975593, size: 108) ) from out-queu to in-queue.
[2025-12-02 09:38:25.149923 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x1afc9710] Assigned MsgHandler: 0x1ae3b8e0.
[2025-12-02 09:38:25.149961 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x1ae3b8e0] Removed MsgHandler: 0x1ae3b8e0 from the in-queue.
[2025-12-02 09:38:25.149999 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x1ae3b8e0 (message: kXR_read (handle: 0x00000000, offset: 1264975593, size: 108) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.150016 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x1ae3b8e0.
[2025-12-02 09:38:25.150070 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.150091 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x1afc9000 (message: kXR_read (handle: 0x00000000, offset: 1264938532, size: 36959) ).
[2025-12-02 09:38:25.150146 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x1afc9000 (message: kXR_read (handle: 0x00000000, offset: 1264938532, size: 36959) ) from out-queu to in-queue.
[2025-12-02 09:38:25.168501 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x1ae3bd60] Assigned MsgHandler: 0x1afc9000.
[2025-12-02 09:38:25.168534 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x1afc9000] Removed MsgHandler: 0x1afc9000 from the in-queue.
[2025-12-02 09:38:25.168582 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x1afc9000 (message: kXR_read (handle: 0x00000000, offset: 1264938532, size: 36959) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.168599 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x1afc9000.
[2025-12-02 09:38:25.177557 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.177610 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166312, size: 132) ).
[2025-12-02 09:38:25.177711 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166312, size: 132) ) from out-queu to in-queue.
[2025-12-02 09:38:25.196016 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x198e3300] Assigned MsgHandler: 0x198e2980.
[2025-12-02 09:38:25.196070 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x198e2980] Removed MsgHandler: 0x198e2980 from the in-queue.
[2025-12-02 09:38:25.196113 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166312, size: 132) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.196133 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x198e2980.
[2025-12-02 09:38:25.196193 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.196218 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166444, size: 138) ).
[2025-12-02 09:38:25.196290 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166444, size: 138) ) from out-queu to in-queue.
[2025-12-02 09:38:25.214535 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x198e34b0] Assigned MsgHandler: 0x198e2980.
[2025-12-02 09:38:25.214582 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x198e2980] Removed MsgHandler: 0x198e2980 from the in-queue.
[2025-12-02 09:38:25.214626 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x198e2980 (message: kXR_read (handle: 0x00000000, offset: 166444, size: 138) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.214648 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x198e2980.
[2025-12-02 09:38:25.214713 +0000][Debug  ][File              ][ 1306] [0xda98260@root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/dd/2f/nu_dune10kt_1x2x6_1109_473_20230827T043048Z_gen_g4_detsim_hitreco__20240229T194346Z_reco2.root?xrdcl.requuid=6144da6c-ec4f-4689-9cd7-9d03dba3eabf] Sending a read command for handle 0x0 to wolf6.grid.surfsara.nl:34270
[2025-12-02 09:38:25.214740 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] MsgHandler created: 0x194251b0 (message: kXR_read (handle: 0x00000000, offset: 12315, size: 3452) ).
[2025-12-02 09:38:25.214808 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Moving MsgHandler: 0x194251b0 (message: kXR_read (handle: 0x00000000, offset: 12315, size: 3452) ) from out-queu to in-queue.
[2025-12-02 09:38:25.233020 +0000][Debug  ][ExDbgMsg          ][ 1306] [msg: 0x19425e50] Assigned MsgHandler: 0x194251b0.
[2025-12-02 09:38:25.233066 +0000][Debug  ][ExDbgMsg          ][ 1306] [handler: 0x194251b0] Removed MsgHandler: 0x194251b0 from the in-queue.
[2025-12-02 09:38:25.233108 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Calling MsgHandler: 0x194251b0 (message: kXR_read (handle: 0x00000000, offset: 12315, size: 3452) ) with status: [SUCCESS] .
[2025-12-02 09:38:25.233128 +0000][Debug  ][ExDbgMsg          ][ 1306] [wolf6.grid.surfsara.nl:34270] Destroying MsgHandler: 0x194251b0.
=== End last 100 lines of lar log file ===
mv: cannot stat 'ClusterValidation_WithAlg.root': No such file or directory
mv: cannot stat 'ClusterValidation_WithoutAlg.root': No such file or directory
justIN time: 2025-12-19 12:14:20 UTC       justIN version: 01.05.03