Jobsub ID 226587.109@dunegpschedd02.fnal.gov
| Jobsub ID | 226587.109@dunegpschedd02.fnal.gov |
| Workflow ID | 8772 |
| Stage ID | 1 |
| User name | lwhite86@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 3600 (1 hours) |
| Submitted time | 2025-10-09 10:14:38 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce507 |
| Last heartbeat | 2025-10-09 10:56:48 |
| From worker node | Hostname | b9p07p9028.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-10-09 10:20:23 |
| Input files | fardet-hd:nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 35m (2158s) |
| CPU time | 29m (1769s = 81%) |
| Max RSS bytes | 1410002944 (1344 MiB) |
| Outputting started | 2025-10-09 10:56:22 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/lwhite86/fnal/08772/1/001/trainingFile_nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2.root
|
| Finished | 2025-10-09 10:56:48 |
| Saved logs | justin-logs:226587.109-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
n processing the 55th record. run: 1121 subRun: 1 event: 20955 at 09-Oct-2025 12:54:06 CEST
46
Begin processing the 56th record. run: 1121 subRun: 1 event: 20956 at 09-Oct-2025 12:54:09 CEST
340
Begin processing the 57th record. run: 1121 subRun: 1 event: 20957 at 09-Oct-2025 12:54:11 CEST
729
184
Begin processing the 58th record. run: 1121 subRun: 1 event: 20958 at 09-Oct-2025 12:54:17 CEST
54
54
Begin processing the 59th record. run: 1121 subRun: 1 event: 20959 at 09-Oct-2025 12:54:19 CEST
226
222
199
26
Begin processing the 60th record. run: 1121 subRun: 1 event: 20960 at 09-Oct-2025 12:54:23 CEST
Begin processing the 61st record. run: 1121 subRun: 1 event: 20961 at 09-Oct-2025 12:54:25 CEST
279
974
Begin processing the 62nd record. run: 1121 subRun: 1 event: 20962 at 09-Oct-2025 12:54:27 CEST
103
130
Begin processing the 63rd record. run: 1121 subRun: 1 event: 20963 at 09-Oct-2025 12:54:29 CEST
431
71
571
44
Begin processing the 64th record. run: 1121 subRun: 1 event: 20964 at 09-Oct-2025 12:54:32 CEST
689
1362
Begin processing the 65th record. run: 1121 subRun: 1 event: 20965 at 09-Oct-2025 12:54:34 CEST
545
393
311
425
101
127
Begin processing the 66th record. run: 1121 subRun: 1 event: 20966 at 09-Oct-2025 12:54:37 CEST
42
Begin processing the 67th record. run: 1121 subRun: 1 event: 20967 at 09-Oct-2025 12:54:39 CEST
Begin processing the 68th record. run: 1121 subRun: 1 event: 20968 at 09-Oct-2025 12:54:41 CEST
39
352
43
113
35
Begin processing the 69th record. run: 1121 subRun: 1 event: 20969 at 09-Oct-2025 12:54:43 CEST
PandoraContentApi::GetList(*this, m_trackPfoListName, pTrackPfoList) return STATUS_CODE_NOT_INITIALIZED
in function: GetAllTrackFeatures
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoradlcontent/LArTrackShowerId/DlTrackCharacterisationAlgorithm.cc line#: 155
Begin processing the 70th record. run: 1121 subRun: 1 event: 20970 at 09-Oct-2025 12:54:45 CEST
389
1737
467
854
265
319
108
60
115
35
52
Begin processing the 71st record. run: 1121 subRun: 1 event: 20971 at 09-Oct-2025 12:54:51 CEST
241
Begin processing the 72nd record. run: 1121 subRun: 1 event: 20972 at 09-Oct-2025 12:54:53 CEST
150
49
Begin processing the 73rd record. run: 1121 subRun: 1 event: 20973 at 09-Oct-2025 12:54:55 CEST
62
123
29
36
Begin processing the 74th record. run: 1121 subRun: 1 event: 20974 at 09-Oct-2025 12:54:57 CEST
47
28
Begin processing the 75th record. run: 1121 subRun: 1 event: 20975 at 09-Oct-2025 12:54:59 CEST
Begin processing the 76th record. run: 1121 subRun: 1 event: 20976 at 09-Oct-2025 12:55:03 CEST
645
57
372
32
Begin processing the 77th record. run: 1121 subRun: 1 event: 20977 at 09-Oct-2025 12:55:06 CEST
PandoraContentApi::GetList(*this, m_trackPfoListName, pTrackPfoList) return STATUS_CODE_NOT_INITIALIZED
in function: GetAllTrackFeatures
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoradlcontent/LArTrackShowerId/DlTrackCharacterisationAlgorithm.cc line#: 155
Begin processing the 78th record. run: 1121 subRun: 1 event: 20978 at 09-Oct-2025 12:55:07 CEST
88
1516
1232
110
44
57
Begin processing the 79th record. run: 1121 subRun: 1 event: 20979 at 09-Oct-2025 12:55:11 CEST
27
Begin processing the 80th record. run: 1121 subRun: 1 event: 20980 at 09-Oct-2025 12:55:13 CEST
1044
120
334
103
47
Begin processing the 81st record. run: 1121 subRun: 1 event: 20981 at 09-Oct-2025 12:55:15 CEST
32
206
367
816
376
Begin processing the 82nd record. run: 1121 subRun: 1 event: 20982 at 09-Oct-2025 12:55:37 CEST
89
261
582
73
27
32
Begin processing the 83rd record. run: 1121 subRun: 1 event: 20983 at 09-Oct-2025 12:55:40 CEST
427
116
155
68
Begin processing the 84th record. run: 1121 subRun: 1 event: 20984 at 09-Oct-2025 12:55:42 CEST
PandoraContentApi::GetList(*this, m_trackPfoListName, pTrackPfoList) return STATUS_CODE_NOT_INITIALIZED
in function: GetAllTrackFeatures
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoradlcontent/LArTrackShowerId/DlTrackCharacterisationAlgorithm.cc line#: 155
Begin processing the 85th record. run: 1121 subRun: 1 event: 20985 at 09-Oct-2025 12:55:44 CEST
119
Begin processing the 86th record. run: 1121 subRun: 1 event: 20986 at 09-Oct-2025 12:55:46 CEST
41
116
Begin processing the 87th record. run: 1121 subRun: 1 event: 20987 at 09-Oct-2025 12:55:47 CEST
156
Begin processing the 88th record. run: 1121 subRun: 1 event: 20988 at 09-Oct-2025 12:55:49 CEST
42
193
Begin processing the 89th record. run: 1121 subRun: 1 event: 20989 at 09-Oct-2025 12:55:53 CEST
794
144
241
41
Begin processing the 90th record. run: 1121 subRun: 1 event: 20990 at 09-Oct-2025 12:55:56 CEST
1493
Begin processing the 91st record. run: 1121 subRun: 1 event: 20991 at 09-Oct-2025 12:55:59 CEST
242
746
638
156
56
182
Begin processing the 92nd record. run: 1121 subRun: 1 event: 20992 at 09-Oct-2025 12:56:01 CEST
229
603
70
212
34
175
106
235
Begin processing the 93rd record. run: 1121 subRun: 1 event: 20993 at 09-Oct-2025 12:56:05 CEST
76
Begin processing the 94th record. run: 1121 subRun: 1 event: 20994 at 09-Oct-2025 12:56:07 CEST
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 95th record. run: 1121 subRun: 1 event: 20995 at 09-Oct-2025 12:56:07 CEST
Begin processing the 96th record. run: 1121 subRun: 1 event: 20996 at 09-Oct-2025 12:56:09 CEST
iter->second->Run() throw STATUS_CODE_FAILURE
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0097, LArDLTrackCharacterisation, STATUS_CODE_FAILURE
Begin processing the 97th record. run: 1121 subRun: 1 event: 20997 at 09-Oct-2025 12:56:11 CEST
77
244
145
42
Begin processing the 98th record. run: 1121 subRun: 1 event: 20998 at 09-Oct-2025 12:56:13 CEST
30
Begin processing the 99th record. run: 1121 subRun: 1 event: 20999 at 09-Oct-2025 12:56:15 CEST
241
71
Begin processing the 100th record. run: 1121 subRun: 1 event: 21000 at 09-Oct-2025 12:56:18 CEST
86
108
126
09-Oct-2025 12:56:20 CEST Closed output file "nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2_reco2.root"
09-Oct-2025 12:56:20 CEST Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/f6/81/nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 0.265875 18.1013 1515.93 1.61559 150.621 100
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0413407 0.0706549 0.0990487 0.0724529 0.00975577 100
reco:pandora2:StandardPandora 0.193073 18.0298 1515.86 1.54707 150.622 100
[art]:TriggerResults:TriggerResultInserter 1.189e-05 2.41658e-05 0.000108961 1.967e-05 1.48559e-05 100
end_path:out1:RootOutput 3.35e-06 5.08773e-06 2.113e-05 4.25e-06 2.26096e-06 100
end_path:out1:RootOutput(write) 0.000161951 0.000599131 0.00379093 0.000368497 0.000640502 100
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 2478.38 MB
Peak resident set size usage (VmHWM): 1410 MB
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0
total 984
-rw-r--r--. 1 duneprd np-comp 216 Oct 9 12:20 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp 0 Oct 9 12:22 debugprod.log
-rw-r--r--. 1 duneprd np-comp 47820 Oct 9 12:56 jobscript.log
-rw-r--r--. 1 duneprd np-comp 166 Oct 9 12:56 justin-processed-pfns.txt
drwxr-xr-x. 4 duneprd np-comp 60 Oct 9 12:20 larpandoracontent
-rw-r--r--. 1 duneprd np-comp 350943 Oct 9 12:56 nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2_reco2.root
-rw-r--r--. 1 duneprd np-comp 519 Oct 9 12:56 reco2_hist.root
-rw-r--r--. 1 duneprd np-comp 589837 Oct 9 12:56 trainingFile_nutau_dune10kt_1x2x6_1121_209_20230828T053323Z_gen_g4_detsim_hitreco__20240219T201349Z_reco2.root 