Jobsub ID 292861.13@dunegpschedd02.fnal.gov
| Jobsub ID | 292861.13@dunegpschedd02.fnal.gov |
| Workflow ID | 12170 |
| Stage ID | 1 |
| User name | lwhite86@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 3600 (1 hours) |
| Submitted time | 2026-01-21 15:04:52 |
| Site | UK_QMUL |
| Entry | DUNE_UK_London_QMUL_arcce03 |
| Last heartbeat | 2026-01-21 15:21:45 |
| From worker node | Hostname | cn078.htc.esc.qmul |
| cpuinfo | Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-21 15:05:34 |
| Input files | fardet-hd:nutau_dune10kt_1x2x6_1103_69_20230826T083314Z_gen_g4_detsim_hitreco__20240219T201936Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 13m (819s) |
| CPU time | 11m (712s = 86%) |
| Max RSS bytes | 1761902592 (1680 MiB) |
| Outputting started | 2026-01-21 15:19:14 |
| Output files | |
| Finished | 2026-01-21 15:21:45 |
| Saved logs | justin-logs:292861.13-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
D: -13, 0.99998, 1.42723e-05, 3.8499e-06, 1.85931e-06
TrackPID: 2212, 0.00183138, 0.021007, 0.973639, 0.00352294
Begin processing the 80th record. run: 1103 subRun: 1 event: 6980 at 21-Jan-2026 15:17:48 GMT
TrackPID: -211, 0.00322937, 0.105108, 0.888936, 0.00272694
TrackPID: 211, 0.139102, 0.654994, 0.171661, 0.0342431
Begin processing the 81st record. run: 1103 subRun: 1 event: 6981 at 21-Jan-2026 15:18:04 GMT
TrackPID: 2212, 0.000662288, 0.00194159, 0.996489, 0.000907132
TrackPID: 2212, 0.000182428, 0.00615675, 0.993069, 0.00059171
Begin processing the 82nd record. run: 1103 subRun: 1 event: 6982 at 21-Jan-2026 15:18:06 GMT
TrackPID: -11, 0.144911, 0.666227, 0.182784, 0.00607824
Begin processing the 83rd record. run: 1103 subRun: 1 event: 6983 at 21-Jan-2026 15:18:08 GMT
TrackPID: 22, 0.084731, 0.743076, 0.166482, 0.00571062
Begin processing the 84th record. run: 1103 subRun: 1 event: 6984 at 21-Jan-2026 15:18:10 GMT
TrackPID: -211, 0.189031, 0.640706, 0.153995, 0.016268
TrackPID: 321, 0.0623356, 0.705506, 0.212877, 0.0192812
TrackPID: 2212, 0.000240773, 0.00345752, 0.995166, 0.00113606
TrackPID: 211, 0.925037, 0.0616631, 0.0118122, 0.00148817
TrackPID: 2212, 0.000132618, 0.00200204, 0.997373, 0.000492541
TrackPID: 2212, 0.000193792, 0.00187742, 0.99732, 0.000608527
TrackPID: -211, 0.12961, 0.690899, 0.172014, 0.00747665
TrackPID: 211, 0.41852, 0.412441, 0.159879, 0.0091599
Begin processing the 85th record. run: 1103 subRun: 1 event: 6985 at 21-Jan-2026 15:18:16 GMT
TrackPID: 2212, 0.000320401, 0.00330274, 0.995497, 0.000879414
Begin processing the 86th record. run: 1103 subRun: 1 event: 6986 at 21-Jan-2026 15:18:19 GMT
TrackPID: 22, 0.0253349, 0.411119, 0.557661, 0.005886
TrackPID: 2212, 0.0917638, 0.519261, 0.381231, 0.00774455
Begin processing the 87th record. run: 1103 subRun: 1 event: 6987 at 21-Jan-2026 15:18:22 GMT
Begin processing the 88th record. run: 1103 subRun: 1 event: 6988 at 21-Jan-2026 15:18:24 GMT
TrackPID: -211, 0.0865395, 0.729395, 0.178106, 0.00595923
TrackPID: -211, 0.0186923, 0.959808, 0.0186859, 0.00281366
TrackPID: 2212, 0.0789448, 0.503591, 0.410514, 0.00695078
TrackPID: 22, 0.0952254, 0.876891, 0.0253403, 0.00254303
TrackPID: 211, 0.113191, 0.624001, 0.255279, 0.00752971
TrackPID: 2212, 0.00192073, 0.018955, 0.975743, 0.00338149
TrackPID: 2212, 0.0158142, 0.711205, 0.250684, 0.0222975
Begin processing the 89th record. run: 1103 subRun: 1 event: 6989 at 21-Jan-2026 15:18:27 GMT
PfoList empty... moving on.
iter->second->Run() throw STATUS_CODE_FAILURE
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0097, LArDLTrackCharacterisation, STATUS_CODE_FAILURE
Begin processing the 90th record. run: 1103 subRun: 1 event: 6990 at 21-Jan-2026 15:18:29 GMT
TrackPID: -211, 0.0373888, 0.559147, 0.386412, 0.0170524
TrackPID: -211, 0.359725, 0.455701, 0.177198, 0.00737477
TrackPID: 211, 0.148839, 0.679494, 0.165741, 0.00592534
TrackPID: 22, 0.15432, 0.606744, 0.232779, 0.00615816
TrackPID: 2212, 0.00383853, 0.0489738, 0.942505, 0.00468271
TrackPID: 2212, 0.0485974, 0.281525, 0.661518, 0.0083595
TrackPID: -211, 0.103734, 0.822253, 0.0703177, 0.00369444
TrackPID: -211, 0.0729683, 0.435869, 0.483017, 0.00814568
TrackPID: 211, 0.134267, 0.595303, 0.262762, 0.00766718
Begin processing the 91st record. run: 1103 subRun: 1 event: 6991 at 21-Jan-2026 15:18:41 GMT
TrackPID: 211, 0.107497, 0.724275, 0.162014, 0.00621337
TrackPID: 2212, 0.000274461, 0.0049419, 0.994067, 0.000716337
TrackPID: 2212, 0.000127603, 0.00159824, 0.997777, 0.000497445
Begin processing the 92nd record. run: 1103 subRun: 1 event: 6992 at 21-Jan-2026 15:18:44 GMT
TrackPID: 2212, 0.000942995, 0.047046, 0.945806, 0.00620538
TrackPID: 2212, 0.00141724, 0.246551, 0.737764, 0.014267
TrackPID: 211, 0.0121788, 0.267288, 0.715474, 0.00505916
TrackPID: 2212, 0.000322547, 0.00308377, 0.995773, 0.000820985
TrackPID: -211, 0.0167124, 0.32384, 0.654188, 0.00525886
TrackPID: 2212, 0.000459131, 0.0151322, 0.983252, 0.00115658
TrackPID: 321, 0.954845, 0.0405408, 0.00350209, 0.0011123
TrackPID: 22, 0.0363917, 0.42982, 0.527126, 0.00666168
TrackPID: 2212, 5.70236e-05, 0.0022963, 0.997248, 0.000398875
TrackPID: 211, 0.189495, 0.6717, 0.132237, 0.00656776
Begin processing the 93rd record. run: 1103 subRun: 1 event: 6993 at 21-Jan-2026 15:18:50 GMT
PfoList empty... moving on.
iter->second->Run() throw STATUS_CODE_FAILURE
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0097, LArDLTrackCharacterisation, STATUS_CODE_FAILURE
Begin processing the 94th record. run: 1103 subRun: 1 event: 6994 at 21-Jan-2026 15:18:52 GMT
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 95th record. run: 1103 subRun: 1 event: 6995 at 21-Jan-2026 15:18:53 GMT
TrackPID: 2212, 0.00761639, 0.120372, 0.869128, 0.00288379
Begin processing the 96th record. run: 1103 subRun: 1 event: 6996 at 21-Jan-2026 15:18:55 GMT
Begin processing the 97th record. run: 1103 subRun: 1 event: 6997 at 21-Jan-2026 15:18:57 GMT
TrackPID: 2212, 0.00042002, 0.0119408, 0.986227, 0.00141195
Begin processing the 98th record. run: 1103 subRun: 1 event: 6998 at 21-Jan-2026 15:19:00 GMT
TrackPID: 211, 0.0119693, 0.401989, 0.576329, 0.0097123
Begin processing the 99th record. run: 1103 subRun: 1 event: 6999 at 21-Jan-2026 15:19:02 GMT
TrackPID: -211, 0.131498, 0.617727, 0.243223, 0.00755251
TrackPID: 2212, 0.104628, 0.45327, 0.432974, 0.00912759
TrackPID: 211, 0.169963, 0.635308, 0.186885, 0.00784427
TrackPID: -211, 0.157333, 0.49694, 0.33658, 0.00914681
Begin processing the 100th record. run: 1103 subRun: 1 event: 7000 at 21-Jan-2026 15:19:05 GMT
TrackPID: -211, 0.268073, 0.498322, 0.211378, 0.0222278
TrackPID: 2212, 0.208297, 0.122517, 0.659727, 0.00945888
TrackPID: -211, 0.0717087, 0.74768, 0.175433, 0.00517823
TrackPID: 1000010020, 0.000320836, 0.00369414, 0.995174, 0.000811423
TrackPID: 211, 0.0350424, 0.412937, 0.531529, 0.0204916
TrackPID: 2212, 8.86721e-05, 0.00164505, 0.997772, 0.000494581
21-Jan-2026 15:19:12 GMT Closed output file "nutau_dune10kt_1x2x6_1103_69_20230826T083314Z_gen_g4_detsim_hitreco__20240219T201936Z_reco2_reco2.root"
21-Jan-2026 15:19:12 GMT Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/a4/b5/nutau_dune10kt_1x2x6_1103_69_20230826T083314Z_gen_g4_detsim_hitreco__20240219T201936Z_reco2.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 0.116724 6.74615 102.079 2.69497 15.5269 100
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0174709 0.124342 0.406552 0.121322 0.0590541 100
reco:pandora2:StandardPandora 0.0658329 6.61981 101.844 2.59188 15.5172 100
[art]:TriggerResults:TriggerResultInserter 1.3095e-05 3.68402e-05 0.000563373 2.58085e-05 5.62126e-05 100
end_path:out1:RootOutput 2.409e-06 3.522e-06 2.1131e-05 3.266e-06 1.851e-06 100
end_path:out1:RootOutput(write) 0.000215727 0.00176624 0.00883803 0.0010912 0.00189852 100
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 2744.36 MB
Peak resident set size usage (VmHWM): 1761.9 MB
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0
total 424
-rw-r--r-- 1 pildune32 pildune 214 Jan 21 15:05 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune 0 Jan 21 15:06 debugprod.log
-rw-r--r-- 1 pildune32 pildune 60699 Jan 21 15:19 jobscript.log
-rw-r--r-- 1 pildune32 pildune 168 Jan 21 15:19 justin-processed-pfns.txt
drwxr-xr-x 4 pildune32 pildune 48 Jan 21 15:05 larpandoracontent
-rw-r--r-- 1 pildune32 pildune 359149 Jan 21 15:19 nutau_dune10kt_1x2x6_1103_69_20230826T083314Z_gen_g4_detsim_hitreco__20240219T201936Z_reco2_reco2.root
-rw-r--r-- 1 pildune32 pildune 519 Jan 21 15:19 reco2_hist.root 