Jobsub ID 292861.11@dunegpschedd02.fnal.gov
| Jobsub ID | 292861.11@dunegpschedd02.fnal.gov |
| Workflow ID | 12170 |
| Stage ID | 1 |
| User name | lwhite86@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 3600 (1 hours) |
| Submitted time | 2026-01-21 15:04:52 |
| Site | UK_QMUL |
| Entry | DUNE_UK_London_QMUL_arcce02 |
| Last heartbeat | 2026-01-21 15:14:08 |
| From worker node | Hostname | cn521.htc.esc.qmul |
| cpuinfo | Intel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-21 15:05:26 |
| Input files | fardet-hd:nu_dune10kt_1x2x6_1100_126_20230826T033413Z_gen_g4_detsim_hitreco__20240229T190036Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 6m (370s) |
| CPU time | 5m (340s = 91%) |
| Max RSS bytes | 1604669440 (1530 MiB) |
| Outputting started | 2026-01-21 15:11:37 |
| Output files | |
| Finished | 2026-01-21 15:14:08 |
| Saved logs | justin-logs:292861.11-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
384, 0.00332259, 0.995776, 0.000717125
Begin processing the 74th record. run: 1100 subRun: 1 event: 12674 at 21-Jan-2026 15:10:26 GMT
TrackPID: -211, 0.0709029, 0.439556, 0.481593, 0.00794896
TrackPID: 2212, 0.000136301, 0.00375447, 0.995231, 0.000877813
TrackPID: 2212, 0.00089887, 0.014619, 0.981987, 0.00249482
TrackPID: -211, 0.00139611, 0.0397899, 0.956794, 0.00201968
TrackPID: -211, 0.0610846, 0.460068, 0.471177, 0.00767029
TrackPID: -211, 0.0293561, 0.856957, 0.104038, 0.00964889
TrackPID: 2212, 7.11648e-05, 0.00181925, 0.997634, 0.000475821
Begin processing the 75th record. run: 1100 subRun: 1 event: 12675 at 21-Jan-2026 15:10:28 GMT
TrackPID: 13, 0.993846, 0.00501892, 0.0010243, 0.000110753
TrackPID: 1000010020, 0.000258724, 0.00742102, 0.991652, 0.000668024
Begin processing the 76th record. run: 1100 subRun: 1 event: 12676 at 21-Jan-2026 15:10:31 GMT
TrackPID: 13, 0.180699, 0.618452, 0.192585, 0.00826348
TrackPID: 211, 0.0178521, 0.422472, 0.549802, 0.00987372
Begin processing the 77th record. run: 1100 subRun: 1 event: 12677 at 21-Jan-2026 15:10:32 GMT
TrackPID: 13, 0.697613, 0.149299, 0.146978, 0.00610981
Begin processing the 78th record. run: 1100 subRun: 1 event: 12678 at 21-Jan-2026 15:10:34 GMT
TrackPID: 13, 0.310124, 0.514368, 0.172697, 0.0028108
TrackPID: 2212, 0.000504796, 0.00501662, 0.992835, 0.00164396
TrackPID: 13, 0.99968, 0.000232131, 6.64344e-05, 2.10456e-05
Begin processing the 79th record. run: 1100 subRun: 1 event: 12679 at 21-Jan-2026 15:10:37 GMT
TrackPID: 211, 0.000201503, 0.0617565, 0.925243, 0.0127988
TrackPID: 211, 0.00203457, 0.0190969, 0.97659, 0.00227844
TrackPID: 2212, 0.000171934, 0.00572245, 0.993463, 0.000642684
TrackPID: 2212, 0.0543925, 0.779502, 0.154441, 0.0116642
TrackPID: 211, 0.448981, 0.426779, 0.116105, 0.00813552
TrackPID: 13, 0.993959, 0.00434573, 0.00117935, 0.000515604
TrackPID: -211, 0.00536991, 0.226445, 0.757938, 0.0102464
TrackPID: 2212, 0.00257669, 0.032475, 0.962007, 0.00294113
TrackPID: 2212, 0.000739751, 0.00697748, 0.989949, 0.00233393
TrackPID: -211, 0.478442, 0.239095, 0.274746, 0.00771762
Begin processing the 80th record. run: 1100 subRun: 1 event: 12680 at 21-Jan-2026 15:10:43 GMT
TrackPID: 2212, 0.0001174, 0.00094733, 0.998441, 0.000494398
Begin processing the 81st record. run: 1100 subRun: 1 event: 12681 at 21-Jan-2026 15:10:45 GMT
Begin processing the 82nd record. run: 1100 subRun: 1 event: 12682 at 21-Jan-2026 15:10:47 GMT
TrackPID: 2212, 0.000218275, 0.00669917, 0.991866, 0.00121604
Begin processing the 83rd record. run: 1100 subRun: 1 event: 12683 at 21-Jan-2026 15:10:48 GMT
TrackPID: 2212, 0.000233021, 0.00233968, 0.996482, 0.000945128
TrackPID: 13, 0.534946, 0.373467, 0.0808255, 0.0107615
Begin processing the 84th record. run: 1100 subRun: 1 event: 12684 at 21-Jan-2026 15:10:50 GMT
TrackPID: 211, 0.0371156, 0.412131, 0.544251, 0.00650303
TrackPID: 2212, 0.000376126, 0.00685723, 0.991602, 0.00116485
TrackPID: 13, 0.0902005, 0.402963, 0.497142, 0.00969474
Begin processing the 85th record. run: 1100 subRun: 1 event: 12685 at 21-Jan-2026 15:10:52 GMT
TrackPID: 13, 0.134968, 0.793472, 0.0626973, 0.00886262
TrackPID: 2212, 0.000286208, 0.00270927, 0.996045, 0.000959348
Begin processing the 86th record. run: 1100 subRun: 1 event: 12686 at 21-Jan-2026 15:10:53 GMT
TrackPID: 13, 0.898823, 0.0808765, 0.0189935, 0.00130747
TrackPID: 13, 0.794594, 0.154514, 0.0442611, 0.00663141
Begin processing the 87th record. run: 1100 subRun: 1 event: 12687 at 21-Jan-2026 15:10:55 GMT
Begin processing the 88th record. run: 1100 subRun: 1 event: 12688 at 21-Jan-2026 15:10:57 GMT
TrackPID: 13, 0.997617, 0.00122701, 0.00106923, 8.69702e-05
TrackPID: 2212, 0.000346259, 0.0234313, 0.972171, 0.00405127
TrackPID: 2212, 0.0280256, 0.66233, 0.300163, 0.00948162
TrackPID: 2212, 0.0415428, 0.446428, 0.505259, 0.00677059
TrackPID: 211, 0.0003435, 0.066406, 0.927453, 0.00579738
Begin processing the 89th record. run: 1100 subRun: 1 event: 12689 at 21-Jan-2026 15:11:01 GMT
TrackPID: 13, 0.999786, 0.00015908, 4.8115e-05, 6.80521e-06
TrackPID: 2212, 0.000128204, 0.0032301, 0.996081, 0.000560841
Begin processing the 90th record. run: 1100 subRun: 1 event: 12690 at 21-Jan-2026 15:11:03 GMT
PandoraContentApi::GetList(*this, m_trackPfoListName, pTrackPfoList) return STATUS_CODE_NOT_INITIALIZED
in function: GetAllTrackFeatures
in file: /exp/dune/app/users/lwhite86/DUNE-FD/pandoraPID/srcs/larpandoracontent/larpandoradlcontent/LArTrackShowerId/DlTrackCharacterisationAlgorithm.cc line#: 223
Begin processing the 91st record. run: 1100 subRun: 1 event: 12691 at 21-Jan-2026 15:11:04 GMT
TrackPID: 211, 0.387293, 0.410393, 0.194275, 0.00804005
Begin processing the 92nd record. run: 1100 subRun: 1 event: 12692 at 21-Jan-2026 15:11:06 GMT
TrackPID: -13, 0.0859257, 0.691369, 0.217268, 0.00543743
TrackPID: 2212, 0.000244324, 0.0114977, 0.987272, 0.00098607
TrackPID: 2212, 0.241077, 0.334919, 0.411084, 0.01292
TrackPID: -211, 0.135507, 0.547831, 0.307361, 0.00930165
TrackPID: 13, 0.998799, 0.000950186, 0.000221632, 2.90471e-05
Begin processing the 93rd record. run: 1100 subRun: 1 event: 12693 at 21-Jan-2026 15:11:09 GMT
TrackPID: 13, 0.32631, 0.403178, 0.25824, 0.0122717
TrackPID: 2212, 0.000245614, 0.00355104, 0.995449, 0.000754089
Begin processing the 94th record. run: 1100 subRun: 1 event: 12694 at 21-Jan-2026 15:11:11 GMT
Begin processing the 95th record. run: 1100 subRun: 1 event: 12695 at 21-Jan-2026 15:11:13 GMT
PfoList empty... moving on.
iter->second->Run() throw STATUS_CODE_FAILURE
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0097, LArDLTrackCharacterisation, STATUS_CODE_FAILURE
Begin processing the 96th record. run: 1100 subRun: 1 event: 12696 at 21-Jan-2026 15:11:15 GMT
TrackPID: -211, 0.184145, 0.65451, 0.138831, 0.0225139
TrackPID: 321, 0.0787213, 0.590198, 0.311648, 0.0194333
TrackPID: 11, 0.0212542, 0.278805, 0.693792, 0.00614974
TrackPID: 211, 0.077816, 0.737882, 0.178745, 0.00555668
Begin processing the 97th record. run: 1100 subRun: 1 event: 12697 at 21-Jan-2026 15:11:18 GMT
TrackPID: 2212, 0.000428622, 0.00439621, 0.993849, 0.00132672
TrackPID: 13, 0.998072, 0.00159212, 0.000261302, 7.45104e-05
Begin processing the 98th record. run: 1100 subRun: 1 event: 12698 at 21-Jan-2026 15:11:20 GMT
TrackPID: 13, 0.457665, 0.214455, 0.319823, 0.00805715
TrackPID: 2212, 0.000135297, 0.00451468, 0.994821, 0.000529092
Begin processing the 99th record. run: 1100 subRun: 1 event: 12699 at 21-Jan-2026 15:11:22 GMT
TrackPID: -211, 0.0184091, 0.950832, 0.02654, 0.00421863
TrackPID: 22, 0.397753, 0.455685, 0.143923, 0.00263852
TrackPID: 211, 0.0305326, 0.590727, 0.36315, 0.0155903
TrackPID: 2212, 0.000235619, 0.00570161, 0.993179, 0.00088422
TrackPID: 211, 0.224293, 0.43441, 0.329903, 0.0113934
TrackPID: -211, 0.175234, 0.338638, 0.475638, 0.0104887
TrackPID: -211, 0.00128277, 0.121129, 0.871051, 0.00653807
Begin processing the 100th record. run: 1100 subRun: 1 event: 12700 at 21-Jan-2026 15:11:32 GMT
TrackPID: 13, 0.968974, 0.0230967, 0.00737953, 0.000549352
21-Jan-2026 15:11:34 GMT Closed output file "nu_dune10kt_1x2x6_1100_126_20230826T033413Z_gen_g4_detsim_hitreco__20240229T190036Z_reco2_reco2.root"
21-Jan-2026 15:11:34 GMT Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/36/48/nu_dune10kt_1x2x6_1100_126_20230826T033413Z_gen_g4_detsim_hitreco__20240229T190036Z_reco2.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 0.0514939 2.87077 38.1263 1.80811 4.12563 100
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00853457 0.0214638 0.0462211 0.0246333 0.00956194 100
reco:pandora2:StandardPandora 0.0417355 2.84846 38.094 1.7829 4.12432 100
[art]:TriggerResults:TriggerResultInserter 1.3141e-05 2.66425e-05 0.00010913 2.191e-05 1.40816e-05 100
end_path:out1:RootOutput 2.693e-06 3.55163e-06 1.7162e-05 3.2935e-06 1.63692e-06 100
end_path:out1:RootOutput(write) 0.000201688 0.000639705 0.00418856 0.00030287 0.00070601 100
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 2592.44 MB
Peak resident set size usage (VmHWM): 1604.67 MB
====================================================================================================
Art has completed and will exit with status 0.
lar exit code 0
total 424
-rw-r--r-- 1 pildune22 pildune 210 Jan 21 15:05 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune 0 Jan 21 15:05 debugprod.log
-rw-r--r-- 1 pildune22 pildune 60479 Jan 21 15:11 jobscript.log
-rw-r--r-- 1 pildune22 pildune 168 Jan 21 15:11 justin-processed-pfns.txt
drwxr-xr-x 4 pildune22 pildune 48 Jan 21 15:05 larpandoracontent
-rw-r--r-- 1 pildune22 pildune 359167 Jan 21 15:11 nu_dune10kt_1x2x6_1100_126_20230826T033413Z_gen_g4_detsim_hitreco__20240229T190036Z_reco2_reco2.root
-rw-r--r-- 1 pildune22 pildune 519 Jan 21 15:11 reco2_hist.root 