Jobsub ID 14065.61@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.61@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | UK_RAL-Tier1 | |
| Entry | LIGO_UK_RAL_arc_ce01 | |
| Last heartbeat | 2025-08-02 22:49:11 | |
| From worker node | Hostname | dune001-8415284.0-host-172-16-102-138.nubes.stfc.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:39:09 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 9m (554s) | |
| CPU time | 7m (434s = 78%) | |
| Max RSS bytes | 1957580800 (1866 MiB) | |
| Outputting started | 2025-08-02 22:48:24 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223924Z_6_ana_tree_hd.root | |
| Finished | 2025-08-02 22:49:11 | |
| Saved logs | justin-logs:14065.61-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
_6_training1_CaloHitListV_graph.data
Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223924Z_6_training1_CaloHitListW_graph.data
Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223924Z_6_training2_CaloHitListU_graph.data
Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223924Z_6_training2_CaloHitListV_graph.data
Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223924Z_6_training2_CaloHitListW_graph.data
Renamed analysiseid.root -> graph_output_2025-08-02T_223924Z_6_analysiseid.root
Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223924Z_6_ana_tree_hd.root
=== Start last 100 lines of lar log file ===
\_ MCPDG 2212, Energy 0.954593, Dist. 0.322062, nMCHits 1 (0, 1, 0)
\_ MCPDG -211, Energy 0.679792, Dist. 14.9551, nMCHits 18 (2, 5, 11)
\_ MCPDG 2212, Energy 1.14652, Dist. 5.81986, nMCHits 11 (5, 2, 4)
\_ MCPDG 1000010020, Energy 1.96521, Dist. 3.80997, nMCHits 12 (3, 7, 2)
\_ MCPDG 2212, Energy 0.991164, Dist. 2.54542, nMCHits 9 (3, 5, 1)
\_ MCPDG 2212, Energy 1.12384, Dist. 23.0818, nMCHits 96 (35, 27, 34)
\_ MCPDG -211, Energy 0.547428, Dist. 50.9694, nMCHits 81 (39, 32, 10)
\_ MCPDG 1000010020, Energy 1.90825, Dist. 0.638946, nMCHits 2 (1, 1, 0)
\_ MCPDG 211, Energy 0.214027, Dist. 16.8612, nMCHits 52 (29, 10, 13)
\_ MCPDG -11, Energy 0.0343209, Dist. 12.1192, nMCHits 41 (16, 10, 15)
--Primary 1, MCPDG -211, Energy 2.58437, Dist. 15.1258, nMCHits 275 (124, 79, 72)
MCPDG -211, Energy 2.58437, Dist. 15.1258, nMCHits 44 (27, 5, 12)
\_ MCPDG -211, Energy 2.53233, Dist. 21.7659, nMCHits 97 (47, 26, 24)
\_ MCPDG 22, Energy 0.134667, Dist. 1.13841, nMCHits 134 (50, 48, 36)
--Primary 2, MCPDG 2212, Energy 1.1761, Dist. 35.1279, nMCHits 122 (19, 59, 44)
MCPDG 2212, Energy 1.1761, Dist. 35.1279, nMCHits 122 (19, 59, 44)
------------------------------------------------------------------------------------------------
Operating in training mode.
The eid is 96
Graph saved to training1_CaloHitListW_graph.data
Size of file training1_CaloHitListW_graph.data is 949392 bytes.
The eid is 96
Graph saved to training1_CaloHitListU_graph.data
Size of file training1_CaloHitListU_graph.data is 1309896 bytes.
The eid is 96
Graph saved to training1_CaloHitListV_graph.data
Size of file training1_CaloHitListV_graph.data is 1257348 bytes.
Operating in inference mode.
Operating in training mode.
The eid is -1
Graph saved to training2_CaloHitListW_graph.data
Size of file training2_CaloHitListW_graph.data is 948788 bytes.
The eid is -1
Graph saved to training2_CaloHitListU_graph.data
Size of file training2_CaloHitListU_graph.data is 1309096 bytes.
The eid is -1
Graph saved to training2_CaloHitListV_graph.data
Size of file training2_CaloHitListV_graph.data is 1255896 bytes.
Boundary wire vector sizes: 3043, 2073, 2178
minwire 0: 950
minwire 1: 544
minwire 2: 875
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0349138,
Output 1: 0.0412771, 0.00890873, 0.0966276, 0.853187,
Output 2: 0.043185, 0.151091, 0.800981, 0.00474247,
Output 3: 0.468641, 0.273517, 0.137854, 0.119988,
Output 4: 0.117533, 0.296715, 0.32277, 0.262983,
Output 5: 0.681175, 0.292988, 0.0230172, 0.00282008,
Output 6: 0.035389, 0.266332, 0.217703, 0.480577,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 444!
02-Aug-2025 22:47:13 UTC Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2_graph_2025-08-02T_223924Z.root"
02-Aug-2025 22:47:13 UTC Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/ae/11/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 0.0821606 2.77023 22.7824 2.86387 2.35596 100
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00151106 0.00396754 0.0163674 0.00345666 0.00235067 100
reco:gaushit:GausHitFinder 0.0163913 0.0719016 0.964521 0.0371657 0.10923 100
reco:spsolve:SpacePointSolver 9.7702e-05 0.0982005 3.95467 0.00417623 0.500554 100
reco:hitfd:DisambigFromSpacePoints 0.000225845 0.0147825 0.670753 0.00124964 0.0725192 100
reco:rns:RandomNumberSaver 2.073e-05 4.15839e-05 0.000373169 3.46455e-05 3.60538e-05 100
reco:pandora:StandardPandora 0.0113375 1.15051 14.5187 0.950706 1.39017 100
reco:pandoraTrack:LArPandoraTrackCreation 0.000128134 0.000248568 0.00326069 0.00020653 0.000309742 100
reco:pandoraShower:LArPandoraModularShowerCreation 0.000148673 0.000291943 0.00416771 0.000247446 0.000393677 100
reco:pandoracalo:Calorimetry 9.9673e-05 0.000191355 0.00203871 0.000173328 0.000188292 100
reco:pandorapid:Chi2ParticleID 3.626e-05 9.48934e-05 0.00219266 5.8877e-05 0.00024355 100
reco:cvnmap:CVNMapper 2.1201e-05 0.012158 0.06384 0.0136062 0.0102555 100
reco:cvneva:CVNEvaluator 1.549e-05 0.825216 3.19153 1.14542 0.580808 100
reco:energyrecnumu:EnergyReco 0.00173551 0.00490495 0.0148796 0.00388388 0.00264568 100
reco:energyrecnue:EnergyReco 0.00153269 0.00334485 0.0146267 0.00299431 0.0016335 100
reco:energyrecnc:EnergyReco 0.00150587 0.00325746 0.0144227 0.00292887 0.00138788 100
reco:energyrecnumurange:EnergyReco 0.00152628 0.0032782 0.0146613 0.00297265 0.00145844 100
reco:energyrecnumumcs:EnergyReco 0.00154904 0.00329768 0.0153353 0.00289512 0.00184408 100
reco:opdec:Deconvolution 0.0083257 0.158643 0.289479 0.160431 0.0683188 100
reco:ophitspe:OpHitFinderDeco 0.00882131 0.0194231 0.0326091 0.0189604 0.00475021 100
reco:opflash:OpFlashFinder 0.000138974 0.00051131 0.00396522 0.000402639 0.00047309 100
reco:opslicer:OpSlicer 4.5452e-05 0.0176053 0.0833667 0.00718639 0.0208113 100
[art]:TriggerResults:TriggerResultInserter 9.97e-06 2.30762e-05 0.000293527 1.86905e-05 2.84729e-05 100
end_path:out1:RootOutput 3.46e-06 7.02603e-06 2.7141e-05 5.9095e-06 3.34822e-06 100
end_path:out1:RootOutput(write) 0.00529096 0.376861 1.02131 0.485416 0.228629 100
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 20257.8 MB
Peak resident set size usage (VmHWM): 1957.58 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
14065.61_dunegpschedd02.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2_graph_2025-08-02T_223924Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2_graph_2025-08-02T_223924Z.root
debugprod.log
graph_output_2025-08-02T_223924Z_6_ana_tree_hd.root
graph_output_2025-08-02T_223924Z_6_analysiseid.root
graph_output_2025-08-02T_223924Z_6_training1_CaloHitListU_graph.data
graph_output_2025-08-02T_223924Z_6_training1_CaloHitListV_graph.data
graph_output_2025-08-02T_223924Z_6_training1_CaloHitListW_graph.data
graph_output_2025-08-02T_223924Z_6_training2_CaloHitListU_graph.data
graph_output_2025-08-02T_223924Z_6_training2_CaloHitListV_graph.data
graph_output_2025-08-02T_223924Z_6_training2_CaloHitListW_graph.data
jobscript.log
justin-processed-pfns.txt
reco_hist.root
secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2_graph_2025-08-02T_223924Z.log
third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_843_20231202T073430Z_gen_g4_detsim_hitreco__20240508T035744Z_reco2_graph_2025-08-02T_223924Z.log
training1_CaloHitListU.csv
training1_CaloHitListV.csv
training1_CaloHitListW.csv
training2_CaloHitListU.csv
training2_CaloHitListV.csv
training2_CaloHitListW.csv