Jobsub ID 14065.26@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.26@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | UK_RAL-Tier1 | |
| Entry | LIGO_UK_RAL_arc_ce03 | |
| Last heartbeat | 2025-08-02 22:54:35 | |
| From worker node | Hostname | dune001-7883454.0-lcg2621.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:38:55 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 14m (892s) | |
| CPU time | 10m (643s = 72%) | |
| Max RSS bytes | 1955430400 (1864 MiB) | |
| Outputting started | 2025-08-02 22:53:48 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223907Z_8_ana_tree_hd.root | |
| Finished | 2025-08-02 22:54:35 | |
| Saved logs | justin-logs:14065.26-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
ta -> graph_output_2025-08-02T_223907Z_8_training1_CaloHitListU_graph.data
Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223907Z_8_training1_CaloHitListV_graph.data
Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223907Z_8_training1_CaloHitListW_graph.data
Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223907Z_8_training2_CaloHitListU_graph.data
Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223907Z_8_training2_CaloHitListV_graph.data
Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223907Z_8_training2_CaloHitListW_graph.data
Renamed analysiseid.root -> graph_output_2025-08-02T_223907Z_8_analysiseid.root
Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223907Z_8_ana_tree_hd.root
=== Start last 100 lines of lar log file ===
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
BeamNeutrinos:
--Primary 0, MCPDG 2212, Energy 1.97072, Dist. 109.543, nMCHits 453 (128, 235, 90)
MCPDG 2212, Energy 1.97072, Dist. 109.543, nMCHits 298 (103, 161, 34)
\_ MCPDG 2212, Energy 1.24816, Dist. 16.8346, nMCHits 51 (1, 31, 19)
\_ MCPDG 2212, Energy 0.979765, Dist. 1.68335, nMCHits 13 (2, 6, 5)
\_ MCPDG 2212, Energy 0.946193, Dist. 0.0911521, nMCHits 1 (0, 1, 0)
\_ MCPDG 211, Energy 0.160537, Dist. 2.06461, nMCHits 10 (2, 4, 4)
\_ MCPDG -11, Energy 0.04112, Dist. 12.4363, nMCHits 80 (20, 32, 28)
--Primary 1, MCPDG 22, Energy 0.230973, Dist. 3.11615, nMCHits 312 (102, 125, 85)
MCPDG 22, Energy 0.230973, Dist. 3.11615, nMCHits 312 (102, 125, 85)
--Primary 2, MCPDG 22, Energy 0.23641, Dist. 29.2167, nMCHits 187 (72, 80, 35)
MCPDG 22, Energy 0.23641, Dist. 29.2167, nMCHits 187 (72, 80, 35)
------------------------------------------------------------------------------------------------
Operating in training mode.
The eid is 99
Graph saved to training1_CaloHitListW_graph.data
Size of file training1_CaloHitListW_graph.data is 1053128 bytes.
The eid is 99
Graph saved to training1_CaloHitListU_graph.data
Size of file training1_CaloHitListU_graph.data is 1664080 bytes.
The eid is 99
Graph saved to training1_CaloHitListV_graph.data
Size of file training1_CaloHitListV_graph.data is 1527692 bytes.
Operating in inference mode.
Operating in training mode.
The eid is -1
Graph saved to training2_CaloHitListW_graph.data
Size of file training2_CaloHitListW_graph.data is 1052796 bytes.
The eid is -1
Graph saved to training2_CaloHitListU_graph.data
Size of file training2_CaloHitListU_graph.data is 1663044 bytes.
The eid is -1
Graph saved to training2_CaloHitListV_graph.data
Size of file training2_CaloHitListV_graph.data is 1526328 bytes.
Boundary wire vector sizes: 379, 505, 266
minwire 0: 612
minwire 1: 2291
minwire 2: 236
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2376, 2875
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0680631,
Output 1: 0.00681917, 0.00320895, 0.0224831, 0.967489,
Output 2: 0.314437, 0.544251, 0.0381806, 0.103131,
Output 3: 0.459233, 0.50859, 0.0289259, 0.00325108,
Output 4: 0.131752, 0.770048, 0.0951433, 0.00305621,
Output 5: 0.975737, 0.0233948, 0.00073427, 0.00013418,
Output 6: 0.0730028, 0.323635, 0.23189, 0.371473,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 221!
02-Aug-2025 22:52:06 UTC Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2_graph_2025-08-02T_223907Z.root"
02-Aug-2025 22:52:06 UTC Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-hd/1a/ca/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 1.16645 4.6274 24.972 4.68609 3.45397 100
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00170647 0.00333705 0.0115301 0.00315231 0.00120587 100
reco:gaushit:GausHitFinder 0.0158295 0.107186 1.42557 0.0438327 0.217332 100
reco:spsolve:SpacePointSolver 0.000191046 0.140523 4.91529 0.00663056 0.564524 100
reco:hitfd:DisambigFromSpacePoints 0.000277139 0.0298558 0.86935 0.00193584 0.115331 100
reco:rns:RandomNumberSaver 2.5741e-05 6.20866e-05 0.000639509 4.97565e-05 6.26369e-05 100
reco:pandora:StandardPandora 0.991219 2.20436 15.6208 1.75299 2.04389 100
reco:pandoraTrack:LArPandoraTrackCreation 0.000147714 0.000364701 0.00492396 0.000287174 0.000473453 100
reco:pandoraShower:LArPandoraModularShowerCreation 0.000177025 0.000382622 0.00502891 0.000317895 0.000475342 100
reco:pandoracalo:Calorimetry 0.000122984 0.000286707 0.00239748 0.000238162 0.000247639 100
reco:pandorapid:Chi2ParticleID 4.0471e-05 9.18316e-05 0.00147802 7.3897e-05 0.000141168 100
reco:cvnmap:CVNMapper 4.5672e-05 0.0345347 0.114172 0.0376303 0.0285811 100
reco:cvneva:CVNEvaluator 2.061e-05 1.40455 5.73121 1.80208 1.12353 100
reco:energyrecnumu:EnergyReco 0.00178031 0.00688641 0.0401763 0.00506779 0.00509027 100
reco:energyrecnue:EnergyReco 0.00181576 0.00434809 0.0137812 0.0038934 0.00208542 100
reco:energyrecnc:EnergyReco 0.00177892 0.00424222 0.0138991 0.00368703 0.002155 100
reco:energyrecnumurange:EnergyReco 0.00178686 0.00420316 0.0137223 0.00368786 0.00209659 100
reco:energyrecnumumcs:EnergyReco 0.00179669 0.00412829 0.0136299 0.00366141 0.00200333 100
reco:opdec:Deconvolution 0.0304911 0.194775 0.385878 0.191049 0.0845858 100
reco:ophitspe:OpHitFinderDeco 0.00716317 0.0155168 0.0312497 0.0148829 0.00505233 100
reco:opflash:OpFlashFinder 0.000201926 0.000750577 0.00893764 0.000609744 0.000871264 100
reco:opslicer:OpSlicer 5.3131e-05 0.0174224 0.0913271 0.0100684 0.0202892 100
[art]:TriggerResults:TriggerResultInserter 1.283e-05 3.81847e-05 0.000216907 3.1131e-05 2.88478e-05 100
end_path:out1:RootOutput 4.17e-06 1.00626e-05 9.6243e-05 7.3405e-06 1.04775e-05 100
end_path:out1:RootOutput(write) 0.0187268 0.447525 1.05184 0.598875 0.29388 100
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 37971.4 MB
Peak resident set size usage (VmHWM): 1955.43 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
14065.26_dunegpschedd02.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2_graph_2025-08-02T_223907Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2_graph_2025-08-02T_223907Z.root
debugprod.log
graph_output_2025-08-02T_223907Z_8_ana_tree_hd.root
graph_output_2025-08-02T_223907Z_8_analysiseid.root
graph_output_2025-08-02T_223907Z_8_training1_CaloHitListU_graph.data
graph_output_2025-08-02T_223907Z_8_training1_CaloHitListV_graph.data
graph_output_2025-08-02T_223907Z_8_training1_CaloHitListW_graph.data
graph_output_2025-08-02T_223907Z_8_training2_CaloHitListU_graph.data
graph_output_2025-08-02T_223907Z_8_training2_CaloHitListV_graph.data
graph_output_2025-08-02T_223907Z_8_training2_CaloHitListW_graph.data
jobscript.log
justin-processed-pfns.txt
reco_hist.root
secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2_graph_2025-08-02T_223907Z.log
third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6483936_669_20231207T093245Z_gen_g4_detsim_hitreco__20240510T041454Z_reco2_graph_2025-08-02T_223907Z.log
training1_CaloHitListU.csv
training1_CaloHitListV.csv
training1_CaloHitListW.csv
training2_CaloHitListU.csv
training2_CaloHitListV.csv
training2_CaloHitListW.csv