Jobsub ID 14065.45@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.45@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | NL_SURFsara | |
| Entry | DUNE_SurfSARA_arc01 | |
| Last heartbeat | 2025-08-02 22:46:01 | |
| From worker node | Hostname | wn-da-24.gina.surf.nl |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:39:04 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 6m (376s) | |
| CPU time | 5m (348s = 92%) | |
| Max RSS bytes | 2010607616 (1917 MiB) | |
| Outputting started | 2025-08-02 22:45:21 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223913Z_5_ana_tree_hd.root | |
| Finished | 2025-08-02 22:46:01 | |
| Saved logs | justin-logs:14065.45-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 10004.3 MB Peak resident set size usage (VmHWM): 963.387 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-08-02T_223913Z_5_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223913Z_5_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223913Z_5_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223913Z_5_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223913Z_5_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223913Z_5_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-08-02T_223913Z_5_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223913Z_5_ana_tree_hd.root === Start last 100 lines of lar log file === 141 potential space points Neighbour search... 2651 tests to find 1428 neighbours Iterating with no regularization... Begin: 7.63886e+06 0 6.69858e+06 1 6.67619e+06 2 6.67435e+06 Now with regularization... Begin: 5.21374e+06 0 5.21252e+06 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG -11, Energy 0.241241, Dist. 22.4419, nMCHits 322 (107, 120, 95) MCPDG -11, Energy 0.241241, Dist. 22.4419, nMCHits 322 (107, 120, 95) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 98 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 583812 bytes. The eid is 98 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 769912 bytes. The eid is 98 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 775436 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 583360 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 769604 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 774544 bytes. Boundary wire vector sizes: 152, 162, 130 minwire 0: 772 minwire 1: 2492 minwire 2: 235 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0191535, Output 1: 0.00373072, 0.989247, 0.00245736, 0.00456477, Output 2: 0.946084, 0.0169744, 0.00299647, 0.0339451, Output 3: 0.157402, 0.809875, 0.0319489, 0.000773932, Output 4: 0.996478, 0.00341222, 3.51876e-05, 7.41151e-05, Output 5: 0.970592, 0.0277293, 0.000269242, 0.00140907, Output 6: 0.00298031, 0.0221938, 0.0461374, 0.928689, Running Ophitfinder with InputDigiType = 'recob' Found hits: 181! 03-Aug-2025 00:44:32 CEST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2_graph_2025-08-02T_223913Z.root" 03-Aug-2025 00:44:32 CEST Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/62/f6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.172414 2.33482 12.1811 2.57819 1.29337 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.00126491 0.0027556 0.0262639 0.00226777 0.00298559 100 reco:gaushit:GausHitFinder 0.0157844 0.0461943 0.780003 0.0257473 0.0832395 100 reco:spsolve:SpacePointSolver 9.9051e-05 0.0174927 0.517495 0.00246705 0.0599811 100 reco:hitfd:DisambigFromSpacePoints 0.000208842 0.00613983 0.299195 0.000825196 0.030747 100 reco:rns:RandomNumberSaver 1.8546e-05 2.57974e-05 0.000275119 2.1997e-05 2.53526e-05 100 reco:pandora:StandardPandora 0.0090213 1.04922 8.44942 0.933991 0.767477 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000125382 0.00016382 0.00205884 0.000136683 0.000192332 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000166869 0.000218746 0.00312467 0.000183798 0.000292524 100 reco:pandoracalo:Calorimetry 0.000103889 0.000132644 0.00137626 0.000115507 0.000125656 100 reco:pandorapid:Chi2ParticleID 3.6079e-05 4.94922e-05 0.000721076 3.96865e-05 6.93346e-05 100 reco:cvnmap:CVNMapper 2.044e-05 0.0140755 0.0439193 0.0187584 0.0102334 100 reco:cvneva:CVNEvaluator 1.6252e-05 0.689513 2.8353 0.978142 0.507494 100 reco:energyrecnumu:EnergyReco 0.00216148 0.00390672 0.00990976 0.00303307 0.00187064 100 reco:energyrecnue:EnergyReco 0.0021079 0.00276726 0.00998966 0.00255601 0.000889361 100 reco:energyrecnc:EnergyReco 0.00215815 0.00273659 0.00992313 0.00249176 0.000908603 100 reco:energyrecnumurange:EnergyReco 0.00216601 0.00275089 0.00979491 0.00249678 0.000928279 100 reco:energyrecnumumcs:EnergyReco 0.00220129 0.00273922 0.010001 0.00246028 0.000942094 100 reco:opdec:Deconvolution 0.0323032 0.161944 0.319038 0.1596 0.0605158 100 reco:ophitspe:OpHitFinderDeco 0.00678621 0.0142945 0.0919577 0.0113979 0.0122843 100 reco:opflash:OpFlashFinder 0.000109401 0.000335285 0.00301467 0.000299486 0.00028293 100 reco:opslicer:OpSlicer 3.4396e-05 0.0105415 0.0645088 0.00580791 0.0132826 100 [art]:TriggerResults:TriggerResultInserter 1.0901e-05 1.49182e-05 6.3292e-05 1.34865e-05 6.50488e-06 100 end_path:out1:RootOutput 3.296e-06 4.63811e-06 1.6161e-05 4.4145e-06 1.37379e-06 100 end_path:out1:RootOutput(write) 0.0166678 0.305868 0.664972 0.404081 0.180614 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 11603.9 MB Peak resident set size usage (VmHWM): 2010.61 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 14065.45_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2_graph_2025-08-02T_223913Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2_graph_2025-08-02T_223913Z.root debugprod.log graph_output_2025-08-02T_223913Z_5_ana_tree_hd.root graph_output_2025-08-02T_223913Z_5_analysiseid.root graph_output_2025-08-02T_223913Z_5_training1_CaloHitListU_graph.data graph_output_2025-08-02T_223913Z_5_training1_CaloHitListV_graph.data graph_output_2025-08-02T_223913Z_5_training1_CaloHitListW_graph.data graph_output_2025-08-02T_223913Z_5_training2_CaloHitListU_graph.data graph_output_2025-08-02T_223913Z_5_training2_CaloHitListV_graph.data graph_output_2025-08-02T_223913Z_5_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2_graph_2025-08-02T_223913Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419587_577_20231204T004037Z_gen_g4_detsim_hitreco__20240509T205459Z_reco2_graph_2025-08-02T_223913Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv