Jobsub ID 14065.41@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.41@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | NL_SURFsara | |
| Entry | DUNE_SurfSARA_arc03 | |
| Last heartbeat | 2025-08-02 22:49:24 | |
| From worker node | Hostname | wn-la-16.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:39:06 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 9m (565s) | |
| CPU time | 8m (535s = 94%) | |
| Max RSS bytes | 2231001088 (2127 MiB) | |
| Outputting started | 2025-08-02 22:48:32 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223917Z_7_ana_tree_hd.root | |
| Finished | 2025-08-02 22:49:24 | |
| Saved logs | justin-logs:14065.41-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
ill exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-08-02T_223917Z_7_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223917Z_7_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223917Z_7_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223917Z_7_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223917Z_7_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223917Z_7_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-08-02T_223917Z_7_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223917Z_7_ana_tree_hd.root === Start last 100 lines of lar log file === 0 4.65965e+09 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 11, Energy 8.9554, Dist. 43.9849, nMCHits 5103 (1910, 2070, 1123) MCPDG 11, Energy 8.9554, Dist. 43.9849, nMCHits 5103 (1910, 2070, 1123) --Primary 1, MCPDG 211, Energy 0.479828, Dist. 88.6188, nMCHits 518 (142, 185, 191) MCPDG 211, Energy 0.479828, Dist. 88.6188, nMCHits 426 (114, 152, 160) \_ MCPDG 2212, Energy 1.09623, Dist. 17.4441, nMCHits 70 (18, 26, 26) \_ MCPDG 2212, Energy 1.00253, Dist. 3.55359, nMCHits 22 (10, 7, 5) --Primary 2, MCPDG 22, Energy 0.455029, Dist. 4.34999, nMCHits 471 (217, 86, 168) MCPDG 22, Energy 0.455029, Dist. 4.34999, nMCHits 471 (217, 86, 168) --Primary 3, MCPDG 22, Energy 0.054361, Dist. 27.4525, nMCHits 79 (26, 31, 22) MCPDG 22, Energy 0.054361, Dist. 27.4525, nMCHits 79 (26, 31, 22) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 98 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 1644908 bytes. The eid is 98 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 2303188 bytes. The eid is 98 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 2210228 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 1644624 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 2301052 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 2207828 bytes. Boundary wire vector sizes: 2407, 2325, 1538 minwire 0: 2476 minwire 1: 229 minwire 2: 2169 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0834482, Output 1: 0.000579351, 0.00390533, 0.0092901, 0.986225, Output 2: 0.245077, 0.487437, 0.0237528, 0.243733, Output 3: 0.91364, 0.0681648, 0.00861809, 0.00957723, Output 4: 0.92208, 0.075041, 0.00256494, 0.000313591, Output 5: 0.939322, 0.0560279, 0.00399604, 0.000653949, Output 6: 0.094146, 0.744969, 0.137846, 0.0230395, Running Ophitfinder with InputDigiType = 'recob' Found hits: 405! 03-Aug-2025 00:47:13 CEST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2_graph_2025-08-02T_223917Z.root" 03-Aug-2025 00:47:13 CEST Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/6d/41/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.0840114 3.69135 22.4772 3.23595 3.35329 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.00213985 0.0042056 0.02115 0.00409709 0.0019689 100 reco:gaushit:GausHitFinder 0.01904 0.0956853 0.90389 0.0394427 0.154083 100 reco:spsolve:SpacePointSolver 0.000120089 0.122442 4.31443 0.00837965 0.495877 100 reco:hitfd:DisambigFromSpacePoints 0.000218437 0.0285894 0.631258 0.00199488 0.0999328 100 reco:rns:RandomNumberSaver 2.2022e-05 3.32701e-05 0.000346308 2.86085e-05 3.21331e-05 100 reco:pandora:StandardPandora 0.0156269 1.87302 16.8937 1.29578 2.34627 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000144537 0.000230595 0.0034877 0.000180305 0.000332085 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000196404 0.000250087 0.00324412 0.000212877 0.00030182 100 reco:pandoracalo:Calorimetry 0.000118237 0.000167443 0.00164118 0.00014364 0.00015211 100 reco:pandorapid:Chi2ParticleID 4.1712e-05 5.48341e-05 0.00080962 4.4622e-05 7.63289e-05 100 reco:cvnmap:CVNMapper 2.8342e-05 0.0246038 0.104191 0.0289227 0.0191268 100 reco:cvneva:CVNEvaluator 2.018e-05 0.865055 3.19621 1.08324 0.517464 100 reco:energyrecnumu:EnergyReco 0.00210035 0.00654172 0.0211195 0.00683124 0.00408274 100 reco:energyrecnue:EnergyReco 0.00218344 0.00332765 0.0204869 0.00257831 0.00249269 100 reco:energyrecnc:EnergyReco 0.00204502 0.00312102 0.0129753 0.00246349 0.00186369 100 reco:energyrecnumurange:EnergyReco 0.00207833 0.0033838 0.020558 0.00250332 0.002853 100 reco:energyrecnumumcs:EnergyReco 0.00217094 0.00350537 0.0161362 0.00249707 0.00285673 100 reco:opdec:Deconvolution 0.00859423 0.193384 0.354591 0.189501 0.070071 100 reco:ophitspe:OpHitFinderDeco 0.016064 0.0224815 0.0393897 0.0219906 0.00327701 100 reco:opflash:OpFlashFinder 0.000130165 0.000425643 0.00344651 0.000357635 0.000341273 100 reco:opslicer:OpSlicer 3.9167e-05 0.0168823 0.0730525 0.00901123 0.0179334 100 [art]:TriggerResults:TriggerResultInserter 1.2298e-05 1.72953e-05 0.000103226 1.5658e-05 9.52008e-06 100 end_path:out1:RootOutput 4.166e-06 5.47512e-06 2.1993e-05 5.072e-06 2.46623e-06 100 end_path:out1:RootOutput(write) 0.00605875 0.422825 0.920528 0.483405 0.216223 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 19937.2 MB Peak resident set size usage (VmHWM): 2231 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 14065.41_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2_graph_2025-08-02T_223917Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2_graph_2025-08-02T_223917Z.root debugprod.log graph_output_2025-08-02T_223917Z_7_ana_tree_hd.root graph_output_2025-08-02T_223917Z_7_analysiseid.root graph_output_2025-08-02T_223917Z_7_training1_CaloHitListU_graph.data graph_output_2025-08-02T_223917Z_7_training1_CaloHitListV_graph.data graph_output_2025-08-02T_223917Z_7_training1_CaloHitListW_graph.data graph_output_2025-08-02T_223917Z_7_training2_CaloHitListU_graph.data graph_output_2025-08-02T_223917Z_7_training2_CaloHitListV_graph.data graph_output_2025-08-02T_223917Z_7_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2_graph_2025-08-02T_223917Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6404651_325_20231202T093812Z_gen_g4_detsim_hitreco__20240507T214343Z_reco2_graph_2025-08-02T_223917Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv