Jobsub ID 14065.80@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.80@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | UK_RAL-Tier1 | |
| Entry | LIGO_UK_RAL_arc_ce01 | |
| Last heartbeat | 2025-08-02 22:51:54 | |
| From worker node | Hostname | dune001-8416003.0-lcg2706.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 9654 96-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:38:52 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 12m (734s) | |
| CPU time | 7m (459s = 62%) | |
| Max RSS bytes | 1986781184 (1894 MiB) | |
| Outputting started | 2025-08-02 22:51:08 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223900Z_5_ana_tree_hd.root | |
| Finished | 2025-08-02 22:51:54 | |
| Saved logs | justin-logs:14065.80-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
used) Peak virtual memory usage (VmPeak) : 36333.5 MB Peak resident set size usage (VmHWM): 863.236 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-08-02T_223900Z_5_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223900Z_5_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223900Z_5_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223900Z_5_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223900Z_5_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223900Z_5_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-08-02T_223900Z_5_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223900Z_5_ana_tree_hd.root === Start last 100 lines of lar log file === Now with regularization... Begin: 3.60398e+07 0 3.59602e+07 1 3.59023e+07 2 3.5856e+07 3 3.58187e+07 4 3.57899e+07 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 13, Energy 0.698, Dist. 189.966, nMCHits 581 (46, 348, 187) MCPDG 13, Energy 0.698, Dist. 189.966, nMCHits 581 (46, 348, 187) --Primary 1, MCPDG 211, Energy 0.936301, Dist. 67.2135, nMCHits 340 (111, 115, 114) MCPDG 211, Energy 0.936301, Dist. 67.2135, nMCHits 249 (80, 76, 93) \_ MCPDG 2212, Energy 1.00562, Dist. 3.95817, nMCHits 14 (5, 4, 5) \_ MCPDG 22, Energy 0.0754758, Dist. 1.63044, nMCHits 77 (26, 35, 16) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 98 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 871316 bytes. The eid is 98 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 1064760 bytes. The eid is 98 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 1031616 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 871316 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 1061956 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 1029660 bytes. Boundary wire vector sizes: 766, 404, 520 minwire 0: 104 minwire 1: 1809 minwire 2: 59 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.00931621, Output 1: 0.00318032, 0.258785, 0.0159911, 0.722044, Output 2: 0.33237, 0.557924, 0.0167965, 0.0929098, Output 3: 0.764039, 0.227575, 0.0071076, 0.00127837, Output 4: 0.0194508, 0.971756, 0.00877784, 1.57917e-05, Output 5: 0.982187, 0.0177041, 8.86673e-05, 2.06736e-05, Output 6: 0.763028, 0.22222, 0.0122939, 0.00245815, Running Ophitfinder with InputDigiType = 'recob' Found hits: 164! 02-Aug-2025 22:48:36 UTC Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2_graph_2025-08-02T_223900Z.root" 02-Aug-2025 22:48:36 UTC Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/19/66/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.497163 3.87585 35.1513 3.57522 3.63452 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.010161 0.092756 2.1322 0.0649843 0.208449 100 reco:gaushit:GausHitFinder 0.078273 0.168159 0.741656 0.134162 0.104255 100 reco:spsolve:SpacePointSolver 0.000142796 0.394443 31.2357 0.0035572 3.12925 100 reco:hitfd:DisambigFromSpacePoints 0.000274173 0.00876982 0.130011 0.00112821 0.0229662 100 reco:rns:RandomNumberSaver 2.3556e-05 4.91228e-05 0.000654317 3.86985e-05 6.28211e-05 100 reco:pandora:StandardPandora 0.0953261 1.73553 9.73502 1.54071 0.930017 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000115254 0.000280231 0.00182111 0.000248209 0.00018393 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000153692 0.000302814 0.00246269 0.000272746 0.000226079 100 reco:pandoracalo:Calorimetry 9.6265e-05 0.000238396 0.00125341 0.000212175 0.000131193 100 reco:pandorapid:Chi2ParticleID 3.3901e-05 6.97799e-05 0.000797714 5.9044e-05 7.46471e-05 100 reco:cvnmap:CVNMapper 3.5814e-05 0.0168719 0.0503913 0.0189035 0.013601 100 reco:cvneva:CVNEvaluator 1.8328e-05 0.704246 3.68765 0.906637 0.572796 100 reco:energyrecnumu:EnergyReco 0.00148596 0.00613083 0.0159004 0.00631399 0.00313396 100 reco:energyrecnue:EnergyReco 0.00148101 0.00302368 0.0098452 0.00274356 0.00137509 100 reco:energyrecnc:EnergyReco 0.00148887 0.00285231 0.0097595 0.00256854 0.00129503 100 reco:energyrecnumurange:EnergyReco 0.00142183 0.0028129 0.00973992 0.00252437 0.00126966 100 reco:energyrecnumumcs:EnergyReco 0.00142501 0.00284088 0.00975465 0.00250851 0.00130818 100 reco:opdec:Deconvolution 0.128426 0.268782 0.538391 0.264912 0.0841448 100 reco:ophitspe:OpHitFinderDeco 0.040246 0.0946686 0.433969 0.0878821 0.051912 100 reco:opflash:OpFlashFinder 0.000118819 0.000550267 0.00208488 0.000462643 0.000301671 100 reco:opslicer:OpSlicer 4.7902e-05 0.0130585 0.125214 0.00621353 0.0193328 100 [art]:TriggerResults:TriggerResultInserter 1.3159e-05 2.45813e-05 8.1723e-05 2.11815e-05 1.19272e-05 100 end_path:out1:RootOutput 4.346e-06 7.55617e-06 4.0982e-05 5.989e-06 5.34453e-06 100 end_path:out1:RootOutput(write) 0.00505058 0.357923 0.793398 0.475995 0.224922 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 38116.5 MB Peak resident set size usage (VmHWM): 1986.78 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 14065.80_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2_graph_2025-08-02T_223900Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2_graph_2025-08-02T_223900Z.root debugprod.log graph_output_2025-08-02T_223900Z_5_ana_tree_hd.root graph_output_2025-08-02T_223900Z_5_analysiseid.root graph_output_2025-08-02T_223900Z_5_training1_CaloHitListU_graph.data graph_output_2025-08-02T_223900Z_5_training1_CaloHitListV_graph.data graph_output_2025-08-02T_223900Z_5_training1_CaloHitListW_graph.data graph_output_2025-08-02T_223900Z_5_training2_CaloHitListU_graph.data graph_output_2025-08-02T_223900Z_5_training2_CaloHitListV_graph.data graph_output_2025-08-02T_223900Z_5_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2_graph_2025-08-02T_223900Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6405486_187_20231202T112025Z_gen_g4_detsim_hitreco__20240507T214622Z_reco2_graph_2025-08-02T_223900Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv