Jobsub ID 14065.9@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.9@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | ES_PIC | |
| Entry | DUNE_T1_ES_PIC_ce15-multicore | |
| Last heartbeat | 2025-08-02 23:19:02 | |
| From worker node | Hostname | tds511.pic.es |
| cpuinfo | AMD EPYC 7513 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:39:14 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 38m (2301s) | |
| CPU time | 7m (457s = 19%) | |
| Max RSS bytes | 1973825536 (1882 MiB) | |
| Outputting started | 2025-08-02 23:17:37 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_ana_tree_hd.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223921Z_10_training1_CaloHitListW_graph.data | |
| Finished | 2025-08-02 23:19:02 | |
| Saved logs | justin-logs:14065.9-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
======================================================================================= Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-08-02T_223921Z_10_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223921Z_10_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223921Z_10_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223921Z_10_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223921Z_10_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223921Z_10_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-08-02T_223921Z_10_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223921Z_10_ana_tree_hd.root === Start last 100 lines of lar log file === 6 3.18975e+06 7 3.18484e+06 8 3.1809e+06 9 3.17807e+06 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 211, Energy 0.305479, Dist. 31.5735, nMCHits 136 (52, 49, 35) MCPDG 211, Energy 0.305479, Dist. 31.5735, nMCHits 110 (47, 39, 24) \_ MCPDG -11, Energy 0.0418633, Dist. 8.1756, nMCHits 26 (5, 10, 11) --Primary 1, MCPDG 13, Energy 0.917991, Dist. 21.6689, nMCHits 101 (37, 41, 23) MCPDG 13, Energy 0.917991, Dist. 21.6689, nMCHits 101 (37, 41, 23) --Primary 2, MCPDG 2212, Energy 1.06422, Dist. 11.6388, nMCHits 48 (27, 5, 16) MCPDG 2212, Energy 1.06422, Dist. 11.6388, nMCHits 48 (27, 5, 16) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 99 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 1150452 bytes. The eid is 99 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 1460044 bytes. The eid is 99 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 1481892 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 1150452 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 1460044 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 1481892 bytes. Boundary wire vector sizes: 118, 101, 79 minwire 0: 2115 minwire 1: 1621 minwire 2: 1704 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0364746, Output 1: 0.0191454, 0.00180246, 0.00809166, 0.97096, Output 2: 0.184815, 0.669863, 0.0200248, 0.125297, Output 3: 0.965619, 0.0318905, 0.00143116, 0.00105973, Output 4: 0.0178723, 0.964202, 0.0178788, 4.75072e-05, Output 5: 0.998031, 0.00192587, 3.18246e-05, 1.1696e-05, Output 6: 0.0586246, 0.808193, 0.116748, 0.0164352, Running Ophitfinder with InputDigiType = 'recob' Found hits: 187! 03-Aug-2025 01:10:31 CEST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2_graph_2025-08-02T_223921Z.root" 03-Aug-2025 01:10:31 CEST Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b9/e1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 1.97817 4.44408 13.2489 4.33022 1.70325 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0463667 0.0695808 0.100494 0.0475519 0.0231394 100 reco:gaushit:GausHitFinder 0.156424 0.253136 1.03651 0.208343 0.139706 100 reco:spsolve:SpacePointSolver 0.000157547 0.0782156 3.39941 0.00645513 0.357466 100 reco:hitfd:DisambigFromSpacePoints 0.000272123 0.0180623 0.397531 0.00174306 0.0571873 100 reco:rns:RandomNumberSaver 1.9306e-05 0.000127857 0.00907435 3.0889e-05 0.000899909 100 reco:pandora:StandardPandora 1.07777 1.43421 6.02586 1.20164 0.774553 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000108023 0.000211435 0.00197039 0.00018057 0.000195456 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000137289 0.000255734 0.00298207 0.000226843 0.000275198 100 reco:pandoracalo:Calorimetry 9.008e-05 0.000157213 0.00124175 0.000145309 0.000111445 100 reco:pandorapid:Chi2ParticleID 3.2321e-05 5.62308e-05 0.000699298 4.7795e-05 6.52273e-05 100 reco:cvnmap:CVNMapper 2.8414e-05 0.0133698 0.070598 0.0134284 0.0088922 100 reco:cvneva:CVNEvaluator 1.59e-05 1.11854 4.61817 1.28838 0.605516 100 reco:energyrecnumu:EnergyReco 0.00168514 0.00675476 0.0220107 0.00706594 0.00340419 100 reco:energyrecnue:EnergyReco 0.00168524 0.0040589 0.0212667 0.00334687 0.0025036 100 reco:energyrecnc:EnergyReco 0.00162873 0.00406364 0.0239704 0.00332101 0.0028354 100 reco:energyrecnumurange:EnergyReco 0.00169186 0.00400427 0.0217963 0.00330938 0.00252268 100 reco:energyrecnumumcs:EnergyReco 0.0015519 0.0038876 0.017065 0.00328178 0.00194257 100 reco:opdec:Deconvolution 0.173833 0.418381 0.697002 0.427675 0.0983308 100 reco:ophitspe:OpHitFinderDeco 0.412571 0.420321 0.449169 0.420046 0.00487076 100 reco:opflash:OpFlashFinder 0.000167275 0.000746521 0.0185922 0.000411571 0.00215984 100 reco:opslicer:OpSlicer 6.0825e-05 0.0261318 0.101039 0.0177082 0.0273855 100 [art]:TriggerResults:TriggerResultInserter 1.3355e-05 2.15309e-05 9.4769e-05 2.0043e-05 9.19518e-06 100 end_path:out1:RootOutput 3.467e-06 5.11211e-06 1.9717e-05 4.849e-06 1.76003e-06 100 end_path:out1:RootOutput(write) 0.0157555 0.568275 1.39573 0.630483 0.251909 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 20727.7 MB Peak resident set size usage (VmHWM): 1973.83 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 14065.9_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2_graph_2025-08-02T_223921Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2_graph_2025-08-02T_223921Z.root debugprod.log graph_output_2025-08-02T_223921Z_10_ana_tree_hd.root graph_output_2025-08-02T_223921Z_10_analysiseid.root graph_output_2025-08-02T_223921Z_10_training1_CaloHitListU_graph.data graph_output_2025-08-02T_223921Z_10_training1_CaloHitListV_graph.data graph_output_2025-08-02T_223921Z_10_training1_CaloHitListW_graph.data graph_output_2025-08-02T_223921Z_10_training2_CaloHitListU_graph.data graph_output_2025-08-02T_223921Z_10_training2_CaloHitListV_graph.data graph_output_2025-08-02T_223921Z_10_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2_graph_2025-08-02T_223921Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492237_144_20231208T052441Z_gen_g4_detsim_hitreco__20240510T042645Z_reco2_graph_2025-08-02T_223921Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv