Jobsub ID 14065.92@dunegpschedd02.fnal.gov
| Jobsub ID | 14065.92@dunegpschedd02.fnal.gov | |
| Workflow ID | 253 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-08-02 22:37:47 | |
| Site | NL_SURFsara | |
| Entry | DUNE_SurfSARA_arc03 | |
| Last heartbeat | 2025-08-02 22:48:01 | |
| From worker node | Hostname | wn-la-11.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-08-02 22:38:55 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 8m (490s) | |
| CPU time | 7m (443s = 90%) | |
| Max RSS bytes | 2101149696 (2003 MiB) | |
| Outputting started | 2025-08-02 22:47:06 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00253/1/001/graph_output_2025-08-02T_223906Z_2_ana_tree_hd.root | |
| Finished | 2025-08-02 22:48:01 | |
| Saved logs | justin-logs:14065.92-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
03 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-08-02T_223906Z_2_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-08-02T_223906Z_2_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-08-02T_223906Z_2_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-08-02T_223906Z_2_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-08-02T_223906Z_2_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-08-02T_223906Z_2_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-08-02T_223906Z_2_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-08-02T_223906Z_2_ana_tree_hd.root === Start last 100 lines of lar log file === 19 -1.74349e+06 20 -1.74593e+06 21 -1.74811e+06 22 -1.74996e+06 23 -1.75154e+06 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 22, Energy 0.123047, Dist. 48.1097, nMCHits 223 (79, 67, 77) MCPDG 22, Energy 0.123047, Dist. 48.1097, nMCHits 223 (79, 67, 77) --Primary 1, MCPDG 2212, Energy 1.11671, Dist. 21.5799, nMCHits 112 (26, 45, 41) MCPDG 2212, Energy 1.11671, Dist. 21.5799, nMCHits 112 (26, 45, 41) --Primary 2, MCPDG 22, Energy 0.0545009, Dist. 25.3274, nMCHits 85 (23, 37, 25) MCPDG 22, Energy 0.0545009, Dist. 25.3274, nMCHits 85 (23, 37, 25) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 98 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 1039140 bytes. The eid is 98 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 1365120 bytes. The eid is 98 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 1407012 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 1038640 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 1363492 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 1404976 bytes. Boundary wire vector sizes: 154, 135, 145 minwire 0: 2395 minwire 1: 398 minwire 2: 2458 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.00117318, Output 1: 0.0159314, 0.00334958, 0.000817702, 0.979901, Output 2: 0.0301738, 0.944038, 0.0104838, 0.0153046, Output 3: 0.00180094, 0.163987, 0.821548, 0.0126635, Output 4: 0.988667, 0.0107529, 0.000492802, 8.78772e-05, Output 5: 0.0049165, 0.989455, 0.00561272, 1.54981e-05, Output 6: 0.728083, 0.249609, 0.0200051, 0.00230298, Running Ophitfinder with InputDigiType = 'recob' Found hits: 129! 03-Aug-2025 00:45:59 CEST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2_graph_2025-08-02T_223906Z.root" 03-Aug-2025 00:45:59 CEST Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/e2/92/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.111539 3.03841 14.2443 3.07351 1.82831 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.00193569 0.00358942 0.0085592 0.00323131 0.00112313 100 reco:gaushit:GausHitFinder 0.0160189 0.0658683 0.714169 0.0278657 0.101582 100 reco:spsolve:SpacePointSolver 0.000127299 0.0753313 2.59262 0.00354214 0.307777 100 reco:hitfd:DisambigFromSpacePoints 0.00022795 0.0140043 0.441216 0.00103841 0.0520794 100 reco:rns:RandomNumberSaver 2.2423e-05 3.3415e-05 0.000343903 2.7557e-05 3.22064e-05 100 reco:pandora:StandardPandora 0.0124467 1.44305 9.74246 1.22091 1.02458 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000148882 0.000217384 0.00246909 0.000179749 0.000241681 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000195222 0.000249797 0.00327863 0.000215852 0.000304998 100 reco:pandoracalo:Calorimetry 0.000125948 0.000169028 0.00151391 0.000149032 0.000137523 100 reco:pandorapid:Chi2ParticleID 4.2094e-05 5.47299e-05 0.000714816 4.5913e-05 6.67421e-05 100 reco:cvnmap:CVNMapper 2.4407e-05 0.0107725 0.0326882 0.0126853 0.00751696 100 reco:cvneva:CVNEvaluator 2.0921e-05 0.813083 3.44471 1.08766 0.55821 100 reco:energyrecnumu:EnergyReco 0.00217344 0.00522099 0.0124686 0.00557951 0.00218025 100 reco:energyrecnue:EnergyReco 0.0021943 0.00289158 0.0089042 0.00252332 0.00111285 100 reco:energyrecnc:EnergyReco 0.00206339 0.00281008 0.00881773 0.00239557 0.00111722 100 reco:energyrecnumurange:EnergyReco 0.00201088 0.00279853 0.00911487 0.00239493 0.0011174 100 reco:energyrecnumumcs:EnergyReco 0.00215812 0.00278956 0.00882884 0.00238328 0.0011142 100 reco:opdec:Deconvolution 0.0240446 0.178027 0.343755 0.182148 0.0736414 100 reco:ophitspe:OpHitFinderDeco 0.00952725 0.0167296 0.0332157 0.0167403 0.00349841 100 reco:opflash:OpFlashFinder 0.000130144 0.000416221 0.00319565 0.000377424 0.000317032 100 reco:opslicer:OpSlicer 3.9548e-05 0.0156862 0.0849263 0.00831411 0.0192241 100 [art]:TriggerResults:TriggerResultInserter 1.2348e-05 1.695e-05 8.4027e-05 1.56085e-05 7.64273e-06 100 end_path:out1:RootOutput 4.456e-06 5.64843e-06 2.0421e-05 5.203e-06 2.20456e-06 100 end_path:out1:RootOutput(write) 0.0146466 0.383533 0.772935 0.478313 0.212805 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 19811.5 MB Peak resident set size usage (VmHWM): 2101.15 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 14065.92_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2_graph_2025-08-02T_223906Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2_graph_2025-08-02T_223906Z.root debugprod.log graph_output_2025-08-02T_223906Z_2_ana_tree_hd.root graph_output_2025-08-02T_223906Z_2_analysiseid.root graph_output_2025-08-02T_223906Z_2_training1_CaloHitListU_graph.data graph_output_2025-08-02T_223906Z_2_training1_CaloHitListV_graph.data graph_output_2025-08-02T_223906Z_2_training1_CaloHitListW_graph.data graph_output_2025-08-02T_223906Z_2_training2_CaloHitListU_graph.data graph_output_2025-08-02T_223906Z_2_training2_CaloHitListV_graph.data graph_output_2025-08-02T_223906Z_2_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2_graph_2025-08-02T_223906Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50584512_910_20231204T075612Z_gen_g4_detsim_hitreco__20240509T220413Z_reco2_graph_2025-08-02T_223906Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv