Jobsub ID 11804.147@dunegpschedd02.fnal.gov
| Jobsub ID | 11804.147@dunegpschedd02.fnal.gov | |
| Workflow ID | 83 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-07-27 20:54:16 | |
| Site | NL_SURFsara | |
| Entry | DUNE_SurfSARA_arc01 | |
| Last heartbeat | 2025-07-27 21:26:55 | |
| From worker node | Hostname | wn-da-06.gina.surf.nl |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-07-27 20:56:04 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 30m (1804s) | |
| CPU time | 11m (666s = 36%) | |
| Max RSS bytes | 2017349632 (1923 MiB) | |
| Outputting started | 2025-07-27 21:26:09 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00083/1/023/graph_output_2025-07-27T_205613Z_8_ana_tree_hd.root | |
| Finished | 2025-07-27 21:26:55 | |
| Saved logs | justin-logs:11804.147-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
======== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 10062.4 MB Peak resident set size usage (VmHWM): 1022.93 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-07-27T_205613Z_8_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-07-27T_205613Z_8_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-07-27T_205613Z_8_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-07-27T_205613Z_8_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-07-27T_205613Z_8_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-07-27T_205613Z_8_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-07-27T_205613Z_8_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-07-27T_205613Z_8_ana_tree_hd.root === Start last 100 lines of lar log file === Begin: 2.41229e+07 0 2.11252e+07 1 2.08995e+07 2 2.08763e+07 3 2.08724e+07 Now with regularization... Begin: 1.85575e+07 0 1.85532e+07 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 11, Energy 0.383806, Dist. 52.9506, nMCHits 530 (211, 150, 169) MCPDG 11, Energy 0.383806, Dist. 52.9506, nMCHits 530 (211, 150, 169) --Primary 1, MCPDG 211, Energy 0.152835, Dist. 0.912771, nMCHits 60 (21, 20, 19) MCPDG 211, Energy 0.152835, Dist. 0.912771, nMCHits 9 (2, 5, 2) \_ MCPDG -11, Energy 0.0408409, Dist. 10.2447, nMCHits 51 (19, 15, 17) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 195 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 2239588 bytes. The eid is 195 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 3099452 bytes. The eid is 195 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 3021860 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 2239304 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 3094296 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 3018844 bytes. Boundary wire vector sizes: 262, 327, 268 minwire 0: 1132 minwire 1: 1917 minwire 2: 651 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0200227, Output 1: 0.0021519, 0.207316, 0.0162388, 0.774293, Output 2: 0.455215, 0.446469, 0.0476752, 0.0506404, Output 3: 0.495146, 0.469515, 0.0338493, 0.00148912, Output 4: 0.975455, 0.0233198, 0.000792555, 0.000432999, Output 5: 0.147305, 0.849406, 0.00317454, 0.000114805, Output 6: 0.0128141, 0.705276, 0.217215, 0.0646947, Running Ophitfinder with InputDigiType = 'recob' Found hits: 351! 27-Jul-2025 23:21:09 CEST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2_graph_2025-07-27T_205613Z.root" 27-Jul-2025 23:21:09 CEST Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/06/ef/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.353027 2.82434 13.6732 2.8221 1.78228 200 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.015614 0.0339166 0.365374 0.0311277 0.0257514 200 reco:gaushit:GausHitFinder 0.0571935 0.11314 0.7246 0.0788407 0.100992 200 reco:spsolve:SpacePointSolver 9.1264e-05 0.0731085 3.00958 0.0033236 0.315407 200 reco:hitfd:DisambigFromSpacePoints 0.000185694 0.0151238 0.40359 0.000946495 0.0553217 200 reco:rns:RandomNumberSaver 1.7433e-05 2.4708e-05 0.00032571 2.128e-05 2.20638e-05 200 reco:pandora:StandardPandora 0.0513383 1.21046 9.25099 0.968674 1.0299 200 reco:pandoraTrack:LArPandoraTrackCreation 0.000114357 0.00015458 0.00213291 0.000131745 0.000144987 200 reco:pandoraShower:LArPandoraModularShowerCreation 0.0001579 0.00019253 0.00311112 0.00017628 0.000207322 200 reco:pandoracalo:Calorimetry 9.5372e-05 0.000123687 0.0018087 0.000108441 0.000121225 200 reco:pandorapid:Chi2ParticleID 3.4655e-05 4.42427e-05 0.000921842 3.83175e-05 6.24038e-05 200 reco:cvnmap:CVNMapper 2.0089e-05 0.0140828 0.0525183 0.0161319 0.0107023 200 reco:cvneva:CVNEvaluator 1.514e-05 0.655121 2.55539 0.925202 0.45403 200 reco:energyrecnumu:EnergyReco 0.00190971 0.0049642 0.0204653 0.003552 0.00356929 200 reco:energyrecnue:EnergyReco 0.0019172 0.00286794 0.00977776 0.00243902 0.00129006 200 reco:energyrecnc:EnergyReco 0.00202851 0.00280557 0.0100647 0.00235993 0.00127546 200 reco:energyrecnumurange:EnergyReco 0.00204019 0.00292783 0.0164488 0.00236619 0.00174977 200 reco:energyrecnumumcs:EnergyReco 0.00204889 0.00306575 0.0166409 0.00236041 0.00214814 200 reco:opdec:Deconvolution 0.0620808 0.230371 0.431421 0.235773 0.0833049 200 reco:ophitspe:OpHitFinderDeco 0.132958 0.141642 0.175081 0.141167 0.00443322 200 reco:opflash:OpFlashFinder 0.00011569 0.000319947 0.00263371 0.000286104 0.000204182 200 reco:opslicer:OpSlicer 3.3163e-05 0.0123333 0.0907982 0.00635155 0.0158388 200 [art]:TriggerResults:TriggerResultInserter 1.006e-05 1.31167e-05 4.2342e-05 1.21425e-05 3.6969e-06 200 end_path:out1:RootOutput 2.825e-06 4.05715e-06 1.9849e-05 3.842e-06 1.46047e-06 200 end_path:out1:RootOutput(write) 0.00333517 0.306578 0.672992 0.393297 0.186919 200 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 11612.4 MB Peak resident set size usage (VmHWM): 2017.35 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 11804.147_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2_graph_2025-07-27T_205613Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2_graph_2025-07-27T_205613Z.root debugprod.log graph_output_2025-07-27T_205613Z_8_ana_tree_hd.root graph_output_2025-07-27T_205613Z_8_analysiseid.root graph_output_2025-07-27T_205613Z_8_training1_CaloHitListU_graph.data graph_output_2025-07-27T_205613Z_8_training1_CaloHitListV_graph.data graph_output_2025-07-27T_205613Z_8_training1_CaloHitListW_graph.data graph_output_2025-07-27T_205613Z_8_training2_CaloHitListU_graph.data graph_output_2025-07-27T_205613Z_8_training2_CaloHitListV_graph.data graph_output_2025-07-27T_205613Z_8_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2_graph_2025-07-27T_205613Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6171714_241_20231126T014257Z_gen_g4_detsim_hitreco__20240507T183418Z_reco2_graph_2025-07-27T_205613Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv