Jobsub ID 255926.4@dunegpschedd02.fnal.gov
| Jobsub ID | 255926.4@dunegpschedd02.fnal.gov | |
| Workflow ID | 10918 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-12-01 17:15:59 | |
| Site | BR_CBPF | |
| Entry | DUNE_BR_CBPF_ce03 | |
| Last heartbeat | 2025-12-01 21:11:16 | |
| From worker node | Hostname | wn65 |
| cpuinfo | AMD EPYC 7713P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-12-01 17:18:11 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 2h (9520s) | |
| CPU time | 9m (576s = 6%) | |
| Max RSS bytes | 2119544832 (2021 MiB) | |
| Outputting started | 2025-12-01 19:56:53 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_171825Z_2_ana_tree_hd.root | |
| Finished | 2025-12-01 21:11:16 | |
| Saved logs | justin-logs:255926.4-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
============================================================ Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-12-01T_171825Z_2_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-12-01T_171825Z_2_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-12-01T_171825Z_2_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-12-01T_171825Z_2_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-12-01T_171825Z_2_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-12-01T_171825Z_2_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-12-01T_171825Z_2_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-12-01T_171825Z_2_ana_tree_hd.root === Start last 100 lines of lar log file === Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 1258076 bytes. The eid is 97 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 1643964 bytes. The eid is 97 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 1777248 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 1257800 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 1642476 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 1775500 bytes. Boundary wire vector sizes: 155, 157, 167 minwire 0: 2016 minwire 1: 368 minwire 2: 2216 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max wires due to vertex determination failure: 2268, 2767 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0100539, Output 1: 0.00935522, 0.00183217, 0.00341925, 0.985393, Output 2: 0.0592783, 0.873503, 0.00674784, 0.0604709, Output 3: 0.106302, 0.76035, 0.128131, 0.0052172, Output 4: 0.0245897, 0.934506, 0.0408468, 5.70788e-05, Output 5: 0.997313, 0.00252268, 0.000135821, 2.90489e-05, Output 6: 0.472096, 0.305263, 0.109176, 0.113465, Running Ophitfinder with InputDigiType = 'recob' Found hits: 196! Begin processing the 100th record. run: 6402377 subRun: 1 event: 29500 at 01-Dec-2025 16:26:48 -03 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 22, Energy 0.00369799, Dist. 2.02891, nMCHits 15 (5, 8, 2) MCPDG 22, Energy 0.00369799, Dist. 2.02891, nMCHits 15 (5, 8, 2) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 98 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 1644368 bytes. The eid is 98 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 1777748 bytes. Operating in inference mode. Operating in training mode. Running Ophitfinder with InputDigiType = 'recob' Found hits: 5! 01-Dec-2025 16:26:59 -03 Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2_graph_2025-12-01T_171825Z.root" 01-Dec-2025 16:27:00 -03 Closed input file "root://xrootd.pic.es:1094/pnfs/pic.es/data/dune/RSE/fardet-hd/55/f6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 4.69961 9.69962 69.0676 9.19426 7.76099 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.227942 0.340399 0.457053 0.240669 0.11348 100 reco:gaushit:GausHitFinder 0.725586 0.889225 3.00772 0.783593 0.356668 100 reco:spsolve:SpacePointSolver 0.000142256 0.272731 15.8201 0.00489706 1.72721 100 reco:hitfd:DisambigFromSpacePoints 0.00024493 0.0349283 1.53743 0.00138413 0.170751 100 reco:rns:RandomNumberSaver 1.8291e-05 3.24617e-05 0.000348444 2.7626e-05 3.25533e-05 100 reco:pandora:StandardPandora 0.693803 2.83044 39.7653 1.9394 4.90311 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000127655 0.000232015 0.00257698 0.000202868 0.000237654 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000162357 0.000302634 0.00351908 0.000270811 0.000324911 100 reco:pandoracalo:Calorimetry 0.000103664 0.000182462 0.00157744 0.000166777 0.000142429 100 reco:pandorapid:Chi2ParticleID 3.8851e-05 6.92318e-05 0.000811373 6.3578e-05 7.51945e-05 100 reco:cvnmap:CVNMapper 2.8261e-05 0.0111227 0.0931333 0.0135612 0.013156 100 reco:cvneva:CVNEvaluator 1.7261e-05 0.997697 4.00958 1.53376 0.818932 100 reco:energyrecnumu:EnergyReco 0.00227365 0.00661052 0.0483665 0.00467894 0.00560246 100 reco:energyrecnue:EnergyReco 0.00226588 0.00501644 0.0348532 0.00401261 0.00402877 100 reco:energyrecnc:EnergyReco 0.00237419 0.0052896 0.0342542 0.00393065 0.00495685 100 reco:energyrecnumurange:EnergyReco 0.00209292 0.00462299 0.0249058 0.00395438 0.00301369 100 reco:energyrecnumumcs:EnergyReco 0.00216645 0.00463942 0.0252356 0.00393592 0.00296508 100 reco:opdec:Deconvolution 0.704107 1.69895 2.36935 1.8605 0.404894 100 reco:ophitspe:OpHitFinderDeco 2.04634 2.0562 2.0907 2.05597 0.00570927 100 reco:opflash:OpFlashFinder 0.000151766 0.000470876 0.00420706 0.000406525 0.000421976 100 reco:opslicer:OpSlicer 5.4592e-05 0.0193119 0.121562 0.0105687 0.0244661 100 [art]:TriggerResults:TriggerResultInserter 1.414e-05 2.10911e-05 8.4423e-05 1.919e-05 1.0177e-05 100 end_path:out1:RootOutput 3.39e-06 4.69432e-06 3.0261e-05 4.2105e-06 3.31836e-06 100 end_path:out1:RootOutput(write) 0.00704369 0.519837 2.40964 0.665254 0.441829 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 20880.9 MB Peak resident set size usage (VmHWM): 2119.54 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 255926.4_dunegpschedd02.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2_graph_2025-12-01T_171825Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2_graph_2025-12-01T_171825Z.root debugprod.log graph_output_2025-12-01T_171825Z_2_ana_tree_hd.root graph_output_2025-12-01T_171825Z_2_analysiseid.root graph_output_2025-12-01T_171825Z_2_training1_CaloHitListU_graph.data graph_output_2025-12-01T_171825Z_2_training1_CaloHitListV_graph.data graph_output_2025-12-01T_171825Z_2_training1_CaloHitListW_graph.data graph_output_2025-12-01T_171825Z_2_training2_CaloHitListU_graph.data graph_output_2025-12-01T_171825Z_2_training2_CaloHitListV_graph.data graph_output_2025-12-01T_171825Z_2_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2_graph_2025-12-01T_171825Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6402377_294_20231202T102141Z_gen_g4_detsim_hitreco__20240507T210652Z_reco2_graph_2025-12-01T_171825Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv