Jobsub ID 261629.53@dunegpschedd01.fnal.gov
| Jobsub ID | 261629.53@dunegpschedd01.fnal.gov | |
| Workflow ID | 10918 | |
| Stage ID | 1 | |
| User name | ichong@fnal.gov | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-12-01 15:07:55 | |
| Site | BR_CBPF | |
| Entry | DUNE_BR_CBPF_ce03 | |
| Last heartbeat | 2025-12-01 21:24:31 | |
| From worker node | Hostname | wn70 |
| cpuinfo | AMD EPYC 7713P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-12-01 15:08:56 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 5h (18077s) | |
| CPU time | 15m (917s = 5%) | |
| Max RSS bytes | 2010525696 (1917 MiB) | |
| Outputting started | 2025-12-01 20:10:14 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training1_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training1_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training1_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training2_CaloHitListU_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training2_CaloHitListV_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_training2_CaloHitListW_graph.data https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_analysiseid.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/10918/1/008/graph_output_2025-12-01T_150908Z_8_ana_tree_hd.root | |
| Finished | 2025-12-01 21:24:31 | |
| Saved logs | justin-logs:261629.53-dunegpschedd01.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
=============================================== Art has completed and will exit with status 0. === End last 100 lines of third lar log file === Renamed training1_CaloHitListU_graph.data -> graph_output_2025-12-01T_150908Z_8_training1_CaloHitListU_graph.data Renamed training1_CaloHitListV_graph.data -> graph_output_2025-12-01T_150908Z_8_training1_CaloHitListV_graph.data Renamed training1_CaloHitListW_graph.data -> graph_output_2025-12-01T_150908Z_8_training1_CaloHitListW_graph.data Renamed training2_CaloHitListU_graph.data -> graph_output_2025-12-01T_150908Z_8_training2_CaloHitListU_graph.data Renamed training2_CaloHitListV_graph.data -> graph_output_2025-12-01T_150908Z_8_training2_CaloHitListV_graph.data Renamed training2_CaloHitListW_graph.data -> graph_output_2025-12-01T_150908Z_8_training2_CaloHitListW_graph.data Renamed analysiseid.root -> graph_output_2025-12-01T_150908Z_8_analysiseid.root Renamed ana_tree_hd.root -> graph_output_2025-12-01T_150908Z_8_ana_tree_hd.root === Start last 100 lines of lar log file === MCPDG 2212, Energy 1.22962, Dist. 49.4192, nMCHits 263 (85, 83, 95) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 192 Graph saved to training1_CaloHitListW_graph.data Size of file training1_CaloHitListW_graph.data is 1981460 bytes. The eid is 192 Graph saved to training1_CaloHitListU_graph.data Size of file training1_CaloHitListU_graph.data is 2381360 bytes. The eid is 192 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 2484084 bytes. Operating in inference mode. Operating in training mode. The eid is -1 Graph saved to training2_CaloHitListW_graph.data Size of file training2_CaloHitListW_graph.data is 1980760 bytes. The eid is -1 Graph saved to training2_CaloHitListU_graph.data Size of file training2_CaloHitListU_graph.data is 2379052 bytes. The eid is -1 Graph saved to training2_CaloHitListV_graph.data Size of file training2_CaloHitListV_graph.data is 2480240 bytes. Boundary wire vector sizes: 1704, 2093, 1893 minwire 0: 1611 minwire 1: 765 minwire 2: 1568 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499 Classifier summary: Output 0: 0.0314745, Output 1: 0.805987, 0.00159703, 0.0374903, 0.154925, Output 2: 0.0638395, 0.568092, 0.351428, 0.016641, Output 3: 0.0455451, 0.933526, 0.0208398, 8.94819e-05, Output 4: 0.507833, 0.466348, 0.0251385, 0.000680004, Output 5: 0.0138234, 0.930035, 0.0554709, 0.000670428, Output 6: 0.576228, 0.387653, 0.0326716, 0.00344741, Running Ophitfinder with InputDigiType = 'recob' Found hits: 434! Begin processing the 200th record. run: 6209058 subRun: 1 event: 69800 at 01-Dec-2025 16:12:43 -03 ---MC-PARTICLE-MONITORING----------------------------------------------------------------------- BeamNeutrinos: --Primary 0, MCPDG 22, Energy 0.00311589, Dist. 20.2088, nMCHits 20 (6, 12, 2) MCPDG 22, Energy 0.00311589, Dist. 20.2088, nMCHits 20 (6, 12, 2) ------------------------------------------------------------------------------------------------ Operating in training mode. The eid is 193 Graph saved to training1_CaloHitListV_graph.data Size of file training1_CaloHitListV_graph.data is 2484592 bytes. Operating in inference mode. Operating in training mode. Running Ophitfinder with InputDigiType = 'recob' Found hits: 0! 01-Dec-2025 16:12:54 -03 Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2_graph_2025-12-01T_150908Z.root" 01-Dec-2025 16:12:54 -03 Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/a0/38/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 4.2991 8.8353 37.5628 8.95868 3.28809 200 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.223109 0.336769 0.653896 0.392042 0.112857 200 reco:gaushit:GausHitFinder 0.691015 0.760877 1.86665 0.713871 0.140071 200 reco:spsolve:SpacePointSolver 0.000131355 0.173208 12.2197 0.00565303 1.12167 200 reco:hitfd:DisambigFromSpacePoints 0.000234209 0.0138173 0.389739 0.00178977 0.0472389 200 reco:rns:RandomNumberSaver 2.3301e-05 3.33354e-05 0.000437317 2.89265e-05 3.0097e-05 200 reco:pandora:StandardPandora 0.678281 2.13624 15.3473 1.89435 1.41193 200 reco:pandoraTrack:LArPandoraTrackCreation 0.000129535 0.000252023 0.00754954 0.000204453 0.000520141 200 reco:pandoraShower:LArPandoraModularShowerCreation 0.000166436 0.00028816 0.00314406 0.000270615 0.000205414 200 reco:pandoracalo:Calorimetry 0.000107854 0.000176103 0.00134949 0.000166647 8.63402e-05 200 reco:pandorapid:Chi2ParticleID 3.6491e-05 6.68166e-05 0.000690707 6.3247e-05 4.59367e-05 200 reco:cvnmap:CVNMapper 2.6151e-05 0.0115439 0.0524182 0.0138868 0.0077461 200 reco:cvneva:CVNEvaluator 1.8521e-05 1.14836 3.63976 1.53241 0.6987 200 reco:energyrecnumu:EnergyReco 0.00215593 0.00624993 0.0157198 0.00524364 0.00255242 200 reco:energyrecnue:EnergyReco 0.00207225 0.00440957 0.014021 0.00409273 0.00142295 200 reco:energyrecnc:EnergyReco 0.00199631 0.00433729 0.0140148 0.00401719 0.00135235 200 reco:energyrecnumurange:EnergyReco 0.00200901 0.00435978 0.0139639 0.00402159 0.00136587 200 reco:energyrecnumumcs:EnergyReco 0.00198347 0.00436163 0.0138319 0.00403095 0.00136904 200 reco:opdec:Deconvolution 0.673348 1.6277 2.16085 1.74038 0.340968 200 reco:ophitspe:OpHitFinderDeco 2.00202 2.01076 2.02737 2.01042 0.00369769 200 reco:opflash:OpFlashFinder 0.000155686 0.000476195 0.00490226 0.000422181 0.000355303 200 reco:opslicer:OpSlicer 5.8322e-05 0.0200647 0.151427 0.00945441 0.0254316 200 [art]:TriggerResults:TriggerResultInserter 1.342e-05 2.10878e-05 8.8914e-05 2.0061e-05 7.5404e-06 200 end_path:out1:RootOutput 3.43e-06 4.20409e-06 2.4251e-05 4.075e-06 1.46897e-06 200 end_path:out1:RootOutput(write) 0.00594074 0.569625 1.75501 0.657606 0.336796 200 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 20774.6 MB Peak resident set size usage (VmHWM): 2010.53 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === === Generated output files === 261629.53_dunegpschedd01.fnal.gov.logs.tgz atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2_graph_2025-12-01T_150908Z.log atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2_graph_2025-12-01T_150908Z.root debugprod.log graph_output_2025-12-01T_150908Z_8_ana_tree_hd.root graph_output_2025-12-01T_150908Z_8_analysiseid.root graph_output_2025-12-01T_150908Z_8_training1_CaloHitListU_graph.data graph_output_2025-12-01T_150908Z_8_training1_CaloHitListV_graph.data graph_output_2025-12-01T_150908Z_8_training1_CaloHitListW_graph.data graph_output_2025-12-01T_150908Z_8_training2_CaloHitListU_graph.data graph_output_2025-12-01T_150908Z_8_training2_CaloHitListV_graph.data graph_output_2025-12-01T_150908Z_8_training2_CaloHitListW_graph.data jobscript.log justin-processed-pfns.txt reco_hist.root secondary_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2_graph_2025-12-01T_150908Z.log third_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_348_20231122T121940Z_gen_g4_detsim_hitreco__20240507T181845Z_reco2_graph_2025-12-01T_150908Z.log training1_CaloHitListU.csv training1_CaloHitListV.csv training1_CaloHitListW.csv training2_CaloHitListU.csv training2_CaloHitListV.csv training2_CaloHitListW.csv