Jobsub ID 45082.2@dunegpschedd02.fnal.gov
| Jobsub ID | 45082.2@dunegpschedd02.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-21 17:04:27 | |
| Site | NL_NIKHEF | |
| Entry | VIRGO_NL_NIKHEF_dissel | |
| Last heartbeat | 2025-09-21 17:48:49 | |
| From worker node | Hostname | wn-lot-030.farm.nikhef.nl |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Started | 2025-09-21 17:05:16 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50598687_580_20231204T223216Z_gen_g4_detsim_hitreco__20240509T221711Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1500_291_20231028T032900Z_gen_g4_detsim_hitreco__20240503T044656Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419113_640_20231203T151708Z_gen_g4_detsim_hitreco__20240508T054537Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574782_29_20231203T090238Z_gen_g4_detsim_hitreco__20240509T193907Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 43m (2594s) | |
| CPU time | 35m (2156s = 83%) | |
| Max RSS bytes | 2149830656 (2050 MiB) | |
| Outputting started | 2025-09-21 17:48:31 | |
| Output files | ||
| Finished | 2025-09-21 17:48:49 | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
.0955867 3.31446 219.669 1.65626 13.5987 500 reco:pandoraTrack:LArPandoraTrackCreation 0.000125294 0.00659429 0.325441 0.00141084 0.0220938 500 reco:pandoraShower:LArPandoraModularShowerCreation 0.000173574 0.0105219 0.188283 0.00262777 0.0218262 500 reco:pandoracalo:Calorimetry 0.000100467 0.0240232 2.43084 0.000936696 0.140982 500 reco:pandorapid:Chi2ParticleID 3.6749e-05 0.00014006 0.00152497 0.000108243 0.000122529 500 reco:cvnmap:CVNMapper 2.0909e-05 0.00917019 0.0414908 0.0114688 0.00609377 500 reco:cvneva:CVNEvaluator 1.7262e-05 0.27565 2.47529 0.368371 0.192482 500 reco:energyrecnumu:EnergyReco 0.000250528 0.0650797 2.52962 0.0122076 0.180066 500 reco:energyrecnue:EnergyReco 0.000144941 0.00232204 0.124339 0.000505284 0.0081046 500 reco:energyrecnc:EnergyReco 0.00218667 0.0891565 0.424912 0.0974701 0.061611 500 reco:energyrecnumurange:EnergyReco 0.000216103 0.0341759 2.28214 0.00235235 0.165061 500 reco:energyrecnumumcs:EnergyReco 0.000585514 0.0348881 2.27402 0.00306952 0.164419 500 reco:energyrecnumumcsllhd:EnergyReco 0.00042312 0.0350813 2.27453 0.00301509 0.165789 500 reco:opdec:Deconvolution 0.0504982 0.241273 0.482234 0.24091 0.0742299 500 reco:ophitspe:OpHitFinderDeco 0.128324 0.144014 0.198376 0.135439 0.0184019 500 reco:opflash:OpFlashFinder 0.000185367 0.00119382 0.0049615 0.00112694 0.000505755 500 reco:opslicer:OpSlicer 5.0213e-05 0.237224 0.906153 0.181375 0.184638 500 reco:rns:RandomNumberSaver 1.9196e-05 3.01106e-05 0.000387854 2.9681e-05 1.6366e-05 500 reco:anglereconue:NuAngularReco 0.000158777 0.00161989 0.0922493 0.000373593 0.00599621 500 reco:anglereconumu:NuAngularReco 7.4759e-05 0.00146212 0.0934314 0.00020857 0.00608753 500 reco:anglereconuepfps:NuAngularReco 0.000122589 0.00156387 0.0943424 0.000308917 0.00609617 500 reco:anglereconumupfps:NuAngularReco 0.000103794 0.00151381 0.0941503 0.000266964 0.00606794 500 reco:anglerecohits:NuAngularReco 0.00011747 0.00208248 0.09989 0.000676434 0.00661633 500 [art]:TriggerResults:TriggerResultInserter 1.0219e-05 1.28981e-05 5.892e-05 1.2458e-05 2.96757e-06 500 end_path:cafmaker:CAFMaker 0.00142715 0.0221718 1.21141 0.0043256 0.0868689 500 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3810.04 MB Peak resident set size usage (VmHWM): 2149.83 MB ==================================================================================================== lar exit code 0 === Start last 100 lines of lar log file === Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00179134, 0.988142, 0.010067, Output 1: 0.724631, 0.268707, 0.00615162, 0.000511166, Output 2: 0.13948, 0.845262, 0.0150556, 0.000201825, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2983! Begin processing the 494th record. run: 50574782 subRun: 1 event: 2994 at 21-Sep-2025 19:48:03 CEST Boundary wire vector sizes: 91, 46, 49 minwire 0: 1133 minwire 1: 1614 minwire 2: 936 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00263342, 0.993382, 0.00398483, Output 1: 0.989099, 0.0107313, 7.68424e-05, 9.28536e-05, Output 2: 0.999346, 0.000624236, 1.8134e-05, 1.12806e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1210! Begin processing the 495th record. run: 50574782 subRun: 1 event: 2995 at 21-Sep-2025 19:48:05 CEST Boundary wire vector sizes: 743, 575, 592 minwire 0: 598 minwire 1: 1603 minwire 2: 171 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.000462335, 0.997696, 0.00184121, Output 1: 0.0470584, 0.719874, 0.220514, 0.0125535, Output 2: 0.819943, 0.175129, 0.00485428, 7.33452e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2679! Begin processing the 496th record. run: 50574782 subRun: 1 event: 2996 at 21-Sep-2025 19:48:09 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 902! Begin processing the 497th record. run: 50574782 subRun: 1 event: 2997 at 21-Sep-2025 19:48:11 CEST Boundary wire vector sizes: 2733, 4075, 2398 minwire 0: 1237 minwire 1: 516 minwire 2: 1439 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00100839, 0.971126, 0.0278652, Output 1: 0.845242, 0.14674, 0.00624378, 0.00177407, Output 2: 0.340559, 0.584751, 0.064415, 0.0102745, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2930! Begin processing the 498th record. run: 50574782 subRun: 1 event: 2998 at 21-Sep-2025 19:48:21 CEST Boundary wire vector sizes: 262, 276, 194 minwire 0: 1898 minwire 1: 453 minwire 2: 2239 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00401039, 0.982024, 0.0139658, Output 1: 0.0373384, 0.218124, 0.531063, 0.213474, Output 2: 0.820379, 0.171698, 0.00765057, 0.000272244, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1381! Begin processing the 499th record. run: 50574782 subRun: 1 event: 2999 at 21-Sep-2025 19:48:24 CEST Running Ophitfinder with InputDigiType = 'recob' Found hits: 307! Begin processing the 500th record. run: 50574782 subRun: 1 event: 3000 at 21-Sep-2025 19:48:27 CEST Boundary wire vector sizes: 208, 234, 197 minwire 0: 2042 minwire 1: 1296 minwire 2: 1778 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00459812, 0.889957, 0.105445, Output 1: 0.00969711, 0.876777, 0.110591, 0.00293462, Output 2: 0.99266, 0.0071671, 0.000168233, 4.23683e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2213! Art has completed and will exit with status 0. === End last 100 lines of lar log file === processed files root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a9/ea/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50598687_580_20231204T223216Z_gen_g4_detsim_hitreco__20240509T221711Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/2f/55/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1500_291_20231028T032900Z_gen_g4_detsim_hitreco__20240503T044656Z_reco2.root root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/d6/10/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419113_640_20231203T151708Z_gen_g4_detsim_hitreco__20240508T054537Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ef/4a/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50574782_29_20231203T090238Z_gen_g4_detsim_hitreco__20240509T193907Z_reco2.root .: total 25292 -rw-r--r--. 1 dune012 dune 9542082 Sep 21 19:48 flatcaf.root -rw-r--r--. 1 dune012 dune 7946046 Sep 21 19:48 caf.root -rw-r--r--. 1 dune012 dune 7946046 Sep 21 19:48 caf_fd_hd_atmo_2501_20250921T170525Z.root -rw-r--r--. 1 dune012 dune 306937 Sep 21 19:48 caf_20250921T170525Z.log -rw-r--r--. 1 dune012 dune 112731 Sep 21 19:48 jobscript.log -rw-r--r--. 1 dune012 dune 762 Sep 21 19:48 caf_20250921T170525Z.file -rw-r--r--. 1 dune012 dune 762 Sep 21 19:48 caf_20250921T170525Z.pfns -rw-r--r--. 1 dune012 dune 762 Sep 21 19:05 file.list -rw-r--r--. 1 dune012 dune 762 Sep 21 19:48 justin-processed-pfns.txt -rw-r--r--. 1 dune012 dune 546 Sep 21 19:05 all-input-dids.txt -rw-r--r--. 1 dune012 dune 546 Sep 21 19:48 caf_20250921T170525Z.did -rw-r--r--. 1 dune012 dune 546 Sep 21 19:05 did.list -rw-r--r--. 1 dune012 dune 0 Sep 21 19:05 debugprod.log