Jobsub ID 46601.109@dunegpschedd01.fnal.gov
| Jobsub ID | 46601.109@dunegpschedd01.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-15 12:17:41 | |
| Site | BR_CBPF | |
| Entry | DUNE_BR_CBPF_ce01 | |
| Last heartbeat | 2025-09-15 13:04:44 | |
| From worker node | Hostname | wn39 |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Started | 2025-09-15 12:19:38 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_789_20231208T050858Z_gen_g4_detsim_hitreco__20240510T042534Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505548_743_20231202T143705Z_gen_g4_detsim_hitreco__20240508T055706Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_336_20231202T081731Z_gen_g4_detsim_hitreco__20240507T204833Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_450_20231208T064310Z_gen_g4_detsim_hitreco__20240510T050412Z_reco2.root | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-09-15 13:04:44 | |
| Saved logs | justin-logs:46601.109-dunegpschedd01.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
larShowerCreation 0.000305708 0.0205681 0.372616 0.00594712 0.0488149 100 reco:pandoracalo:Calorimetry 0.000190963 0.0413002 1.98057 0.00206698 0.206784 100 reco:pandorapid:Chi2ParticleID 7.0449e-05 0.000333962 0.00644572 0.000236424 0.000633496 100 reco:cvnmap:CVNMapper 3.9426e-05 0.0190183 0.0572832 0.0189228 0.012325 100 reco:cvneva:CVNEvaluator 3.4693e-05 0.65061 3.79737 0.778462 0.458409 100 reco:energyrecnumu:EnergyReco 0.000621775 0.423374 5.16844 0.0510013 0.703567 100 reco:energyrecnue:EnergyReco 0.000328376 0.00363269 0.0433618 0.00141227 0.00644117 100 reco:energyrecnc:EnergyReco 0.00301108 1.00822 2.17124 1.24638 0.632444 100 reco:energyrecnumurange:EnergyReco 0.000394425 0.0933579 4.85077 0.00516883 0.49116 100 reco:energyrecnumumcs:EnergyReco 0.000999057 0.0924831 4.81494 0.00552705 0.487604 100 reco:energyrecnumumcsllhd:EnergyReco 0.000828354 0.0921648 4.80063 0.0062535 0.48618 100 reco:opdec:Deconvolution 0.628042 1.07758 2.11331 1.0477 0.217147 100 reco:ophitspe:OpHitFinderDeco 1.83705 1.84772 1.90916 1.84702 0.00789995 100 reco:opflash:OpFlashFinder 0.000360694 0.0021581 0.0143765 0.00185551 0.00164317 100 reco:opslicer:OpSlicer 0.000348909 0.37535 1.46252 0.286122 0.299482 100 reco:rns:RandomNumberSaver 3.1355e-05 6.53202e-05 0.00054249 5.74645e-05 4.96066e-05 100 reco:anglereconue:NuAngularReco 0.000254435 0.00197039 0.0292215 0.000746445 0.00376944 100 reco:anglereconumu:NuAngularReco 0.000161907 0.00168191 0.026667 0.000473928 0.0035394 100 reco:anglereconuepfps:NuAngularReco 0.000215917 0.00188352 0.0269578 0.000621163 0.00358748 100 reco:anglereconumupfps:NuAngularReco 0.000205248 0.00182684 0.0270435 0.00061673 0.00359643 100 reco:anglerecohits:NuAngularReco 0.000182722 0.00305561 0.0341429 0.00130608 0.00510813 100 [art]:TriggerResults:TriggerResultInserter 1.7279e-05 2.38369e-05 7.7836e-05 2.29075e-05 6.83033e-06 100 end_path:cafmaker:CAFMaker 0.00184631 0.0322455 0.51019 0.010213 0.0723944 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3805.56 MB Peak resident set size usage (VmHWM): 2128.87 MB ==================================================================================================== %MSG-s ArtException: PostEndJob 15-Sep-2025 10:00:15 -03 ModuleEndJob ---- FileOpenError BEGIN RootInputFileSequence::initFile(): Input file root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d0/4f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505548_743_20231202T143705Z_gen_g4_detsim_hitreco__20240508T055706Z_reco2.root was not found or could not be opened. ---- FileOpenError END %MSG lar exit code 1 === Start last 100 lines of lar log file === Running Ophitfinder with InputDigiType = 'recob' Found hits: 2520! Begin processing the 92nd record. run: 6492179 subRun: 1 event: 78992 at 15-Sep-2025 09:58:23 -03 Running Ophitfinder with InputDigiType = 'recob' Found hits: 788! Begin processing the 93rd record. run: 6492179 subRun: 1 event: 78993 at 15-Sep-2025 09:58:34 -03 Boundary wire vector sizes: 167, 101, 112 minwire 0: 680 minwire 1: 1750 minwire 2: 793 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00742424, 0.0217728, 0.970803, Output 1: 0.867078, 0.125192, 0.0066821, 0.00104779, Output 2: 0.995148, 0.00473195, 0.000109455, 1.04227e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 945! Begin processing the 94th record. run: 6492179 subRun: 1 event: 78994 at 15-Sep-2025 09:58:45 -03 Boundary wire vector sizes: 181, 219, 100 minwire 0: 1547 minwire 1: 1310 minwire 2: 1438 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.997027, 0.000293263, 0.00267986, Output 1: 0.00368852, 0.110286, 0.860762, 0.0252636, Output 2: 0.973508, 0.0253753, 0.00110063, 1.59133e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1770! Begin processing the 95th record. run: 6492179 subRun: 1 event: 78995 at 15-Sep-2025 09:58:57 -03 Boundary wire vector sizes: 677, 450, 487 minwire 0: 1406 minwire 1: 1571 minwire 2: 1115 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00165266, 0.987888, 0.0104596, Output 1: 0.0429162, 0.242004, 0.41032, 0.30476, Output 2: 0.675326, 0.292293, 0.02987, 0.00251109, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2710! Begin processing the 96th record. run: 6492179 subRun: 1 event: 78996 at 15-Sep-2025 09:59:14 -03 Running Ophitfinder with InputDigiType = 'recob' Found hits: 553! Begin processing the 97th record. run: 6492179 subRun: 1 event: 78997 at 15-Sep-2025 09:59:25 -03 Running Ophitfinder with InputDigiType = 'recob' Found hits: 891! Begin processing the 98th record. run: 6492179 subRun: 1 event: 78998 at 15-Sep-2025 09:59:35 -03 Running Ophitfinder with InputDigiType = 'recob' Found hits: 658! Begin processing the 99th record. run: 6492179 subRun: 1 event: 78999 at 15-Sep-2025 09:59:45 -03 Boundary wire vector sizes: 1109, 1481, 760 minwire 0: 1286 minwire 1: 1082 minwire 2: 1362 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00736472, 0.929754, 0.0628809, Output 1: 0.026661, 0.148146, 0.197262, 0.627931, Output 2: 0.414154, 0.474861, 0.0930178, 0.0179673, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2473! Begin processing the 100th record. run: 6492179 subRun: 1 event: 79000 at 15-Sep-2025 10:00:00 -03 Boundary wire vector sizes: 91, 124, 97 minwire 0: 1531 minwire 1: 1207 minwire 2: 1398 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0864699, 0.0123926, 0.901138, Output 1: 0.00428062, 0.0273801, 0.0590004, 0.909339, Output 2: 0.978666, 0.0203434, 0.000877854, 0.000112321, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2251! Art has completed and will exit with status 1. === End last 100 lines of lar log file === processed files root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/b9/28/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_789_20231208T050858Z_gen_g4_detsim_hitreco__20240510T042534Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/d0/4f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74505548_743_20231202T143705Z_gen_g4_detsim_hitreco__20240508T055706Z_reco2.root root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/89/60/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_336_20231202T081731Z_gen_g4_detsim_hitreco__20240507T204833Z_reco2.root root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/b0/e3/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472619_450_20231208T064310Z_gen_g4_detsim_hitreco__20240510T050412Z_reco2.root .: total 5212 -rw-r--r-- 1 nobody nobody 2032734 Sep 15 10:00 flatcaf.root -rw-r--r-- 1 nobody nobody 1572878 Sep 15 10:00 caf.root -rw-r--r-- 1 nobody nobody 1572878 Sep 15 10:00 caf_fd_hd_atmo_2501_20250915T121959Z.root -rw-r--r-- 1 nobody nobody 77738 Sep 15 10:00 caf_20250915T121959Z.log -rw-r--r-- 1 nobody nobody 28087 Sep 15 10:00 jobscript.log -rw-r--r-- 1 nobody nobody 749 Sep 15 10:00 caf_20250915T121959Z.file -rw-r--r-- 1 nobody nobody 749 Sep 15 10:00 caf_20250915T121959Z.pfns -rw-r--r-- 1 nobody nobody 749 Sep 15 09:19 file.list -rw-r--r-- 1 nobody nobody 749 Sep 15 10:00 justin-processed-pfns.txt -rw-r--r-- 1 nobody nobody 549 Sep 15 09:19 all-input-dids.txt -rw-r--r-- 1 nobody nobody 549 Sep 15 10:00 caf_20250915T121959Z.did -rw-r--r-- 1 nobody nobody 549 Sep 15 09:19 did.list -rw-r--r-- 1 nobody nobody 0 Sep 15 09:21 debugprod.log