Jobsub ID 40043.112@dunegpschedd02.fnal.gov
| Jobsub ID | 40043.112@dunegpschedd02.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-15 14:25:46 | |
| Site | UK_Bristol | |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 | |
| Last heartbeat | 2025-09-15 15:03:52 | |
| From worker node | Hostname | hd85.dice.priv |
| cpuinfo | AMD EPYC 7551P 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 259200 (72 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Started | 2025-09-15 14:26:38 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6435715_94_20231207T041204Z_gen_g4_detsim_hitreco__20240510T055108Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74540126_355_20231207T050353Z_gen_g4_detsim_hitreco__20240510T052307Z_reco2.root | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-09-15 15:03:52 | |
| Saved logs | justin-logs:40043.112-dunegpschedd02.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
138.452 2.1744 15.7611 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000155553 0.00662508 0.149981 0.00175341 0.0209072 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000216107 0.0141688 0.155497 0.00341656 0.0270908 100 reco:pandoracalo:Calorimetry 0.00012733 0.0286603 1.48412 0.00153078 0.159959 100 reco:pandorapid:Chi2ParticleID 4.6267e-05 0.00019659 0.00139253 0.00014341 0.000189173 100 reco:cvnmap:CVNMapper 2.7953e-05 0.012799 0.063687 0.013727 0.0104081 100 reco:cvneva:CVNEvaluator 2.1771e-05 0.6063 3.28792 0.770154 0.445583 100 reco:energyrecnumu:EnergyReco 0.000374255 0.0829226 1.27064 0.0152736 0.184379 100 reco:energyrecnue:EnergyReco 0.000210146 0.00265976 0.0615931 0.000837542 0.00775273 100 reco:energyrecnc:EnergyReco 0.00293891 0.107172 0.296463 0.130701 0.0682803 100 reco:energyrecnumurange:EnergyReco 0.000372351 0.0410787 1.09393 0.00318729 0.15843 100 reco:energyrecnumumcs:EnergyReco 0.000735866 0.0421185 1.08156 0.00404861 0.156891 100 reco:energyrecnumumcsllhd:EnergyReco 0.000565455 0.0418655 1.09829 0.00401499 0.155399 100 reco:opdec:Deconvolution 0.0837895 0.28473 0.574438 0.286179 0.0902651 100 reco:ophitspe:OpHitFinderDeco 0.179446 0.186812 0.198755 0.186514 0.00381166 100 reco:opflash:OpFlashFinder 0.000227869 0.00156513 0.00473437 0.00149869 0.000787238 100 reco:opslicer:OpSlicer 0.000362032 0.315161 0.933272 0.277104 0.248837 100 reco:rns:RandomNumberSaver 2.4767e-05 4.61219e-05 0.000480685 3.7175e-05 4.54286e-05 100 reco:anglereconue:NuAngularReco 0.000177795 0.00147417 0.0399648 0.000429504 0.00490336 100 reco:anglereconumu:NuAngularReco 0.000102393 0.00125698 0.0361375 0.000267143 0.00461125 100 reco:anglereconuepfps:NuAngularReco 0.000151755 0.00137931 0.0345249 0.00038795 0.00447688 100 reco:anglereconumupfps:NuAngularReco 0.000146807 0.0013518 0.0383177 0.000335097 0.00474805 100 reco:anglerecohits:NuAngularReco 0.000411785 0.00217589 0.0428638 0.000842006 0.00554348 100 [art]:TriggerResults:TriggerResultInserter 1.2143e-05 1.59559e-05 8.5351e-05 1.38465e-05 8.34435e-06 100 end_path:cafmaker:CAFMaker 0.00138288 0.0225385 0.545702 0.00578596 0.0711355 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3806.41 MB Peak resident set size usage (VmHWM): 2122.84 MB ==================================================================================================== %MSG-s ArtException: PostEndJob 15-Sep-2025 14:41:15 UTC ModuleEndJob ---- FileOpenError BEGIN RootInputFileSequence::initFile(): Input file root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3e/dc/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74540126_355_20231207T050353Z_gen_g4_detsim_hitreco__20240510T052307Z_reco2.root was not found or could not be opened. ---- FileOpenError END %MSG lar exit code 1 === Start last 100 lines of lar log file === Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00462115, 0.00141246, 0.993966, Output 1: 0.0642166, 0.251593, 0.657071, 0.0271194, Output 2: 0.973374, 0.024816, 0.00175906, 5.14476e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1624! Begin processing the 94th record. run: 6435715 subRun: 1 event: 9494 at 15-Sep-2025 14:39:24 UTC Boundary wire vector sizes: 9671, 9498, 6914 minwire 0: 432 minwire 1: 377 minwire 2: 66 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0220156, 0.9504, 0.0275841, Output 1: 0.362767, 0.558558, 0.0430133, 0.0356611, Output 2: 0.0220937, 0.105605, 0.283744, 0.588557, Running Ophitfinder with InputDigiType = 'recob' Found hits: 3402! Begin processing the 95th record. run: 6435715 subRun: 1 event: 9495 at 15-Sep-2025 14:40:51 UTC Boundary wire vector sizes: 563, 800, 477 minwire 0: 2096 minwire 1: 1141 minwire 2: 1854 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.000311143, 0.998519, 0.00116983, Output 1: 0.0105417, 0.97988, 0.00921807, 0.000359911, Output 2: 0.995531, 0.0043491, 0.000116731, 3.49353e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 2863! Begin processing the 96th record. run: 6435715 subRun: 1 event: 9496 at 15-Sep-2025 14:40:57 UTC Running Ophitfinder with InputDigiType = 'recob' Found hits: 690! Begin processing the 97th record. run: 6435715 subRun: 1 event: 9497 at 15-Sep-2025 14:41:00 UTC Boundary wire vector sizes: 97, 118, 90 minwire 0: 1207 minwire 1: 1296 minwire 2: 1235 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0098833, 0.371763, 0.618354, Output 1: 0.096329, 0.899364, 0.00413383, 0.000173285, Output 2: 0.992089, 0.00761279, 0.000282812, 1.51251e-05, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1256! Begin processing the 98th record. run: 6435715 subRun: 1 event: 9498 at 15-Sep-2025 14:41:04 UTC Boundary wire vector sizes: 111, 134, 134 minwire 0: 404 minwire 1: 1731 minwire 2: 661 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.991619, 0.00168622, 0.00669484, Output 1: 0.0388567, 0.95476, 0.00624295, 0.000140111, Output 2: 0.992435, 0.00742561, 0.000136451, 2.71495e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1222! Begin processing the 99th record. run: 6435715 subRun: 1 event: 9499 at 15-Sep-2025 14:41:08 UTC Running Ophitfinder with InputDigiType = 'recob' Found hits: 293! Begin processing the 100th record. run: 6435715 subRun: 1 event: 9500 at 15-Sep-2025 14:41:11 UTC Boundary wire vector sizes: 164, 104, 133 minwire 0: 606 minwire 1: 2466 minwire 2: 98 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00119587, 0.981559, 0.0172449, Output 1: 0.822329, 0.176498, 0.00112594, 4.69538e-05, Output 2: 0.998839, 0.00113242, 2.63788e-05, 1.69399e-06, Running Ophitfinder with InputDigiType = 'recob' Found hits: 1297! Art has completed and will exit with status 1. === End last 100 lines of lar log file === processed files root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/f9/0d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6435715_94_20231207T041204Z_gen_g4_detsim_hitreco__20240510T055108Z_reco2.root root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/3e/dc/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74540126_355_20231207T050353Z_gen_g4_detsim_hitreco__20240510T052307Z_reco2.root .: total 4144 -rw-r--r-- 1 dnpil001 users 1634547 Sep 15 14:41 flatcaf.root -rw-r--r-- 1 dnpil001 users 1228858 Sep 15 14:41 caf.root -rw-r--r-- 1 dnpil001 users 1228858 Sep 15 14:41 caf_fd_hd_atmo_2501_20250915T142651Z.root -rw-r--r-- 1 dnpil001 users 72677 Sep 15 14:41 caf_20250915T142651Z.log -rw-r--r-- 1 dnpil001 users 30578 Sep 15 14:41 jobscript.log -rw-r--r-- 1 dnpil001 users 392 Sep 15 14:41 caf_20250915T142651Z.file -rw-r--r-- 1 dnpil001 users 392 Sep 15 14:41 caf_20250915T142651Z.pfns -rw-r--r-- 1 dnpil001 users 392 Sep 15 14:26 file.list -rw-r--r-- 1 dnpil001 users 392 Sep 15 14:41 justin-processed-pfns.txt -rw-r--r-- 1 dnpil001 users 274 Sep 15 14:26 all-input-dids.txt -rw-r--r-- 1 dnpil001 users 274 Sep 15 14:41 caf_20250915T142651Z.did -rw-r--r-- 1 dnpil001 users 274 Sep 15 14:26 did.list -rw-r--r-- 1 dnpil001 users 0 Sep 15 14:27 debugprod.log