Jobsub ID 41104.80@dunegpschedd02.fnal.gov
| Jobsub ID | 41104.80@dunegpschedd02.fnal.gov |
| Workflow ID | 2501 |
| Stage ID | 1 |
| User name | pgranger@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-17 17:17:32 |
| Site | UK_Bristol |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 |
| Last heartbeat | 2025-09-17 17:26:44 |
| From worker node | Hostname | hd87.dice.priv |
| cpuinfo | AMD EPYC 7702P 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 259200 (72 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-09-17 17:18:37 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_497_20231208T003836Z_gen_g4_detsim_hitreco__20240510T030741Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 7m (457s) |
| CPU time | 4m (292s = 63%) |
| Max RSS bytes | 1959391232 (1868 MiB) |
| Outputting started | 2025-09-17 17:26:15 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T171846Z.root
|
| Finished | 2025-09-17 17:26:44 |
| Saved logs | justin-logs:41104.80-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
0.0726799 1.75261 40.0665 1.19903 3.89304 100
reco:pandoraTrack:LArPandoraTrackCreation 0.0001097 0.00403724 0.0632097 0.00147254 0.00852609 100
reco:pandoraShower:LArPandoraModularShowerCreation 0.000149396 0.0386144 1.46694 0.00299152 0.195343 100
reco:pandoracalo:Calorimetry 8.7317e-05 0.00974156 0.395516 0.00113371 0.0413723 100
reco:pandorapid:Chi2ParticleID 3.3374e-05 0.000142794 0.00143492 0.000107791 0.00016893 100
reco:cvnmap:CVNMapper 1.6912e-05 0.0488356 2.03952 0.00933337 0.232076 100
reco:cvneva:CVNEvaluator 1.4337e-05 0.291926 1.936 0.328132 0.237743 100
reco:energyrecnumu:EnergyReco 0.000264427 0.0447095 0.54634 0.00875604 0.0797328 100
reco:energyrecnue:EnergyReco 0.000126492 0.00124772 0.0135032 0.000493395 0.00209276 100
reco:energyrecnc:EnergyReco 0.00191293 0.0815886 1.16179 0.0786512 0.117295 100
reco:energyrecnumurange:EnergyReco 0.000214411 0.0191032 0.401758 0.00215152 0.0560358 100
reco:energyrecnumumcs:EnergyReco 0.000376019 0.0197465 0.403089 0.00287494 0.0550067 100
reco:energyrecnumumcsllhd:EnergyReco 0.000279595 0.0197936 0.398062 0.00296822 0.0556614 100
reco:opdec:Deconvolution 0.0588697 0.195241 0.395503 0.194538 0.0532878 100
reco:ophitspe:OpHitFinderDeco 0.0973055 0.379891 19.521 0.102382 1.95331 100
reco:opflash:OpFlashFinder 0.000146861 0.0010788 0.00406543 0.00107262 0.000483696 100
reco:opslicer:OpSlicer 4.6499e-05 0.221551 1.01887 0.189849 0.163836 100
reco:rns:RandomNumberSaver 1.9758e-05 2.99277e-05 0.000306416 2.632e-05 2.82122e-05 100
reco:anglereconue:NuAngularReco 0.000148294 0.00085756 0.00927023 0.000366281 0.00151446 100
reco:anglereconumu:NuAngularReco 6.778e-05 0.000718454 0.00959635 0.000212237 0.0015575 100
reco:anglereconuepfps:NuAngularReco 0.000105692 0.00081991 0.00938013 0.000306537 0.00156257 100
reco:anglereconumupfps:NuAngularReco 9.3008e-05 0.000765155 0.00956278 0.000262207 0.00154212 100
reco:anglerecohits:NuAngularReco 7.9522e-05 0.00126767 0.0115366 0.000608845 0.00190887 100
[art]:TriggerResults:TriggerResultInserter 8.366e-06 1.08873e-05 4.9365e-05 9.8635e-06 5.02403e-06 100
end_path:cafmaker:CAFMaker 0.00127347 0.417249 40.059 0.00408018 3.98455 100
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3656.19 MB
Peak resident set size usage (VmHWM): 1959.39 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 1: 2189
minwire 2: 15
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.995069, 0.000654832, 0.00427588,
Output 1: 0.101738, 0.81884, 0.0774947, 0.00192715,
Output 2: 0.97871, 0.0206302, 0.000645933, 1.42351e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1413!
Begin processing the 94th record. run: 6470038 subRun: 1 event: 49794 at 17-Sep-2025 17:25:56 UTC
Boundary wire vector sizes: 267, 110, 109
minwire 0: 1003
minwire 1: 1544
minwire 2: 459
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.618503, 0.270551, 0.110946,
Output 1: 0.94838, 0.0501001, 0.00135565, 0.000164123,
Output 2: 0.218291, 0.768962, 0.012528, 0.000219434,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2313!
Begin processing the 95th record. run: 6470038 subRun: 1 event: 49795 at 17-Sep-2025 17:25:59 UTC
Boundary wire vector sizes: 109, 93, 93
minwire 0: 1428
minwire 1: 633
minwire 2: 1462
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0190971, 0.00187191, 0.979031,
Output 1: 0.0803302, 0.912837, 0.00640819, 0.000424206,
Output 2: 0.822789, 0.173346, 0.00378802, 7.71408e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1141!
Begin processing the 96th record. run: 6470038 subRun: 1 event: 49796 at 17-Sep-2025 17:26:01 UTC
Boundary wire vector sizes: 2793, 1978, 2528
minwire 0: 698
minwire 1: 463
minwire 2: 636
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.975478, 0.00537904, 0.0191428,
Output 1: 0.765555, 0.225452, 0.00807387, 0.000918686,
Output 2: 0.123206, 0.779311, 0.0910056, 0.00647747,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3069!
Begin processing the 97th record. run: 6470038 subRun: 1 event: 49797 at 17-Sep-2025 17:26:06 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1822!
Begin processing the 98th record. run: 6470038 subRun: 1 event: 49798 at 17-Sep-2025 17:26:08 UTC
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 0!
Begin processing the 99th record. run: 6470038 subRun: 1 event: 49799 at 17-Sep-2025 17:26:09 UTC
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1115!
Begin processing the 100th record. run: 6470038 subRun: 1 event: 49800 at 17-Sep-2025 17:26:10 UTC
Boundary wire vector sizes: 98, 109, 79
minwire 0: 2469
minwire 1: 952
minwire 2: 2253
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.0877252, 0.00958361, 0.902691,
Output 1: 0.0100291, 0.476832, 0.500617, 0.0125226,
Output 2: 0.973823, 0.0253787, 0.000785062, 1.27261e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1739!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/6d/8d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_497_20231208T003836Z_gen_g4_detsim_hitreco__20240510T030741Z_reco2.root
.:
total 4091
-rw-r--r-- 1 dnpil001 users 1633485 Sep 17 17:26 flatcaf.root
-rw-r--r-- 1 dnpil001 users 1222790 Sep 17 17:26 caf.root
-rw-r--r-- 1 dnpil001 users 1222790 Sep 17 17:26 caf_fd_hd_atmo_2501_20250917T171846Z.root
-rw-r--r-- 1 dnpil001 users 75702 Sep 17 17:26 caf_20250917T171846Z.log
-rw-r--r-- 1 dnpil001 users 29634 Sep 17 17:26 jobscript.log
-rw-r--r-- 1 dnpil001 users 218 Sep 17 17:26 caf_20250917T171846Z.file
-rw-r--r-- 1 dnpil001 users 218 Sep 17 17:26 caf_20250917T171846Z.pfns
-rw-r--r-- 1 dnpil001 users 218 Sep 17 17:18 file.list
-rw-r--r-- 1 dnpil001 users 218 Sep 17 17:26 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users 137 Sep 17 17:18 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users 137 Sep 17 17:26 caf_20250917T171846Z.did
-rw-r--r-- 1 dnpil001 users 137 Sep 17 17:18 did.list
-rw-r--r-- 1 dnpil001 users 0 Sep 17 17:19 debugprod.log 