Jobsub ID 47492.6@dunegpschedd01.fnal.gov
| Jobsub ID | 47492.6@dunegpschedd01.fnal.gov | |
| Workflow ID | 2501 | |
| Stage ID | 1 | |
| User name | pgranger@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 18000 (5 hours) | |
| Submitted time | 2025-09-17 11:59:10 | |
| Site | NL_NIKHEF | |
| Entry | VIRGO_NL_NIKHEF_dissel | |
| Last heartbeat | 2025-09-17 12:33:49 | |
| From worker node | Hostname | wn-snel-032.farm.nikhef.nl |
| cpuinfo | AMD EPYC 7H12 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 129600 (36 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-09-17 12:00:31 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6454800_156_20231207T092419Z_gen_g4_detsim_hitreco__20240510T045912Z_reco2.root fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_805_20231205T055837Z_gen_g4_detsim_hitreco__20240510T023500Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 32m (1977s) | |
| CPU time | 30m (1821s = 92%) | |
| Max RSS bytes | 2088136704 (1991 MiB) | |
| Outputting started | 2025-09-17 12:33:29 | |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pgranger/cafs/fnal/02501/1/002/caf_fd_hd_atmo_2501_20250917T120040Z.root | |
| Finished | 2025-09-17 12:33:49 | |
| Saved logs | justin-logs:47492.6-dunegpschedd01.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
0.000953109 0.0237555 200
reco:pandoraShower:LArPandoraModularShowerCreation 0.000145032 0.0101048 0.283977 0.0022349 0.0303985 200
reco:pandoracalo:Calorimetry 8.4047e-05 0.0284054 1.70979 0.000700432 0.154473 200
reco:pandorapid:Chi2ParticleID 3.156e-05 0.000124199 0.00115738 9.26785e-05 0.00012114 200
reco:cvnmap:CVNMapper 1.9998e-05 0.00709939 0.0343105 0.00889042 0.00580185 200
reco:cvneva:CVNEvaluator 1.3987e-05 0.22504 1.62338 0.313116 0.176772 200
reco:energyrecnumu:EnergyReco 0.000227536 0.0988126 6.95055 0.00767218 0.528313 200
reco:energyrecnue:EnergyReco 0.000124874 0.00224233 0.0974062 0.000351038 0.00877914 200
reco:energyrecnc:EnergyReco 0.0018652 0.0834131 0.546739 0.0945572 0.0669337 200
reco:energyrecnumurange:EnergyReco 0.000209001 0.0657435 6.08795 0.00223178 0.469902 200
reco:energyrecnumumcs:EnergyReco 0.000516326 0.0658654 6.07401 0.00264767 0.4685 200
reco:energyrecnumumcsllhd:EnergyReco 0.000367117 0.0668025 6.11557 0.00261233 0.473114 200
reco:opdec:Deconvolution 0.0451149 0.21484 0.425928 0.21041 0.0646769 200
reco:ophitspe:OpHitFinderDeco 0.127786 0.133757 0.144857 0.13357 0.00250101 200
reco:opflash:OpFlashFinder 0.000169077 0.00115066 0.00351949 0.00105268 0.000520034 200
reco:opslicer:OpSlicer 4.7489e-05 0.217521 0.79745 0.172886 0.163947 200
reco:rns:RandomNumberSaver 1.8916e-05 2.76983e-05 0.000275095 2.682e-05 1.79294e-05 200
reco:anglereconue:NuAngularReco 0.000148679 0.0015903 0.0671108 0.000321332 0.00633924 200
reco:anglereconumu:NuAngularReco 7.6834e-05 0.00147276 0.0707525 0.000175994 0.00657687 200
reco:anglereconuepfps:NuAngularReco 0.000119403 0.00154694 0.0692874 0.00027492 0.00644478 200
reco:anglereconumupfps:NuAngularReco 0.000109775 0.00150716 0.0682379 0.000238761 0.00640398 200
reco:anglerecohits:NuAngularReco 0.000114775 0.00202485 0.0745527 0.000570483 0.00704087 200
[art]:TriggerResults:TriggerResultInserter 8.636e-06 1.19041e-05 3.4244e-05 1.1802e-05 2.5903e-06 200
end_path:cafmaker:CAFMaker 0.00123583 0.021385 1.17921 0.00369091 0.0971595 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3747.97 MB
Peak resident set size usage (VmHWM): 2088.14 MB
====================================================================================================
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 1: 1266
minwire 2: 2062
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.996071, 0.000605319, 0.00332336,
Output 1: 0.00700389, 0.987059, 0.00578536, 0.000151166,
Output 2: 0.996159, 0.00374653, 9.22219e-05, 2.12595e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1481!
Begin processing the 194th record. run: 66020489 subRun: 1 event: 80594 at 17-Sep-2025 14:33:08 CEST
Boundary wire vector sizes: 1988, 2884, 1660
minwire 0: 1286
minwire 1: 886
minwire 2: 1455
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.634759, 0.262233, 0.103008,
Output 1: 0.257249, 0.513174, 0.155527, 0.0740501,
Output 2: 0.0105676, 0.0379449, 0.186844, 0.764643,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3000!
Begin processing the 195th record. run: 66020489 subRun: 1 event: 80595 at 17-Sep-2025 14:33:14 CEST
Boundary wire vector sizes: 198, 175, 108
minwire 0: 2457
minwire 1: 244
minwire 2: 2383
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.98496, 0.00109932, 0.013941,
Output 1: 0.172584, 0.824343, 0.0029355, 0.000137879,
Output 2: 0.975041, 0.0241854, 0.000744843, 2.83409e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 820!
Begin processing the 196th record. run: 66020489 subRun: 1 event: 80596 at 17-Sep-2025 14:33:16 CEST
Boundary wire vector sizes: 415, 399, 371
minwire 0: 1229
minwire 1: 2313
minwire 2: 661
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.00222621, 0.891288, 0.106486,
Output 1: 0.989451, 0.0102452, 0.000219416, 8.47468e-05,
Output 2: 0.997589, 0.00233602, 5.47189e-05, 2.02862e-05,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2231!
Begin processing the 197th record. run: 66020489 subRun: 1 event: 80597 at 17-Sep-2025 14:33:19 CEST
Boundary wire vector sizes: 226, 373, 289
minwire 0: 0
minwire 1: 1830
minwire 2: 298
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299
Classifier summary:
Output 0: 0.000290838, 0.998324, 0.00138543,
Output 1: 0.00365769, 0.990297, 0.00583952, 0.0002056,
Output 2: 0.997104, 0.00282402, 6.97821e-05, 1.95938e-06,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1218!
Begin processing the 198th record. run: 66020489 subRun: 1 event: 80598 at 17-Sep-2025 14:33:22 CEST
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 686!
Begin processing the 199th record. run: 66020489 subRun: 1 event: 80599 at 17-Sep-2025 14:33:24 CEST
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
in function: GetVolumeIdToHitListMap
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 0!
Begin processing the 200th record. run: 66020489 subRun: 1 event: 80600 at 17-Sep-2025 14:33:25 CEST
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 353!
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
processed files
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/cc/a4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6454800_156_20231207T092419Z_gen_g4_detsim_hitreco__20240510T045912Z_reco2.root
root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/7d/2f/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66020489_805_20231205T055837Z_gen_g4_detsim_hitreco__20240510T023500Z_reco2.root
.:
total 8960
-rw-r--r--. 1 dune012 dune 3448329 Sep 17 14:33 flatcaf.root
-rw-r--r--. 1 dune012 dune 2746343 Sep 17 14:33 caf.root
-rw-r--r--. 1 dune012 dune 2746343 Sep 17 14:33 caf_fd_hd_atmo_2501_20250917T120040Z.root
-rw-r--r--. 1 dune012 dune 129256 Sep 17 14:33 caf_20250917T120040Z.log
-rw-r--r--. 1 dune012 dune 57346 Sep 17 14:33 jobscript.log
-rw-r--r--. 1 dune012 dune 373 Sep 17 14:33 caf_20250917T120040Z.file
-rw-r--r--. 1 dune012 dune 373 Sep 17 14:33 caf_20250917T120040Z.pfns
-rw-r--r--. 1 dune012 dune 373 Sep 17 14:00 file.list
-rw-r--r--. 1 dune012 dune 373 Sep 17 14:33 justin-processed-pfns.txt
-rw-r--r--. 1 dune012 dune 275 Sep 17 14:00 all-input-dids.txt
-rw-r--r--. 1 dune012 dune 275 Sep 17 14:33 caf_20250917T120040Z.did
-rw-r--r--. 1 dune012 dune 275 Sep 17 14:00 did.list
-rw-r--r--. 1 dune012 dune 0 Sep 17 14:00 debugprod.log