Jobsub ID 252068.0@dunegpschedd01.fnal.gov
| Jobsub ID | 252068.0@dunegpschedd01.fnal.gov |
| Workflow ID | 10224 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2025-11-15 16:43:53 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2025-11-15 18:39:58 |
| From worker node | Hostname | dunegli-7767682-0-fnpc23031.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-11-15 16:46:33 |
| Input files | monte-carlo-010224-000001
|
| Outputting started | 2025-11-15 18:39:58 |
| Output files | |
| Finished | 2025-11-15 19:23:45 |
| List job events (HTCondor job logs unavailable) |
Jobscript log (last 10,000 characters)
tPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 106586 traces, 0 summary
Retagger: tagging trace set: gauss with 81428 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 81428 traces tagged "gauss"
FrameSaver: q=2.82229e+06 n=1418024 tag=gauss
wclsFrameSaver: saving 106586 traces tagged "wiener"
FrameSaver: q=2.98119e+06 n=1256733 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31146 XUs and 31238 XVs -> 262 XUVs
C:0 T:1 55018 XUs and 51582 XVs -> 2810 XUVs
C:0 T:2 106052 XUs and 100273 XVs -> 6281 XUVs
C:0 T:3 56223 XUs and 43122 XVs -> 292 XUVs
C:0 T:4 14084 XUs and 13321 XVs -> 166 XUVs
C:0 T:5 52198 XUs and 46417 XVs -> 4385 XUVs
C:0 T:6 35451 XUs and 31447 XVs -> 1770 XUVs
C:0 T:7 18508 XUs and 16302 XVs -> 52 XUVs
C:0 T:8 5693 XUs and 6844 XVs -> 34 XUVs
C:0 T:9 30915 XUs and 47601 XVs -> 2725 XUVs
C:0 T:10 18613 XUs and 7407 XVs -> 1778 XUVs
C:0 T:11 138 XUs and 81 XVs -> 18 XUVs
20573 XUVs total
8592 collection wire objects
19357 potential space points
Neighbour search...
671849 tests to find 389744 neighbours
Iterating with no regularization...
Begin: 5.77908e+07
0 5.15409e+07
1 5.10126e+07
2 5.09382e+07
3 5.09186e+07
Now with regularization...
Begin: 4.26621e+07
0 4.2647e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410532699
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410532699
fLifetime: 30000 [us]
15-Nov-2025 18:39:57 UTC Closed output file "reco_stage1.root"
15-Nov-2025 18:39:57 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 137.08 193.705 244.058 191.386 30.8047 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000847773 0.00246859 0.00396731 0.00290095 0.00102191 10
reco:ophit:OpHitFinder 0.172472 0.1958 0.215593 0.198217 0.0119614 10
reco:opflash:OpFlashFinder 0.0916 0.11837 0.140199 0.116763 0.015096 10
reco:opslicer:OpSlicer 0.330154 0.440013 0.531259 0.438584 0.0616028 10
reco:caldata:DataPrepModule 3.41685 3.49078 3.64384 3.4596 0.0717212 10
reco:wclsdatasp:WireCellToolkit 21.5996 22.9587 23.6302 23.0111 0.528465 10
reco:gaushit:GausHitFinder 0.482895 0.683024 0.88719 0.696466 0.120644 10
reco:nhitsfilter:NumberOfHitsFilter 0.000238958 0.000319249 0.000430675 0.000313104 5.32864e-05 10
reco:reco3d:SpacePointSolver 6.15168 9.53167 15.0526 9.52851 2.73687 10
reco:hitpdune:DisambigFromSpacePoints 0.470827 0.896952 1.28233 0.876768 0.221794 10
reco:pandora:StandardPandora 7.15643 16.9814 25.2463 17.4517 4.79525 10
reco:pandoraWriter:StandardPandora 0.16858 0.236313 0.297784 0.235114 0.0350321 10
reco:pandoraTrack:LArPandoraTrackCreation 1.21817 1.85282 2.59464 1.82398 0.362089 10
reco:pandoraShower:LArPandoraShowerCreation 0.835787 2.06467 3.54996 1.95804 0.712626 10
reco:pandoracalo:Calorimetry 0.689437 1.13696 1.67084 1.12751 0.262472 10
reco:pandoracalonosce:Calorimetry 0.631004 1.04416 1.52259 1.03713 0.240889 10
reco:pandorapid:Chi2ParticleID 0.00254749 0.00320952 0.0042779 0.00306999 0.000444245 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0213779 0.029799 0.0385771 0.0306838 0.00436447 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00241069 0.00284787 0.0034449 0.00279273 0.00026949 10
reco:pandoraShowercalo:ShowerCalorimetry 0.861329 2.31829 4.07236 2.26416 0.87847 10
reco:pandoraShowercalonosce:ShowerCalorimetry 0.834084 2.26067 4.00905 2.20325 0.870707 10
reco:emtrkmichelid:EmTrackMichelId 77.4448 104.711 129.114 101.521 14.7784 10
reco:crttag:SingleCRTMatchingProducer 0.00478546 0.007286 0.00974824 0.00711051 0.00134959 10
reco:crtreco:TwoCRTMatchingProducer 0.00341105 0.00464743 0.00590418 0.00452122 0.00076142 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.402446 0.553821 0.724701 0.553086 0.099085 10
reco:pandora2Track:LArPandoraTrackCreation 2.84598 5.99822 9.20754 5.68383 1.80707 10
reco:pandora2calo:Calorimetry 1.49475 3.31599 5.53811 3.23179 1.11348 10
reco:pandora2calonosce:Calorimetry 1.40822 3.13912 5.01506 3.0606 1.03166 10
reco:pandora2pid:Chi2ParticleID 0.00501443 0.00762331 0.00993737 0.007224 0.00145429 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0263334 0.036943 0.0473987 0.0371316 0.00583023 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00480748 0.00715957 0.00960104 0.00706341 0.00120238 10
reco:pandora2Shower:LArPandoraShowerCreation 1.13367 2.27147 3.33863 2.16512 0.626982 10
reco:pandora2Showercalo:ShowerCalorimetry 1.31834 2.89081 4.51294 2.80977 0.917358 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.23527 2.69102 4.11305 2.60971 0.848083 10
[art]:TriggerResults:TriggerResultInserter 2.8595e-05 4.37168e-05 0.000108127 3.3985e-05 2.35229e-05 10
end_path:out1:RootOutput 5.28e-06 1.00812e-05 1.4167e-05 1.03745e-05 2.69861e-06 10
end_path:out1:RootOutput(write) 1.33836 1.76588 2.28382 1.7158 0.265301 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4919.1 MB
Peak resident set size usage (VmHWM): 4109.05 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.15 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.19 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.08 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.03 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9621776
-rw-r--r-- 1 dunegli fnalgrid 2279381922 Nov 15 18:06 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2165591398 Nov 15 18:05 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1948762878 Nov 15 18:39 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1733774898 Nov 15 16:56 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1653988768 Nov 15 16:49 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 70575800 Nov 15 18:38 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 242910 Nov 15 18:39 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Nov 15 18:39 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Nov 15 18:39 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Nov 15 18:39 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 18:06 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 16:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Nov 15 18:39 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Nov 15 16:46 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Nov 15 18:39 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Nov 15 18:09 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Nov 15 18:06 debugprod.log
