Jobsub ID 266073.1@dunegpschedd02.fnal.gov
| Jobsub ID | 266073.1@dunegpschedd02.fnal.gov |
| Workflow ID | 11529 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-05 08:00:05 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce03_mcore_op_duneonly |
| Last heartbeat | 2026-01-05 11:36:38 |
| From worker node | Hostname | dunegli-8089765-0-fnpc19110.fnal.gov |
| cpuinfo | AMD EPYC 7502 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-05 08:26:54 |
| Input files | monte-carlo-011529-000001
|
| Jobscript | Exit code | 0 |
| Real time | 3h (11370s) |
| CPU time | 3h (11264s = 99%) |
| Max RSS bytes | 4297408512 (4098 MiB) |
| Outputting started | 2026-01-05 11:36:25 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11529/1/001/reco_stage1.root
|
| Finished | 2026-01-05 11:36:38 |
| Saved logs | justin-logs:266073.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
leKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 172394 traces, 0 summary
Retagger: tagging trace set: gauss with 129827 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 129827 traces tagged "gauss"
FrameSaver: q=4.70912e+06 n=2286099 tag=gauss
wclsFrameSaver: saving 172394 traces tagged "wiener"
FrameSaver: q=4.99386e+06 n=2007145 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36082 XUs and 32212 XVs -> 501 XUVs
C:0 T:1 83263 XUs and 63364 XVs -> 7670 XUVs
C:0 T:2 172486 XUs and 206048 XVs -> 11917 XUVs
C:0 T:3 74983 XUs and 74223 XVs -> 657 XUVs
C:0 T:4 20584 XUs and 15561 XVs -> 144 XUVs
C:0 T:5 107429 XUs and 56942 XVs -> 7034 XUVs
C:0 T:6 172590 XUs and 197869 XVs -> 10281 XUVs
C:0 T:7 54300 XUs and 56328 XVs -> 754 XUVs
C:0 T:8 10070 XUs and 7553 XVs -> 111 XUVs
C:0 T:9 86095 XUs and 73478 XVs -> 7197 XUVs
C:0 T:10 46279 XUs and 22906 XVs -> 4890 XUVs
C:0 T:11 124 XUs and 92 XVs -> 25 XUVs
51181 XUVs total
16898 collection wire objects
45633 potential space points
Neighbour search...
4299975 tests to find 1976966 neighbours
Iterating with no regularization...
Begin: 1.08561e+08
0 8.17486e+07
1 7.9576e+07
2 7.91855e+07
3 7.9048e+07
4 7.89895e+07
Now with regularization...
Begin: 6.29067e+07
0 6.28132e+07
1 6.27688e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411551362
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411551362
fLifetime: 30000 [us]
05-Jan-2026 11:36:23 UTC Closed output file "reco_stage1.root"
05-Jan-2026 11:36:23 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 235.888 330.105 422.1 336.278 62.8895 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000665662 0.00265249 0.00352993 0.00314688 0.00100194 10
reco:ophit:OpHitFinder 0.23182 0.256529 0.290048 0.259849 0.0180842 10
reco:opflash:OpFlashFinder 0.126962 0.161355 0.198204 0.160942 0.0248858 10
reco:opslicer:OpSlicer 0.479905 0.614297 0.757078 0.599734 0.0811452 10
reco:caldata:DataPrepModule 4.2994 4.40721 4.54522 4.38741 0.07692 10
reco:wclsdatasp:WireCellToolkit 26.593 31.3077 33.999 32.0041 2.1748 10
reco:gaushit:GausHitFinder 0.818128 1.03027 1.23949 1.00901 0.147093 10
reco:nhitsfilter:NumberOfHitsFilter 0.000438451 0.00054664 0.000625633 0.000541137 5.71923e-05 10
reco:reco3d:SpacePointSolver 7.18357 13.0401 20.416 12.5106 5.05862 10
reco:hitpdune:DisambigFromSpacePoints 0.969027 1.50564 2.08354 1.53547 0.382507 10
reco:pandora:StandardPandora 18.9615 32.33 48.3321 32.0891 8.88052 10
reco:pandoraWriter:StandardPandora 0.320012 0.414903 0.508468 0.414033 0.0578171 10
reco:pandoraTrack:LArPandoraTrackCreation 2.17396 3.79551 5.19641 3.72572 1.02476 10
reco:pandoraShower:LArPandoraShowerCreation 2.10585 4.26606 6.43313 4.0775 1.4296 10
reco:pandoracalo:Calorimetry 1.10987 1.91271 2.56854 1.95923 0.466977 10
reco:pandoracalonosce:Calorimetry 0.999381 1.71375 2.32221 1.74385 0.425383 10
reco:pandorapid:Chi2ParticleID 0.00401968 0.00507679 0.0057828 0.00512719 0.000489258 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0373461 0.0491713 0.0588732 0.0503207 0.00695574 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00391931 0.00482975 0.00567775 0.00493715 0.000537577 10
reco:pandoraShowercalo:ShowerCalorimetry 1.92742 3.96386 5.91825 3.9178 1.26142 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.7814 3.85406 5.91663 3.78944 1.25866 10
reco:emtrkmichelid:EmTrackMichelId 139.006 181.561 217.146 187.328 27.365 10
reco:crttag:SingleCRTMatchingProducer 0.00789474 0.0115856 0.0187516 0.0113007 0.00281844 10
reco:crtreco:TwoCRTMatchingProducer 0.00581128 0.00839362 0.0151822 0.00822081 0.00262937 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.617025 0.845375 1.0825 0.846103 0.15013 10
reco:pandora2Track:LArPandoraTrackCreation 6.25562 13.414 23.3469 13.296 5.11717 10
reco:pandora2calo:Calorimetry 2.7459 6.05107 10.2123 5.75647 2.4038 10
reco:pandora2calonosce:Calorimetry 2.56364 5.62815 9.54342 5.42601 2.15902 10
reco:pandora2pid:Chi2ParticleID 0.00949287 0.0127464 0.0154257 0.0123978 0.00182235 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0439673 0.0619024 0.0761011 0.0633343 0.0101668 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00925024 0.0128244 0.0153339 0.0126559 0.00190932 10
reco:pandora2Shower:LArPandoraShowerCreation 2.54472 5.22997 8.6921 5.01686 2.04187 10
reco:pandora2Showercalo:ShowerCalorimetry 2.37108 5.04393 7.3522 5.02258 1.68245 10
reco:pandora2Showercalonosce:ShowerCalorimetry 2.16897 4.54553 6.25999 4.74589 1.45262 10
[art]:TriggerResults:TriggerResultInserter 3.922e-05 4.99122e-05 9.9881e-05 4.5745e-05 1.69845e-05 10
end_path:out1:RootOutput 8.81e-06 1.0388e-05 1.457e-05 9.53e-06 1.67526e-06 10
end_path:out1:RootOutput(write) 2.22042 2.92886 3.70071 3.02978 0.491635 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5109.63 MB
Peak resident set size usage (VmHWM): 4297.41 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.73 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.10 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10318232
-rw-r--r-- 1 dunegli fnalgrid 2429944054 Jan 5 10:39 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2306201339 Jan 5 10:37 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2152131116 Jan 5 11:36 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1832769995 Jan 5 08:46 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1761169587 Jan 5 08:31 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 83024610 Jan 5 11:31 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 242878 Jan 5 11:36 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 5 11:36 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 5 11:36 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 5 11:36 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 5 10:39 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 5 08:46 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 5 11:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 5 08:26 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 5 11:36 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 5 10:43 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 5 10:39 debugprod.log
