Jobsub ID 279035.0@dunegpschedd01.fnal.gov
| Jobsub ID | 279035.0@dunegpschedd01.fnal.gov |
| Workflow ID | 11960 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-15 10:55:47 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2026-01-15 16:10:54 |
| From worker node | Hostname | dunegli-8324234-0-fnpc18005.fnal.gov |
| cpuinfo | AMD EPYC 7551 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-15 10:56:55 |
| Input files | monte-carlo-011960-000001
|
| Jobscript | Exit code | 0 |
| Real time | 5h (18816s) |
| CPU time | 5h (18288s = 97%) |
| Max RSS bytes | 4654411776 (4438 MiB) |
| Outputting started | 2026-01-15 16:10:31 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11960/1/001/reco_stage1.root
|
| Finished | 2026-01-15 16:10:54 |
| Saved logs | justin-logs:279035.0-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
aleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148457 traces, 0 summary
Retagger: tagging trace set: gauss with 112092 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 112092 traces tagged "gauss"
FrameSaver: q=4.22294e+06 n=1968366 tag=gauss
wclsFrameSaver: saving 148457 traces tagged "wiener"
FrameSaver: q=4.5235e+06 n=1729264 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44236 XUs and 47366 XVs -> 841 XUVs
C:0 T:1 119170 XUs and 103951 XVs -> 11212 XUVs
C:0 T:2 103062 XUs and 144467 XVs -> 8272 XUVs
C:0 T:3 44040 XUs and 45962 XVs -> 271 XUVs
C:0 T:4 13961 XUs and 17490 XVs -> 499 XUVs
C:0 T:5 66845 XUs and 68389 XVs -> 5866 XUVs
C:0 T:6 84259 XUs and 77131 XVs -> 4002 XUVs
C:0 T:7 33382 XUs and 28690 XVs -> 235 XUVs
C:0 T:8 9409 XUs and 8551 XVs -> 103 XUVs
C:0 T:9 92339 XUs and 118293 XVs -> 8567 XUVs
C:0 T:10 51444 XUs and 43619 XVs -> 5075 XUVs
C:0 T:11 406 XUs and 254 XVs -> 18 XUVs
44961 XUVs total
15724 collection wire objects
40625 potential space points
Neighbour search...
3849433 tests to find 1494754 neighbours
Iterating with no regularization...
Begin: 1.89649e+08
0 1.30628e+08
1 1.25592e+08
2 1.25037e+08
3 1.24897e+08
4 1.24844e+08
Now with regularization...
Begin: 1.01245e+08
0 1.0097e+08
1 1.00903e+08
run: 1 ; subrun: 0 ; event: 10
evttime: 411754968
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411754968
fLifetime: 30000 [us]
15-Jan-2026 16:10:29 UTC Closed output file "reco_stage1.root"
15-Jan-2026 16:10:29 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 419.677 600.402 789.717 603.239 109.412 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00173678 0.00434086 0.00560543 0.00496348 0.00130705 10
reco:ophit:OpHitFinder 0.32488 0.409264 0.563355 0.403312 0.065179 10
reco:opflash:OpFlashFinder 0.208792 0.288485 0.387959 0.285688 0.0597429 10
reco:opslicer:OpSlicer 0.806962 1.05318 1.54406 0.972851 0.217424 10
reco:caldata:DataPrepModule 6.4284 7.25595 9.1876 6.80255 0.975468 10
reco:wclsdatasp:WireCellToolkit 42.2068 47.5958 56.4535 46.6737 4.22923 10
reco:gaushit:GausHitFinder 1.16718 1.70954 2.34266 1.66625 0.286315 10
reco:nhitsfilter:NumberOfHitsFilter 0.000520942 0.00080185 0.0012743 0.000748851 0.000228876 10
reco:reco3d:SpacePointSolver 12.2177 22.1674 39.5413 20.2307 7.50396 10
reco:hitpdune:DisambigFromSpacePoints 1.63158 2.60481 4.28218 2.39804 0.717599 10
reco:pandora:StandardPandora 24.8272 63.0619 86.8036 66.1336 18.5326 10
reco:pandoraWriter:StandardPandora 0.465555 1.00414 4.04527 0.706266 1.01878 10
reco:pandoraTrack:LArPandoraTrackCreation 5.74084 9.65744 12.072 10.0932 2.0399 10
reco:pandoraShower:LArPandoraShowerCreation 5.08588 10.9243 14.4919 11.4249 3.02898 10
reco:pandoracalo:Calorimetry 2.82458 5.12136 7.16864 4.97828 1.38018 10
reco:pandoracalonosce:Calorimetry 2.80671 4.86888 7.11076 4.78218 1.33948 10
reco:pandorapid:Chi2ParticleID 0.00824279 0.00959827 0.0120695 0.00917301 0.00113495 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0524609 0.0748862 0.108315 0.0702196 0.0160392 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00782409 0.00891122 0.0116261 0.00838178 0.00121658 10
reco:pandoraShowercalo:ShowerCalorimetry 5.18562 11.0085 16.5272 10.5504 3.48726 10
reco:pandoraShowercalonosce:ShowerCalorimetry 5.08921 10.5607 15.8673 10.1266 3.34154 10
reco:emtrkmichelid:EmTrackMichelId 217.135 299.397 436.207 276.378 74.4307 10
reco:crttag:SingleCRTMatchingProducer 0.0148924 0.0258528 0.0394478 0.0256519 0.0082001 10
reco:crtreco:TwoCRTMatchingProducer 0.0125727 0.0184081 0.0273905 0.0177703 0.00536431 10
reco:anodepiercerst0:T0RecoAnodePiercers 1.04019 1.29123 1.78903 1.24913 0.248588 10
reco:pandora2Track:LArPandoraTrackCreation 16.6565 29.9546 38.6967 31.3462 6.28307 10
reco:pandora2calo:Calorimetry 7.81022 14.6002 21.5242 14.6789 3.98649 10
reco:pandora2calonosce:Calorimetry 7.2419 13.992 20.3484 13.8098 4.09193 10
reco:pandora2pid:Chi2ParticleID 0.0179414 0.0243327 0.0335846 0.0224677 0.00479928 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0703937 0.0944219 0.126824 0.0941127 0.01533 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.0149608 0.0214856 0.0308599 0.0213022 0.00405552 10
reco:pandora2Shower:LArPandoraShowerCreation 6.00604 10.8989 15.4208 11.2549 2.88171 10
reco:pandora2Showercalo:ShowerCalorimetry 6.57153 12.2451 17.9976 11.2667 3.8104 10
reco:pandora2Showercalonosce:ShowerCalorimetry 6.08642 10.9288 17.4278 9.651 3.51805 10
[art]:TriggerResults:TriggerResultInserter 5.0816e-05 8.25855e-05 0.000165292 7.183e-05 3.43192e-05 10
end_path:out1:RootOutput 5.68e-06 1.18341e-05 2.0208e-05 1.11155e-05 3.84556e-06 10
end_path:out1:RootOutput(write) 3.63341 7.33855 16.6665 5.05713 4.40244 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5467.53 MB
Peak resident set size usage (VmHWM): 4654.41 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.11 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.29 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 4.61 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.16 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.07 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11437676
-rw-r--r-- 1 dunegli fnalgrid 2648845446 Jan 15 14:25 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2518347031 Jan 15 14:20 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2367386881 Jan 15 16:10 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2083488358 Jan 15 11:24 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2010420674 Jan 15 11:05 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 83063770 Jan 15 16:01 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 243394 Jan 15 16:10 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 15 16:10 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 15 16:10 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 15 16:10 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 15 14:25 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 15 11:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 15 16:10 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 15 10:56 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 15 16:10 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 15 14:33 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 15 14:25 debugprod.log
