Jobsub ID 296617.1@dunegpschedd02.fnal.gov
| Jobsub ID | 296617.1@dunegpschedd02.fnal.gov |
| Workflow ID | 12543 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-30 07:26:54 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2026-01-30 10:02:45 |
| From worker node | Hostname | dunegli-8484075-0-fnpc19130.fnal.gov |
| cpuinfo | AMD EPYC 7502 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-30 07:28:08 |
| Input files | monte-carlo-012543-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9236s) |
| CPU time | 2h (9154s = 99%) |
| Max RSS bytes | 4294279168 (4095 MiB) |
| Outputting started | 2026-01-30 10:02:26 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12543/1/001/reco_stage1.root
|
| Finished | 2026-01-30 10:02:45 |
| Saved logs | justin-logs:296617.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 106979 traces, 0 summary
Retagger: tagging trace set: gauss with 80976 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80976 traces tagged "gauss"
FrameSaver: q=2.68302e+06 n=1424564 tag=gauss
wclsFrameSaver: saving 106979 traces tagged "wiener"
FrameSaver: q=2.85194e+06 n=1253543 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23650 XUs and 14131 XVs -> 306 XUVs
C:0 T:1 35615 XUs and 26330 XVs -> 2308 XUVs
C:0 T:2 49257 XUs and 61073 XVs -> 2697 XUVs
C:0 T:3 26235 XUs and 28900 XVs -> 356 XUVs
C:0 T:4 20150 XUs and 13646 XVs -> 2489 XUVs
C:0 T:5 37288 XUs and 16889 XVs -> 2862 XUVs
C:0 T:6 86392 XUs and 82811 XVs -> 5257 XUVs
C:0 T:7 41224 XUs and 51884 XVs -> 3661 XUVs
C:0 T:8 11159 XUs and 7863 XVs -> 200 XUVs
C:0 T:9 136043 XUs and 91696 XVs -> 12688 XUVs
C:0 T:10 30003 XUs and 19960 XVs -> 3561 XUVs
C:0 T:11 213 XUs and 153 XVs -> 20 XUVs
36405 XUVs total
10515 collection wire objects
33098 potential space points
Neighbour search...
3304892 tests to find 1568330 neighbours
Iterating with no regularization...
Begin: 6.27298e+07
0 3.77007e+07
1 3.62791e+07
2 3.60271e+07
3 3.5947e+07
4 3.59108e+07
5 3.58905e+07
Now with regularization...
Begin: 2.8123e+07
0 2.80786e+07
1 2.80521e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412053840
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412053840
fLifetime: 30000 [us]
30-Jan-2026 10:02:04 UTC Closed output file "reco_stage1.root"
30-Jan-2026 10:02:04 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 205.782 287.593 422.205 286.322 55.9967 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000817957 0.0027865 0.00383198 0.00325714 0.00106872 10
reco:ophit:OpHitFinder 0.186504 0.22946 0.295273 0.228923 0.0287178 10
reco:opflash:OpFlashFinder 0.0964948 0.134145 0.206249 0.128465 0.030465 10
reco:opslicer:OpSlicer 0.403058 0.555312 0.742333 0.555307 0.0858098 10
reco:caldata:DataPrepModule 4.15335 4.30558 4.4907 4.26737 0.101675 10
reco:wclsdatasp:WireCellToolkit 26.7508 30.1605 33.3324 29.8411 2.17819 10
reco:gaushit:GausHitFinder 0.652616 0.923064 1.27122 0.909919 0.151257 10
reco:nhitsfilter:NumberOfHitsFilter 0.000386064 0.000489721 0.000642685 0.000484824 6.8949e-05 10
reco:reco3d:SpacePointSolver 6.8002 12.3928 19.5521 11.6211 4.03747 10
reco:hitpdune:DisambigFromSpacePoints 0.742902 1.21327 2.09206 1.17246 0.362038 10
reco:pandora:StandardPandora 14.3312 28.0793 57.0321 25.3544 11.1621 10
reco:pandoraWriter:StandardPandora 0.282954 0.372685 0.504188 0.371077 0.0583698 10
reco:pandoraTrack:LArPandoraTrackCreation 1.98787 3.42481 4.92703 3.32437 1.03811 10
reco:pandoraShower:LArPandoraShowerCreation 1.6331 3.42062 6.55552 3.13274 1.39579 10
reco:pandoracalo:Calorimetry 0.986506 1.5876 2.30417 1.5584 0.415299 10
reco:pandoracalonosce:Calorimetry 0.876781 1.46122 2.11713 1.4204 0.40615 10
reco:pandorapid:Chi2ParticleID 0.0042011 0.0046538 0.00554033 0.00448758 0.000458769 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0314963 0.0403244 0.0572146 0.0400439 0.00732697 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.0032951 0.00446511 0.00560484 0.00443092 0.000588823 10
reco:pandoraShowercalo:ShowerCalorimetry 1.45337 3.22474 5.71766 3.14876 1.26041 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.4192 3.18654 5.57519 3.13327 1.22121 10
reco:emtrkmichelid:EmTrackMichelId 122.132 158.348 216.42 158.197 24.546 10
reco:crttag:SingleCRTMatchingProducer 0.00643605 0.00997944 0.0140307 0.00932521 0.002433 10
reco:crtreco:TwoCRTMatchingProducer 0.0046464 0.00716504 0.0105191 0.00689108 0.00193082 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.533102 0.708012 1.05682 0.667054 0.149616 10
reco:pandora2Track:LArPandoraTrackCreation 5.37363 10.274 19.7746 9.8906 3.9013 10
reco:pandora2calo:Calorimetry 2.30397 4.6784 7.98413 4.41235 1.70358 10
reco:pandora2calonosce:Calorimetry 2.14194 4.38805 7.51104 4.05793 1.61419 10
reco:pandora2pid:Chi2ParticleID 0.00901856 0.0114719 0.0155626 0.0110541 0.00187883 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0379255 0.0513174 0.0722503 0.0515822 0.00951526 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00893762 0.0113191 0.0151854 0.0108793 0.00183647 10
reco:pandora2Shower:LArPandoraShowerCreation 2.0735 4.0244 6.88445 3.66446 1.47613 10
reco:pandora2Showercalo:ShowerCalorimetry 1.99227 3.99322 6.44207 3.9359 1.37074 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.83782 3.71118 6.36176 3.51234 1.32057 10
[art]:TriggerResults:TriggerResultInserter 2.5171e-05 4.09373e-05 9.0601e-05 3.6525e-05 1.77567e-05 10
end_path:out1:RootOutput 7.05e-06 8.633e-06 1.144e-05 8.555e-06 1.27115e-06 10
end_path:out1:RootOutput(write) 1.89402 2.54549 3.52813 2.52879 0.431143 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5110.78 MB
Peak resident set size usage (VmHWM): 4294.28 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.06 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.66 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.10 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9959264
-rw-r--r-- 1 dunegli fnalgrid 2351837709 Jan 30 09:12 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2239686760 Jan 30 09:10 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2028569269 Jan 30 10:02 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1782189122 Jan 30 07:41 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1722275321 Jan 30 07:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 73038920 Jan 30 09:59 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 243074 Jan 30 10:02 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 30 10:02 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 30 10:02 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 30 10:02 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 09:12 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 07:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 10:02 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 30 07:28 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 30 10:02 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 30 09:16 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 30 09:12 debugprod.log
