Jobsub ID 296618.1@dunegpschedd02.fnal.gov
| Jobsub ID | 296618.1@dunegpschedd02.fnal.gov |
| Workflow ID | 12545 |
| Stage ID | 1 |
| User name | pmelas@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 8388608000 (8000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-01-30 07:26:55 |
| Site | US_FNAL-FermiGrid |
| Entry | FNAL_GPGrid_ce04_mcore_op_duneonly |
| Last heartbeat | 2026-01-30 10:03:00 |
| From worker node | Hostname | dunegli-8484075-0-fnpc19130.fnal.gov |
| cpuinfo | AMD EPYC 7502 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 10485760000 (10000 MiB) |
| Wall seconds limit | 172800 (48 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-30 07:28:39 |
| Input files | monte-carlo-012545-000001
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9218s) |
| CPU time | 2h (9149s = 99%) |
| Max RSS bytes | 4140515328 (3948 MiB) |
| Outputting started | 2026-01-30 10:02:36 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12545/1/001/reco_stage1.root
|
| Finished | 2026-01-30 10:03:00 |
| Saved logs | justin-logs:296618.1-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ToolBasedRawDigitPrepService:prepare: Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare: Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 91359 traces, 0 summary
Retagger: tagging trace set: gauss with 72178 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 72178 traces tagged "gauss"
FrameSaver: q=2.43243e+06 n=1270542 tag=gauss
wclsFrameSaver: saving 91359 traces tagged "wiener"
FrameSaver: q=2.55636e+06 n=1130032 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21343 XUs and 22273 XVs -> 219 XUVs
C:0 T:1 41334 XUs and 40142 XVs -> 2170 XUVs
C:0 T:2 63789 XUs and 66055 XVs -> 1813 XUVs
C:0 T:3 36793 XUs and 30257 XVs -> 180 XUVs
C:0 T:4 6963 XUs and 6800 XVs -> 82 XUVs
C:0 T:5 15086 XUs and 11669 XVs -> 1459 XUVs
C:0 T:6 61311 XUs and 61808 XVs -> 2503 XUVs
C:0 T:7 24516 XUs and 24020 XVs -> 201 XUVs
C:0 T:8 6265 XUs and 5653 XVs -> 35 XUVs
C:0 T:9 39157 XUs and 47484 XVs -> 2254 XUVs
C:0 T:10 53097 XUs and 24372 XVs -> 6029 XUVs
C:0 T:11 213 XUs and 140 XVs -> 16 XUVs
16961 XUVs total
7191 collection wire objects
16017 potential space points
Neighbour search...
1129435 tests to find 533678 neighbours
Iterating with no regularization...
Begin: 5.85458e+07
0 4.99343e+07
1 4.95532e+07
2 4.9508e+07
Now with regularization...
Begin: 4.35513e+07
0 4.35455e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412053817
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412053817
fLifetime: 30000 [us]
30-Jan-2026 10:02:16 UTC Closed output file "reco_stage1.root"
30-Jan-2026 10:02:16 UTC Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob: # events skipped: 0
====================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================================
Full event 176.579 276.322 442.028 273.021 74.2099 10
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000660785 0.00261736 0.00460009 0.00266633 0.00116484 10
reco:ophit:OpHitFinder 0.171151 0.237807 0.276794 0.241243 0.028604 10
reco:opflash:OpFlashFinder 0.0827027 0.141494 0.176096 0.145596 0.0262686 10
reco:opslicer:OpSlicer 0.424938 0.5817 0.727122 0.590447 0.0945751 10
reco:caldata:DataPrepModule 4.10514 4.27258 4.4802 4.27467 0.105886 10
reco:wclsdatasp:WireCellToolkit 25.6654 29.5081 33.1543 30.8349 2.76323 10
reco:gaushit:GausHitFinder 0.560377 0.887754 1.17503 0.904809 0.185877 10
reco:nhitsfilter:NumberOfHitsFilter 0.000293082 0.000491188 0.000671296 0.000470479 0.00010403 10
reco:reco3d:SpacePointSolver 4.43078 10.4491 21.2089 8.24123 5.19133 10
reco:hitpdune:DisambigFromSpacePoints 0.587363 1.11919 1.85563 1.00695 0.403221 10
reco:pandora:StandardPandora 11.9933 29.7936 97.7921 22.6859 23.9863 10
reco:pandoraWriter:StandardPandora 0.220284 0.344136 0.461469 0.353973 0.071923 10
reco:pandoraTrack:LArPandoraTrackCreation 1.1808 2.6912 3.93796 2.83451 0.890653 10
reco:pandoraShower:LArPandoraShowerCreation 1.39882 3.40246 7.82904 2.87012 1.9013 10
reco:pandoracalo:Calorimetry 0.646865 1.49896 2.56916 1.55132 0.568201 10
reco:pandoracalonosce:Calorimetry 0.561059 1.34673 2.23647 1.37457 0.518548 10
reco:pandorapid:Chi2ParticleID 0.00286111 0.00437538 0.00531805 0.00448506 0.000672504 10
reco:pandoracalinoxyzt:CalibrationdEdXPDSP 0.0276546 0.040994 0.0525835 0.0424414 0.00700374 10
reco:pandoracalipidnoxyzt:Chi2ParticleID 0.00267966 0.00418301 0.00510834 0.00437551 0.000683945 10
reco:pandoraShowercalo:ShowerCalorimetry 1.30927 3.22955 6.65846 3.02315 1.68383 10
reco:pandoraShowercalonosce:ShowerCalorimetry 1.26378 3.12539 5.97161 2.89749 1.61713 10
reco:emtrkmichelid:EmTrackMichelId 101.598 152.07 202.674 153.668 28.7241 10
reco:crttag:SingleCRTMatchingProducer 0.00615812 0.0105959 0.0143726 0.0109573 0.00213808 10
reco:crtreco:TwoCRTMatchingProducer 0.00486284 0.00678195 0.00837851 0.00704284 0.00122385 10
reco:anodepiercerst0:T0RecoAnodePiercers 0.428757 0.693517 0.891554 0.736977 0.133362 10
reco:pandora2Track:LArPandoraTrackCreation 5.1422 9.74365 15.61 10.1554 3.45037 10
reco:pandora2calo:Calorimetry 2.08632 4.14206 7.26812 4.41691 1.61607 10
reco:pandora2calonosce:Calorimetry 1.94674 3.9269 6.88959 4.16499 1.54893 10
reco:pandora2pid:Chi2ParticleID 0.00842346 0.011152 0.0145227 0.0110821 0.00214504 10
reco:pandora2calinoxyzt:CalibrationdEdXPDSP 0.0402662 0.0523843 0.0699564 0.0519467 0.00913698 10
reco:pandora2calipidnoxyzt:Chi2ParticleID 0.00836722 0.0113032 0.0147015 0.0111442 0.00240005 10
reco:pandora2Shower:LArPandoraShowerCreation 1.80706 3.58985 6.3259 3.88501 1.35942 10
reco:pandora2Showercalo:ShowerCalorimetry 1.80989 3.58072 5.97443 3.85643 1.3431 10
reco:pandora2Showercalonosce:ShowerCalorimetry 1.68269 3.2278 5.12882 3.41886 1.15582 10
[art]:TriggerResults:TriggerResultInserter 2.814e-05 4.31334e-05 9.5441e-05 3.777e-05 1.80769e-05 10
end_path:out1:RootOutput 7.5e-06 8.9901e-06 1.282e-05 8.675e-06 1.58463e-06 10
end_path:out1:RootOutput(write) 1.67842 2.4709 3.37056 2.54185 0.499625 10
====================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4958.15 MB
Peak resident set size usage (VmHWM): 4140.52 MB
Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor: Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader: 0.05 sec/event
ToolBasedRawDigitPrepService:dtor: adcSampleFiller: 0.18 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleAdcToKe: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdspTailPedRemovalZKe: 2.63 sec/event
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder: 0.10 sec/event
ToolBasedRawDigitPrepService:dtor: adcScaleKeToAdc: 0.04 sec/event
ToolBasedRawDigitPrepService:dtor: pdsp_RemoveBadChannels: 0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9658124
-rw-r--r-- 1 dunegli fnalgrid 2282041566 Jan 30 09:14 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2165744795 Jan 30 09:12 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1949135794 Jan 30 10:02 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1744247390 Jan 30 07:42 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1677950584 Jan 30 07:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 70113470 Jan 30 09:59 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 242662 Jan 30 10:02 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid 217088 Jan 30 10:02 mem.db
-rw-r--r-- 1 dunegli fnalgrid 62376 Jan 30 10:02 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid 36864 Jan 30 10:02 time.db
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 09:14 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 07:42 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid 519 Jan 30 10:02 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid 52 Jan 30 07:28 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid 7 Jan 30 10:02 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 30 09:18 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid 0 Jan 30 09:14 debugprod.log
