justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265102.0@dunegpschedd02.fnal.gov

Jobsub ID265102.0@dunegpschedd02.fnal.gov
Workflow ID11417
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 14:14:44
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2025-12-30 18:38:26
From worker nodeHostnameccwcondor0564
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 14:15:30
Input filesmonte-carlo-011417-000001
JobscriptExit code0
Real time4h (15672s)
CPU time4h (15034s = 95%)
Max RSS bytes4252905472 (4055 MiB)
Outputting started2025-12-30 18:36:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11417/1/001/reco_stage1.root
Finished2025-12-30 18:38:26
Saved logsjustin-logs:265102.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 156631 traces, 0 summary
Retagger: tagging trace set: gauss with 121723 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 121723 traces tagged "gauss"
FrameSaver: q=4.03364e+06 n=2187002 tag=gauss
wclsFrameSaver: saving 156631 traces tagged "wiener"
FrameSaver: q=4.24967e+06 n=1948215 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22767 XUs and 34773 XVs -> 334 XUVs
C:0 T:1 52341 XUs and 69165 XVs -> 4331 XUVs
C:0 T:2 77212 XUs and 149655 XVs -> 14216 XUVs
C:0 T:3 33684 XUs and 44300 XVs -> 489 XUVs
C:0 T:4 17958 XUs and 21970 XVs -> 184 XUVs
C:0 T:5 75967 XUs and 85838 XVs -> 7208 XUVs
C:0 T:6 154171 XUs and 223671 XVs -> 19606 XUVs
C:0 T:7 53553 XUs and 58793 XVs -> 806 XUVs
C:0 T:8 6794 XUs and 9600 XVs -> 94 XUVs
C:0 T:9 44797 XUs and 81467 XVs -> 5231 XUVs
C:0 T:10 45367 XUs and 42015 XVs -> 5920 XUVs
C:0 T:11 102 XUs and 83 XVs -> 10 XUVs
58429 XUVs total
14713 collection wire objects
53602 potential space points
Neighbour search...
9707156 tests to find 4185462 neighbours
Iterating with no regularization...
Begin: 8.896e+07
0 7.81244e+07
1 7.74049e+07
2 7.73084e+07
3 7.72789e+07
Now with regularization...
Begin: 6.67755e+07
0 6.67248e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411435889
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411435889
fLifetime: 30000 [us]
30-Dec-2025 19:36:40 CET  Closed output file "reco_stage1.root"
30-Dec-2025 19:36:40 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          435.624       562.574       751.459       552.107       85.4832        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0020881    0.00740597     0.0158885    0.00657341    0.00377233       10     
reco:ophit:OpHitFinder                              0.34106      0.374835      0.426037      0.378969      0.0273788       10     
reco:opflash:OpFlashFinder                         0.177145      0.213384      0.257042      0.211337      0.0272648       10     
reco:opslicer:OpSlicer                             0.714268       0.87218       1.03833      0.873996      0.0879107       10     
reco:caldata:DataPrepModule                         6.86512       7.3264        7.95037       7.29023      0.364245        10     
reco:wclsdatasp:WireCellToolkit                     43.5716       47.3585       54.7542       45.8799       3.78604        10     
reco:gaushit:GausHitFinder                          1.49684       1.80319       2.05087       1.83093       0.18622        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000614959   0.000803216   0.00120827    0.000780699   0.000156709      10     
reco:reco3d:SpacePointSolver                        17.0847       28.2166       47.7536       26.5227       8.22338        10     
reco:hitpdune:DisambigFromSpacePoints               1.86394       2.40516       3.02413       2.32928      0.397141        10     
reco:pandora:StandardPandora                        34.1476       60.0006       106.16        57.386        18.8086        10     
reco:pandoraWriter:StandardPandora                 0.472568      0.632972      0.775032      0.644207      0.0834662       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.28585       8.26019       12.1562       8.19079       1.85704        10     
reco:pandoraShower:LArPandoraShowerCreation         7.55382       11.7218       16.6163       11.2721       2.85602        10     
reco:pandoracalo:Calorimetry                        3.28145       5.04101       6.91681       4.96862       1.01185        10     
reco:pandoracalonosce:Calorimetry                   2.9674        4.85871       6.55982       5.23152      0.983739        10     
reco:pandorapid:Chi2ParticleID                    0.00670193    0.00805922     0.0110835    0.00763563    0.00129849       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0689746     0.0858563     0.115798      0.0852145     0.0125368       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00679103    0.00775122    0.00953245    0.00716634    0.000923641      10     
reco:pandoraShowercalo:ShowerCalorimetry            6.86431       13.3029       19.1405       13.3317       3.33719        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.34533       12.7924       19.3961       12.7357       3.17991        10     
reco:emtrkmichelid:EmTrackMichelId                  214.719       253.165       311.634       253.533       27.3518        10     
reco:crttag:SingleCRTMatchingProducer              0.0231746     0.0316795     0.064582      0.0277038     0.0114068       10     
reco:crtreco:TwoCRTMatchingProducer                0.017975      0.0225269     0.0390803     0.020375     0.00604729       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.899942       1.10847       1.34185       1.07192      0.127234        10     
reco:pandora2Track:LArPandoraTrackCreation          19.8436       28.8352       41.1815       28.4907       6.16681        10     
reco:pandora2calo:Calorimetry                       11.5596       16.5664       23.3927       15.9712       3.73494        10     
reco:pandora2calonosce:Calorimetry                  11.1392       16.0767       22.6945       15.9528       3.68747        10     
reco:pandora2pid:Chi2ParticleID                    0.0165722     0.0200147     0.0239097     0.0198189    0.00248151       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0819624     0.110652      0.136677      0.111492      0.013068        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0150643     0.0192509     0.0232274     0.0184557    0.00238847       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.80013       10.3931       14.6635       10.494        2.21584        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.33723       14.3421       21.7379       14.3431       3.52114        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.2162        12.6527       20.0587       12.6976       3.32474        10     
[art]:TriggerResults:TriggerResultInserter         8.569e-05    0.000127525   0.000251319   0.000111107   4.76406e-05      10     
end_path:out1:RootOutput                          1.3115e-05    1.69318e-05   2.5637e-05    1.53485e-05   4.13719e-06      10     
end_path:out1:RootOutput(write)                     3.13027       3.68487       4.60521       3.71915      0.466311        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5078.65 MB
  Peak resident set size usage (VmHWM): 4252.91 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.55 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10507916
-rw-r--r-- 1 dune001 lbno 2457238492 Dec 30 18:00 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2331124613 Dec 30 17:57 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2171261048 Dec 30 19:36 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1904937820 Dec 30 15:39 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1813722645 Dec 30 15:23 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   81205870 Dec 30 19:28 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243303 Dec 30 19:36 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Dec 30 19:36 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Dec 30 19:36 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Dec 30 19:36 time.db
-rw-r--r-- 1 dune001 lbno        519 Dec 30 18:00 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 30 15:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Dec 30 19:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Dec 30 15:15 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Dec 30 19:36 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Dec 30 18:11 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Dec 30 18:00 debugprod.log
justIN time: 2026-02-07 00:50:28 UTC       justIN version: 01.06.00