justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302153.0@dunegpschedd01.fnal.gov

Jobsub ID302153.0@dunegpschedd01.fnal.gov
Workflow ID12530
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-29 16:40:09
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-29 21:00:18
From worker nodeHostnameheplnc158.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 16:41:06
Input filesmonte-carlo-012530-000001
JobscriptExit code0
Real time4h (15245s)
CPU time3h (14284s = 93%)
Max RSS bytes3946967040 (3764 MiB)
Outputting started2026-01-29 20:55:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12530/1/001/reco_stage1.root
Finished2026-01-29 21:00:18
Saved logsjustin-logs:302153.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123963 traces, 0 summary
Retagger: tagging trace set: gauss with 93816 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93816 traces tagged "gauss"
FrameSaver: q=3.1874e+06 n=1672626 tag=gauss
wclsFrameSaver: saving 123963 traces tagged "wiener"
FrameSaver: q=3.35856e+06 n=1475027 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29845 XUs and 16487 XVs -> 207 XUVs
C:0 T:1 57645 XUs and 37113 XVs -> 5422 XUVs
C:0 T:2 61195 XUs and 86367 XVs -> 3529 XUVs
C:0 T:3 32258 XUs and 37812 XVs -> 198 XUVs
C:0 T:4 11375 XUs and 8273 XVs -> 66 XUVs
C:0 T:5 87443 XUs and 35224 XVs -> 14144 XUVs
C:0 T:6 74177 XUs and 70728 XVs -> 3282 XUVs
C:0 T:7 29396 XUs and 28186 XVs -> 164 XUVs
C:0 T:8 9350 XUs and 10380 XVs -> 146 XUVs
C:0 T:9 111694 XUs and 105242 XVs -> 17641 XUVs
C:0 T:10 60069 XUs and 34459 XVs -> 4583 XUVs
C:0 T:11 393 XUs and 257 XVs -> 43 XUVs
49425 XUVs total
11245 collection wire objects
46911 potential space points
Neighbour search...
8584137 tests to find 3860038 neighbours
Iterating with no regularization...
Begin: 6.50541e+07
0 5.57962e+07
1 5.51974e+07
2 5.50981e+07
3 5.50659e+07
Now with regularization...
Begin: 4.66305e+07
0 4.65976e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412041439
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412041439
fLifetime: 30000 [us]
29-Jan-2026 20:55:07 GMT  Closed output file "reco_stage1.root"
29-Jan-2026 20:55:07 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          327.995       475.534       606.269       466.419       99.8721        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00172128    0.00438562    0.00746395    0.00442071    0.00197873       10     
reco:ophit:OpHitFinder                              0.34255      0.404834      0.460993      0.397666      0.0423405       10     
reco:opflash:OpFlashFinder                         0.147267      0.203055      0.246502      0.205227      0.0301993       10     
reco:opslicer:OpSlicer                             0.803773      0.957719       1.16438      0.919271      0.120366        10     
reco:caldata:DataPrepModule                         7.15605       7.78442       8.50099       7.82886       0.3862         10     
reco:wclsdatasp:WireCellToolkit                     41.2828       44.2647       46.324        44.862        1.52709        10     
reco:gaushit:GausHitFinder                          1.29746       1.84904       2.27531       1.88161       0.31896        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000670161    0.0010288    0.00303681    0.000846924   0.000675654      10     
reco:reco3d:SpacePointSolver                        11.7654       23.7428       38.8899       23.5824       7.99116        10     
reco:hitpdune:DisambigFromSpacePoints               1.41721       2.5072        3.23367       2.53053      0.626408        10     
reco:pandora:StandardPandora                        23.2304       46.8735       73.2638       39.9131       17.155         10     
reco:pandoraWriter:StandardPandora                 0.562847      0.687251      0.786638      0.710084      0.0791113       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.52156       5.37508       7.73044       5.41403       1.40765        10     
reco:pandoraShower:LArPandoraShowerCreation         2.52332       6.63861       10.399        6.80082       2.90017        10     
reco:pandoracalo:Calorimetry                        2.06058       3.2756        4.44013       3.45625      0.946143        10     
reco:pandoracalonosce:Calorimetry                   1.84041       3.03221       4.15828       3.27742      0.913589        10     
reco:pandorapid:Chi2ParticleID                    0.00695631    0.00854931     0.0109115    0.00841843    0.00113912       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0729094     0.0923848     0.124255      0.0925101     0.0153482       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00723593     0.008108     0.00969913    0.00781912    0.000866574      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.58891       7.09006       11.7463       6.48281       3.26926        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.37006       6.94308       11.4955       6.54148       3.21798        10     
reco:emtrkmichelid:EmTrackMichelId                  194.876       248.97        296.128       246.467       38.7892        10     
reco:crttag:SingleCRTMatchingProducer              0.0149015     0.0329686     0.0725303     0.0280904     0.017179        10     
reco:crtreco:TwoCRTMatchingProducer               0.00925437     0.0161674     0.033588      0.015536     0.00653157       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.845083       1.13013       1.63748       1.14627      0.231927        10     
reco:pandora2Track:LArPandoraTrackCreation          7.88722       18.1878       27.8234       17.3751       6.87677        10     
reco:pandora2calo:Calorimetry                       3.98154       9.35264       14.9628       9.27456       3.64419        10     
reco:pandora2calonosce:Calorimetry                  3.65516       9.12252       14.2925       8.31965       3.80663        10     
reco:pandora2pid:Chi2ParticleID                    0.0149281     0.0206597     0.0295157     0.0202793    0.00417787       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.074039      0.114704       0.14624      0.115654      0.025196        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0122334     0.0197507     0.027543      0.0198321    0.00425755       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.85046       6.79886       10.8442       6.81141       2.7015         10     
reco:pandora2Showercalo:ShowerCalorimetry           3.26199       8.37583       11.9289       8.99355       3.26481        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.11695       7.28219       10.3519       7.27493       2.61821        10     
[art]:TriggerResults:TriggerResultInserter        7.5593e-05    0.000104903   0.000193705   8.4299e-05    3.79392e-05      10     
end_path:out1:RootOutput                          1.2454e-05    0.00101073    0.00994788    1.6366e-05    0.00297905       10     
end_path:out1:RootOutput(write)                     3.12454       4.18395       5.56089       4.21909      0.734567        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4979.75 MB
  Peak resident set size usage (VmHWM): 3946.97 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.88 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9745648
-rw-r--r-- 1 pltdune04 pltdune 2300559923 Jan 29 19:33 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2183100017 Jan 29 19:30 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1995394052 Jan 29 20:55 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1742358573 Jan 29 17:07 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1681116285 Jan 29 16:52 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   76398980 Jan 29 20:50 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243199 Jan 29 20:55 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 29 20:55 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 29 20:55 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 29 20:55 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 29 19:33 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 29 17:07 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 29 20:55 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 29 16:41 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 29 20:55 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 29 19:39 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 29 19:33 debugprod.log
justIN time: 2026-02-04 15:18:03 UTC       justIN version: 01.06.00