justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279033.1@dunegpschedd01.fnal.gov

Jobsub ID279033.1@dunegpschedd01.fnal.gov
Workflow ID11957
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-15 10:55:47
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-01-15 15:52:38
From worker nodeHostnamecn514.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-15 10:56:45
Input filesmonte-carlo-011957-000001
JobscriptExit code0
Real time4h (17539s)
CPU time4h (17309s = 98%)
Max RSS bytes4275290112 (4077 MiB)
Outputting started2026-01-15 15:49:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11957/1/001/reco_stage1.root
Finished2026-01-15 15:52:38
Saved logsjustin-logs:279033.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

caleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 133291 traces, 0 summary
Retagger: tagging trace set: gauss with 99398 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 99398 traces tagged "gauss"
FrameSaver: q=3.48109e+06 n=1718251 tag=gauss
wclsFrameSaver: saving 133291 traces tagged "wiener"
FrameSaver: q=3.69556e+06 n=1512721 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37358 XUs and 40053 XVs -> 551 XUVs
C:0 T:1 79114 XUs and 87470 XVs -> 9607 XUVs
C:0 T:2 78358 XUs and 159071 XVs -> 9231 XUVs
C:0 T:3 32956 XUs and 33888 XVs -> 240 XUVs
C:0 T:4 8316 XUs and 9125 XVs -> 277 XUVs
C:0 T:5 25174 XUs and 22748 XVs -> 3350 XUVs
C:0 T:6 117949 XUs and 120097 XVs -> 10408 XUVs
C:0 T:7 43994 XUs and 37857 XVs -> 479 XUVs
C:0 T:8 9071 XUs and 6990 XVs -> 136 XUVs
C:0 T:9 50491 XUs and 52002 XVs -> 3928 XUVs
C:0 T:10 46560 XUs and 33413 XVs -> 5439 XUVs
C:0 T:11 279 XUs and 277 XVs -> 31 XUVs
43677 XUVs total
11227 collection wire objects
39246 potential space points
Neighbour search...
4641420 tests to find 2251050 neighbours
Iterating with no regularization...
Begin: 6.23551e+07
0 5.10988e+07
1 5.03128e+07
2 5.01907e+07
3 5.01492e+07
Now with regularization...
Begin: 4.01585e+07
0 4.01122e+07
1 4.00795e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411755002
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411755002
fLifetime: 30000 [us]
15-Jan-2026 15:49:01 GMT  Closed output file "reco_stage1.root"
15-Jan-2026 15:49:01 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          548.756       698.481       850.508       711.546       114.043        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00104526    0.00511753    0.00977046    0.00472422    0.00265829       10     
reco:ophit:OpHitFinder                             0.326603      0.456308      0.628752      0.436958      0.0978924       10     
reco:opflash:OpFlashFinder                         0.243779      0.341378      0.494778       0.33361      0.0690468       10     
reco:opslicer:OpSlicer                             0.679565       1.06894       1.59316       1.01107      0.276136        10     
reco:caldata:DataPrepModule                         7.02638       9.12448       11.331        9.15604       1.97358        10     
reco:wclsdatasp:WireCellToolkit                     54.7412       67.7151       86.3501       64.3617       11.7469        10     
reco:gaushit:GausHitFinder                          1.61141       2.08841       2.88113       1.89858      0.485191        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000685327   0.000962587   0.00163766    0.000854702   0.000271694      10     
reco:reco3d:SpacePointSolver                        11.2797       21.6462       40.519        19.1316       8.11127        10     
reco:hitpdune:DisambigFromSpacePoints               1.97503       3.19141       4.23616       3.03806      0.719163        10     
reco:pandora:StandardPandora                        31.9739       50.8148       64.5332       51.6392       9.06004        10     
reco:pandoraWriter:StandardPandora                  0.49921      0.699482      0.939217      0.646021      0.139822        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.49203       11.8503       15.8545       11.7815       2.63006        10     
reco:pandoraShower:LArPandoraShowerCreation         7.39645       12.8058       18.0648       13.5345       3.12069        10     
reco:pandoracalo:Calorimetry                        2.91875       5.47163       7.4842        5.3394        1.31357        10     
reco:pandoracalonosce:Calorimetry                   2.76616       5.16027       6.87967       5.23845       1.18241        10     
reco:pandorapid:Chi2ParticleID                    0.00612428    0.00928461     0.0125933    0.00908027    0.00220075       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0639118     0.0940097     0.120745      0.0869036     0.0176389       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00607601    0.00865774     0.0121273    0.00762474    0.00206325       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.67871       12.1808       17.457        12.7057       3.39497        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.39293       11.867        18.1569       12.0396       3.52067        10     
reco:emtrkmichelid:EmTrackMichelId                  247.638       351.043       446.89        363.848       69.1638        10     
reco:crttag:SingleCRTMatchingProducer              0.0250868     0.0365108     0.0507123     0.0380556    0.00729259       10     
reco:crtreco:TwoCRTMatchingProducer                0.0206421     0.0296826     0.0371778     0.0297618    0.00546945       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.29824       2.0305        2.90113       1.98014      0.457633        10     
reco:pandora2Track:LArPandoraTrackCreation          20.7786       42.6832       57.8023       43.1368       10.6738        10     
reco:pandora2calo:Calorimetry                       7.46324       17.8978       25.6165       18.6796       5.28446        10     
reco:pandora2calonosce:Calorimetry                  7.6632        17.6075       24.8833       17.8238       5.08255        10     
reco:pandora2pid:Chi2ParticleID                    0.0147806     0.0251911     0.033657      0.0268643    0.00724521       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0921967     0.131659      0.165764      0.136948      0.0285467       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0144309     0.0248767     0.0333354     0.0264759    0.00704511       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.91981       13.9177       22.2984       14.4128       4.68977        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.04936       16.2113       24.6705       17.4194       5.39514        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.2232        15.248        22.5386       16.2948       5.50103        10     
[art]:TriggerResults:TriggerResultInserter        5.9649e-05    9.53278e-05   0.000125559   9.73995e-05   2.23367e-05      10     
end_path:out1:RootOutput                           9.686e-06    1.44859e-05   1.9534e-05    1.44685e-05   2.93612e-06      10     
end_path:out1:RootOutput(write)                     2.85187       4.82313       6.27506       5.11411       1.12688        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5085.93 MB
  Peak resident set size usage (VmHWM): 4275.29 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.74 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10468784
-rw-r--r-- 1 pildune22 pildune 2459318774 Jan 15 13:49 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2332930586 Jan 15 13:47 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2147469068 Jan 15 15:49 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1894147950 Jan 15 11:29 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1810232103 Jan 15 11:05 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   75319110 Jan 15 15:38 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243160 Jan 15 15:49 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Jan 15 15:49 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Jan 15 15:49 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Jan 15 15:49 time.db
-rw-r--r-- 1 pildune22 pildune        519 Jan 15 13:49 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 15 11:29 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 15 15:49 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Jan 15 10:56 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Jan 15 15:49 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Jan 15 13:59 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Jan 15 13:49 debugprod.log
justIN time: 2026-02-06 00:48:54 UTC       justIN version: 01.06.00