justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271061.1@dunegpschedd02.fnal.gov

Jobsub ID271061.1@dunegpschedd02.fnal.gov
Workflow ID11789
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-13 06:13:01
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-13 11:24:02
From worker nodeHostnamecn527.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-13 06:13:44
Input filesmonte-carlo-011789-000001
JobscriptExit code0
Real time5h (18370s)
CPU time5h (18066s = 98%)
Max RSS bytes4463038464 (4256 MiB)
Outputting started2026-01-13 11:19:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11789/1/001/reco_stage1.root
Finished2026-01-13 11:23:55
Saved logsjustin-logs:271061.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

p_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 163590 traces, 0 summary
Retagger: tagging trace set: gauss with 125613 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125613 traces tagged "gauss"
FrameSaver: q=4.30231e+06 n=2250763 tag=gauss
wclsFrameSaver: saving 163590 traces tagged "wiener"
FrameSaver: q=4.53385e+06 n=1995399 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28795 XUs and 33189 XVs -> 276 XUVs
C:0 T:1 77216 XUs and 73374 XVs -> 5352 XUVs
C:0 T:2 98390 XUs and 141665 XVs -> 6612 XUVs
C:0 T:3 47396 XUs and 61360 XVs -> 266 XUVs
C:0 T:4 14738 XUs and 21675 XVs -> 196 XUVs
C:0 T:5 64946 XUs and 68809 XVs -> 7456 XUVs
C:0 T:6 110019 XUs and 114193 XVs -> 3455 XUVs
C:0 T:7 46701 XUs and 46834 XVs -> 538 XUVs
C:0 T:8 14291 XUs and 13178 XVs -> 131 XUVs
C:0 T:9 100390 XUs and 116612 XVs -> 6610 XUVs
C:0 T:10 32504 XUs and 54846 XVs -> 7094 XUVs
C:0 T:11 47 XUs and 37 XVs -> 5 XUVs
37991 XUVs total
14935 collection wire objects
35108 potential space points
Neighbour search...
1945238 tests to find 1047802 neighbours
Iterating with no regularization...
Begin: 8.26517e+07
0 6.9347e+07
1 6.83801e+07
2 6.82509e+07
3 6.82143e+07
Now with regularization...
Begin: 5.59911e+07
0 5.59683e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411710799
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411710799
fLifetime: 30000 [us]
13-Jan-2026 11:19:50 GMT  Closed output file "reco_stage1.root"
13-Jan-2026 11:19:50 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          654.998       878.693       1174.65       879.113       146.421        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00151113    0.00586489     0.0141149    0.00600726    0.00333337       10     
reco:ophit:OpHitFinder                              0.51249      0.652751      0.828248      0.658387      0.105996        10     
reco:opflash:OpFlashFinder                         0.320605      0.416359      0.550888      0.395048      0.0706945       10     
reco:opslicer:OpSlicer                             0.862767       1.38556       1.76107       1.43807      0.263197        10     
reco:caldata:DataPrepModule                         7.55681       10.9594       12.3519       11.5506       1.53004        10     
reco:wclsdatasp:WireCellToolkit                     61.6595       76.5596       90.4423       79.0352       9.80014        10     
reco:gaushit:GausHitFinder                          1.94466       2.9309        3.49968       3.14955      0.512665        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000802698   0.00120605    0.00173502    0.00110437    0.000305021      10     
reco:reco3d:SpacePointSolver                        18.5619       33.9347       45.7447       37.3857       8.38638        10     
reco:hitpdune:DisambigFromSpacePoints               3.46977       6.22464       8.83259       6.33312       1.68767        10     
reco:pandora:StandardPandora                        67.5044       88.6379       134.057       76.2931       23.3934        10     
reco:pandoraWriter:StandardPandora                 0.727525       0.91057       1.2179       0.860148      0.156906        10     
reco:pandoraTrack:LArPandoraTrackCreation           14.1333       17.915        22.655        18.1622       2.91837        10     
reco:pandoraShower:LArPandoraShowerCreation         15.2658       20.1701       29.8017       18.8752       4.02383        10     
reco:pandoracalo:Calorimetry                        6.76323       8.35961       9.97914       8.46251      0.980731        10     
reco:pandoracalonosce:Calorimetry                   6.43632       8.06424       9.64363       8.2536       0.953796        10     
reco:pandorapid:Chi2ParticleID                    0.00853571     0.0121027     0.0153459     0.0129307    0.00230827       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0946361     0.122963      0.148834      0.123776      0.0191792       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00829728     0.0115679     0.0141948     0.0123028    0.00214858       10     
reco:pandoraShowercalo:ShowerCalorimetry            14.2002       19.8108       32.4664       17.9801       5.15224        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       10.5666       18.9041       30.4406       17.2267       5.47082        10     
reco:emtrkmichelid:EmTrackMichelId                  292.212       389.092       497.811       377.229       61.7341        10     
reco:crttag:SingleCRTMatchingProducer              0.0323642     0.0447587     0.0596707     0.0443054    0.00939741       10     
reco:crtreco:TwoCRTMatchingProducer                0.0263411     0.0342292     0.0455748     0.0334562    0.00641904       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.82493       2.32323       3.23889       2.23722      0.420555        10     
reco:pandora2Track:LArPandoraTrackCreation          38.6944       59.5985       86.0052       61.1765       14.7176        10     
reco:pandora2calo:Calorimetry                       15.407        25.5139       39.2945       24.6665        7.084         10     
reco:pandora2calonosce:Calorimetry                  13.5681       23.9631       37.0517       21.759        7.22068        10     
reco:pandora2pid:Chi2ParticleID                    0.0180935     0.0262582     0.0401396     0.0228764     0.007206        10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.101897      0.142019      0.202321      0.131217      0.0317919       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0173741     0.0272555     0.0401181     0.0247572    0.00718577       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.65096       17.1784       26.3397       16.6413       5.67091        10     
reco:pandora2Showercalo:ShowerCalorimetry           10.7256       19.9159       30.3914       18.2406       6.41746        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.27802       19.3722       26.8541       19.6499       6.44762        10     
[art]:TriggerResults:TriggerResultInserter        5.4113e-05    8.74262e-05   0.000167863   8.54105e-05   3.12245e-05      10     
end_path:out1:RootOutput                           8.083e-06    1.08659e-05   1.7167e-05    1.0618e-05    2.56744e-06      10     
end_path:out1:RootOutput(write)                     3.41153       5.26794       6.99613       5.13211       1.24237        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5270.51 MB
  Peak resident set size usage (VmHWM): 4463.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.72 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11338368
-rw-r--r-- 1 pildune32 pildune 2610358251 Jan 13 08:49 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2468748460 Jan 13 08:46 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2350118727 Jan 13 11:19 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2087685569 Jan 13 06:40 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 2005186305 Jan 13 06:20 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   87775180 Jan 13 11:08 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243102 Jan 13 11:19 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 13 11:19 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 13 11:19 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 13 11:19 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 13 08:49 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 13 06:40 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 13 11:19 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 13 06:13 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 13 11:19 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 13 09:00 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 13 08:49 debugprod.log
justIN time: 2026-02-06 07:39:25 UTC       justIN version: 01.06.00