justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274602.0@dunegpschedd01.fnal.gov

Jobsub ID274602.0@dunegpschedd01.fnal.gov
Workflow ID11726
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 13:06:49
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-11 15:39:13
From worker nodeHostnamewn2208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 13:07:30
Input filesmonte-carlo-011726-000001
JobscriptExit code0
Real time2h (9008s)
CPU time2h (8819s = 97%)
Max RSS bytes4641509376 (4426 MiB)
Outputting started2026-01-11 15:37:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11726/1/001/reco_stage1.root
Finished2026-01-11 15:39:13
Saved logsjustin-logs:274602.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 157597 traces, 0 summary
Retagger: tagging trace set: gauss with 127149 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 127149 traces tagged "gauss"
FrameSaver: q=4.57661e+06 n=2380017 tag=gauss
wclsFrameSaver: saving 157597 traces tagged "wiener"
FrameSaver: q=4.81113e+06 n=2157348 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28204 XUs and 30756 XVs -> 299 XUVs
C:0 T:1 53373 XUs and 75548 XVs -> 5066 XUVs
C:0 T:2 139158 XUs and 158728 XVs -> 5843 XUVs
C:0 T:3 68034 XUs and 64086 XVs -> 545 XUVs
C:0 T:4 16117 XUs and 19203 XVs -> 168 XUVs
C:0 T:5 63070 XUs and 70819 XVs -> 4582 XUVs
C:0 T:6 103625 XUs and 109884 XVs -> 3896 XUVs
C:0 T:7 42231 XUs and 42715 XVs -> 321 XUVs
C:0 T:8 12512 XUs and 13470 XVs -> 163 XUVs
C:0 T:9 105099 XUs and 163117 XVs -> 17784 XUVs
C:0 T:10 55074 XUs and 44632 XVs -> 4727 XUVs
C:0 T:11 596 XUs and 467 XVs -> 74 XUVs
43468 XUVs total
13647 collection wire objects
40449 potential space points
Neighbour search...
6139347 tests to find 2492602 neighbours
Iterating with no regularization...
Begin: 1.18393e+08
0 1.00329e+08
1 9.90676e+07
2 9.89266e+07
3 9.88934e+07
Now with regularization...
Begin: 8.53943e+07
0 8.53677e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411676408
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411676408
fLifetime: 30000 [us]
11-Jan-2026 15:37:38 GMT  Closed output file "reco_stage1.root"
11-Jan-2026 15:37:38 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          214.765       265.662       361.212       257.312       42.1516        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00100477    0.00293205    0.00495096    0.00286155    0.00122794       10     
reco:ophit:OpHitFinder                             0.211159      0.246055      0.293258      0.240104      0.0249162       10     
reco:opflash:OpFlashFinder                         0.142703      0.190009      0.249285      0.186727      0.030643        10     
reco:opslicer:OpSlicer                             0.432853      0.541016      0.664996      0.515311      0.0832531       10     
reco:caldata:DataPrepModule                         3.52931       3.61789       3.90392       3.57952      0.101701        10     
reco:wclsdatasp:WireCellToolkit                     23.4254       24.283        25.1524       24.3207      0.414448        10     
reco:gaushit:GausHitFinder                         0.767531      0.953681       1.15895      0.957993      0.142591        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000354175   0.000431242   0.000732977   0.000382999   0.000114128      10     
reco:reco3d:SpacePointSolver                        7.58022       14.439        27.5356       11.7212       6.63638        10     
reco:hitpdune:DisambigFromSpacePoints               1.14127       1.50016       2.25401       1.4266       0.339635        10     
reco:pandora:StandardPandora                        16.8056       28.6709       52.719        26.4896       10.2306        10     
reco:pandoraWriter:StandardPandora                 0.263421       0.30901      0.406555      0.295313      0.0417328       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.29083       2.9101        4.06008       2.85233      0.510999        10     
reco:pandoraShower:LArPandoraShowerCreation         2.25913       3.25291       5.27964       3.00534      0.859123        10     
reco:pandoracalo:Calorimetry                        1.41249       1.74327       2.30423       1.69298      0.270595        10     
reco:pandoracalonosce:Calorimetry                   1.30839       1.61377       2.1478        1.56891      0.251977        10     
reco:pandorapid:Chi2ParticleID                    0.00356346    0.00434366    0.00669533    0.00430087    0.000881829      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0330075     0.0396359     0.0487913     0.0381309    0.00466828       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00336416    0.00383956    0.00453218     0.0039198    0.000358159      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.64371       3.71775       5.72844       3.49292      0.908212        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.61901       3.65881       5.58941       3.44291      0.878973        10     
reco:emtrkmichelid:EmTrackMichelId                  117.777       138.578       177.389       135.617       17.2251        10     
reco:crttag:SingleCRTMatchingProducer             0.00654969    0.00967996     0.0147285    0.00895144    0.00270701       10     
reco:crtreco:TwoCRTMatchingProducer                0.0050736    0.00599087     0.0073143    0.00587487    0.000675985      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.628245      0.747284      0.924951      0.728748      0.0824136       10     
reco:pandora2Track:LArPandoraTrackCreation          6.86605       9.65674       15.1774       8.8697        2.36104        10     
reco:pandora2calo:Calorimetry                       3.88538       5.22293       7.65664       4.89453       1.10913        10     
reco:pandora2calonosce:Calorimetry                  3.69555       5.00647       7.32863       4.68781       1.05884        10     
reco:pandora2pid:Chi2ParticleID                   0.00807498    0.00996379     0.0137908    0.00945836    0.00170145       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.038887      0.0475916     0.0610019     0.0467677    0.00589592       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00773459    0.00940536     0.0115463    0.00907479     0.0011847       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.54346       3.48934       5.28391       3.28079      0.752352        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.42337       4.57879       6.63235       4.30827       0.91339        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.14409       4.22921       6.20532       4.04149      0.853298        10     
[art]:TriggerResults:TriggerResultInserter        2.8714e-05    4.96443e-05   0.00018129    3.26465e-05   4.43089e-05      10     
end_path:out1:RootOutput                           8.696e-06    1.21216e-05    2.152e-05    1.15265e-05   3.34487e-06      10     
end_path:out1:RootOutput(write)                     1.89607       2.31171       3.08763       2.24197      0.342918        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5456.7 MB
  Peak resident set size usage (VmHWM): 4641.51 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.29 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11540580
-rw-r--r-- 1 dune01 dune01 2656713611 Jan 11 14:52 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2516095228 Jan 11 14:50 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2411374266 Jan 11 15:37 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2102968462 Jan 11 13:21 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2038873232 Jan 11 13:12 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   90895550 Jan 11 15:34 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242685 Jan 11 15:37 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 11 15:37 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 11 15:37 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 11 15:37 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 11 14:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 13:21 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 15:37 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 13:07 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 11 15:37 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 11 14:56 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 11 14:52 debugprod.log
justIN time: 2026-02-06 10:21:20 UTC       justIN version: 01.06.00