justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274093.0@dunegpschedd01.fnal.gov

Jobsub ID274093.0@dunegpschedd01.fnal.gov
Workflow ID11710
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 05:48:26
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-11 08:10:24
From worker nodeHostnamewn2208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 05:49:33
Input filesmonte-carlo-011710-000001
JobscriptExit code0
Real time2h (8367s)
CPU time2h (8154s = 97%)
Max RSS bytes4500467712 (4291 MiB)
Outputting started2026-01-11 08:09:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11710/1/001/reco_stage1.root
Finished2026-01-11 08:10:24
Saved logsjustin-logs:274093.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 121395 traces, 0 summary
Retagger: tagging trace set: gauss with 91636 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91636 traces tagged "gauss"
FrameSaver: q=3.17903e+06 n=1587649 tag=gauss
wclsFrameSaver: saving 121395 traces tagged "wiener"
FrameSaver: q=3.36268e+06 n=1387777 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39452 XUs and 31497 XVs -> 292 XUVs
C:0 T:1 73407 XUs and 62424 XVs -> 4104 XUVs
C:0 T:2 111639 XUs and 115653 XVs -> 7473 XUVs
C:0 T:3 45235 XUs and 42399 XVs -> 388 XUVs
C:0 T:4 12345 XUs and 10068 XVs -> 96 XUVs
C:0 T:5 45871 XUs and 28269 XVs -> 5218 XUVs
C:0 T:6 89314 XUs and 82655 XVs -> 5610 XUVs
C:0 T:7 36896 XUs and 33750 XVs -> 434 XUVs
C:0 T:8 5532 XUs and 4468 XVs -> 50 XUVs
C:0 T:9 32643 XUs and 37399 XVs -> 2466 XUVs
C:0 T:10 15625 XUs and 10356 XVs -> 1995 XUVs
C:0 T:11 85 XUs and 66 XVs -> 9 XUVs
28135 XUVs total
10930 collection wire objects
25416 potential space points
Neighbour search...
1265226 tests to find 684764 neighbours
Iterating with no regularization...
Begin: 5.06566e+07
0 4.09475e+07
1 4.02427e+07
2 4.01354e+07
3 4.01064e+07
Now with regularization...
Begin: 3.08965e+07
0 3.08679e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411670239
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411670239
fLifetime: 30000 [us]
11-Jan-2026 08:09:00 GMT  Closed output file "reco_stage1.root"
11-Jan-2026 08:09:00 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          182.661       224.917       274.251       217.494       28.9921        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000637829   0.00294757    0.00486176    0.00315442     0.0013442       10     
reco:ophit:OpHitFinder                              0.20578      0.227916      0.251539      0.229379      0.014987        10     
reco:opflash:OpFlashFinder                          0.13428      0.163845      0.199979      0.159759      0.0197633       10     
reco:opslicer:OpSlicer                             0.414465      0.480341      0.586875      0.461403      0.0620282       10     
reco:caldata:DataPrepModule                         3.55938       3.67494       3.97139       3.64415       0.10887        10     
reco:wclsdatasp:WireCellToolkit                     23.8637       24.6693       25.7262       24.5095      0.631736        10     
reco:gaushit:GausHitFinder                         0.639374      0.754892       1.02299      0.728757      0.108128        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000258004   0.000335142   0.000488358   0.000313318   6.23229e-05      10     
reco:reco3d:SpacePointSolver                        7.10382       11.9569       29.5639       9.97364        6.277         10     
reco:hitpdune:DisambigFromSpacePoints               0.80373       1.11534       1.55227       1.06479      0.207308        10     
reco:pandora:StandardPandora                        11.5804       20.3892       32.3115       19.3797       6.0563         10     
reco:pandoraWriter:StandardPandora                 0.205669      0.266235       0.32069      0.258883      0.0351225       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.58294       2.32785       2.7237        2.39789      0.325457        10     
reco:pandoraShower:LArPandoraShowerCreation         1.87292       2.55762       3.65845       2.61442      0.502651        10     
reco:pandoracalo:Calorimetry                        0.93473       1.41316       1.59971       1.48165      0.191953        10     
reco:pandoracalonosce:Calorimetry                  0.845383       1.31095       1.51847       1.37514       0.18817        10     
reco:pandorapid:Chi2ParticleID                     0.0028928     0.0036416    0.00482307    0.00360039    0.000480824      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0299634     0.0344988     0.0392535     0.0343001    0.00291852       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00271709     0.0033944     0.0038187     0.0034173    0.000318475      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.87396       2.92201       4.26377       2.98666      0.637179        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.82256       2.85945       4.19001       2.92908       0.62938        10     
reco:emtrkmichelid:EmTrackMichelId                  101.226        119.5        144.152       115.777       12.7622        10     
reco:crttag:SingleCRTMatchingProducer             0.00631327    0.00826625     0.012055     0.00799686    0.00159191       10     
reco:crtreco:TwoCRTMatchingProducer                0.0042807    0.00534168    0.00630538    0.00545071    0.000582257      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.551498      0.692363      0.820404      0.691907      0.0834905       10     
reco:pandora2Track:LArPandoraTrackCreation          5.10638       7.45231       10.0815       7.71024       1.33566        10     
reco:pandora2calo:Calorimetry                       2.6511        4.14182       5.52093       4.29382      0.749115        10     
reco:pandora2calonosce:Calorimetry                  2.52356       3.95373       5.28281       4.09307      0.717269        10     
reco:pandora2pid:Chi2ParticleID                   0.00700526     0.0084869     0.0101319    0.00877927    0.000908862      10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0355416     0.0444472     0.0680358     0.0412436     0.0088962       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00679045    0.00823758    0.00997445    0.00853033    0.000861561      10     
reco:pandora2Shower:LArPandoraShowerCreation        2.05975        2.812        3.69229       2.8314       0.455986        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.34858       3.68152       4.92731       3.75717      0.670905        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.20415       3.41935       4.57712       3.43778      0.627148        10     
[art]:TriggerResults:TriggerResultInserter        2.9596e-05    3.98969e-05   8.8125e-05    3.28965e-05   1.70417e-05      10     
end_path:out1:RootOutput                           5.861e-06    7.9148e-06    1.2213e-05     7.644e-06    1.6087e-06       10     
end_path:out1:RootOutput(write)                     1.64378       1.99943       2.42188       1.92923      0.241601        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5306.43 MB
  Peak resident set size usage (VmHWM): 4500.47 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.31 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10806656
-rw-r--r-- 1 dune01 dune01 2512811777 Jan 11 07:30 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2386782484 Jan 11 07:28 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2205217361 Jan 11 08:09 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1974639543 Jan 11 06:03 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1909781407 Jan 11 05:55 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   76149060 Jan 11 08:06 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242921 Jan 11 08:09 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 11 08:09 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 11 08:09 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 11 08:09 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 11 07:30 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 06:03 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 08:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 05:49 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 11 08:09 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 11 07:34 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 11 07:30 debugprod.log
justIN time: 2026-02-06 11:42:54 UTC       justIN version: 01.06.00