justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 254069.1@dunegpschedd01.fnal.gov

Jobsub ID254069.1@dunegpschedd01.fnal.gov
Workflow ID10377
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-19 09:33:46
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-19 17:01:31
From worker nodeHostnamewn5917301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-19 09:35:19
Input filesmonte-carlo-010377-000001
JobscriptExit code0
Real time7h (25840s)
CPU time6h (24844s = 96%)
Max RSS bytes4161646592 (3968 MiB)
Outputting started2025-11-19 16:46:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10377/1/001/reco_stage1.root
Finished2025-11-19 17:01:31
Saved logsjustin-logs:254069.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lBasedRawDigitPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 107032 traces, 0 summary
Retagger: tagging trace set: gauss with 77094 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 77094 traces tagged "gauss"
FrameSaver: q=2.52697e+06 n=1277167 tag=gauss
wclsFrameSaver: saving 107032 traces tagged "wiener"
FrameSaver: q=2.70654e+06 n=1099980 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 3335 XUs and 6093 XVs -> 15 XUVs
C:0 T:1 8111 XUs and 9614 XVs -> 563 XUVs
C:0 T:2 20398 XUs and 38860 XVs -> 2845 XUVs
C:0 T:3 10626 XUs and 14371 XVs -> 72 XUVs
C:0 T:4 8158 XUs and 11082 XVs -> 149 XUVs
C:0 T:5 36293 XUs and 56465 XVs -> 9220 XUVs
C:0 T:6 61195 XUs and 71501 XVs -> 3937 XUVs
C:0 T:7 25040 XUs and 26173 XVs -> 203 XUVs
C:0 T:8 11132 XUs and 15153 XVs -> 172 XUVs
C:0 T:9 93052 XUs and 157854 XVs -> 9002 XUVs
C:0 T:10 29913 XUs and 37011 XVs -> 5132 XUVs
31310 XUVs total
10524 collection wire objects
29637 potential space points
Neighbour search...
4023695 tests to find 1834956 neighbours
Iterating with no regularization...
Begin: 4.92127e+07
0 4.04371e+07
1 3.98813e+07
2 3.98016e+07
3 3.97782e+07
Now with regularization...
Begin: 3.21437e+07
0 3.21156e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410607177
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410607177
fLifetime: 30000 [us]
19-Nov-2025 16:45:56 GMT  Closed output file "reco_stage1.root"
19-Nov-2025 16:45:56 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          658.665       991.084       1268.51       1028.74       202.293        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00460896     0.0109063     0.0250621    0.00838813    0.00710283       10     
reco:ophit:OpHitFinder                             0.375219      0.461394      0.587547      0.445133      0.0654016       10     
reco:opflash:OpFlashFinder                         0.213913      0.294271       0.39784      0.274505      0.0555932       10     
reco:opslicer:OpSlicer                              0.93178       1.23206       1.53945       1.25243      0.195187        10     
reco:caldata:DataPrepModule                         9.31264       10.2527       11.166        10.2649      0.517626        10     
reco:wclsdatasp:WireCellToolkit                     75.7205       87.7814       97.8959       88.0111       8.02841        10     
reco:gaushit:GausHitFinder                          1.76256       2.51132       3.02547       2.52754      0.408306        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00152533    0.00242729    0.00364796    0.00232282    0.000614975      10     
reco:reco3d:SpacePointSolver                        13.1006       30.1405       78.0677       24.466        17.7977        10     
reco:hitpdune:DisambigFromSpacePoints               1.97966       3.49549       4.71212       3.60596      0.829714        10     
reco:pandora:StandardPandora                        33.5978       84.7762       128.83        89.5518       27.9042        10     
reco:pandoraWriter:StandardPandora                 0.879069       1.1659        1.46321       1.22871      0.177586        10     
reco:pandoraTrack:LArPandoraTrackCreation           12.9139       18.6864       22.3811       19.6335       3.08412        10     
reco:pandoraShower:LArPandoraShowerCreation         12.4663       22.3357       31.2791       24.3725        5.672         10     
reco:pandoracalo:Calorimetry                        6.17503       11.175        14.2902       12.1614       2.60819        10     
reco:pandoracalonosce:Calorimetry                   5.65279       10.7744       13.7748       11.7227        2.568         10     
reco:pandorapid:Chi2ParticleID                    0.00925086     0.0141672     0.0195671     0.0135013    0.00295053       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0803784     0.123311       0.15716      0.126183      0.0261515       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.008819      0.0138756     0.023733      0.0131054    0.00385204       10     
reco:pandoraShowercalo:ShowerCalorimetry            9.96505       26.3737       46.2067       27.6305       10.7466        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.16361       26.0258       37.5787       30.112        9.35895        10     
reco:emtrkmichelid:EmTrackMichelId                  341.73        433.648       514.726       439.546       62.4326        10     
reco:crttag:SingleCRTMatchingProducer              0.0313653     0.0587577      0.11111      0.0506819     0.024092        10     
reco:crtreco:TwoCRTMatchingProducer                0.0246845     0.0478876     0.0881022     0.0441359     0.017832        10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.16549       1.70511       2.4651        1.68287      0.407995        10     
reco:pandora2Track:LArPandoraTrackCreation          34.2589       60.9951       81.6953       62.8725       16.6221        10     
reco:pandora2calo:Calorimetry                       12.5707       33.5902       50.0645       37.2541       12.1519        10     
reco:pandora2calonosce:Calorimetry                  12.3656       34.4184       54.6076       37.3591       13.7322        10     
reco:pandora2pid:Chi2ParticleID                    0.0222099     0.0341897     0.0557255     0.0289309     0.0100824       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.109193      0.161299      0.214641       0.15519      0.0381937       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.022673      0.0315763     0.0445215     0.0304057    0.00644686       10     
reco:pandora2Shower:LArPandoraShowerCreation        12.0664       22.6336       29.8791       24.4342       6.17143        10     
reco:pandora2Showercalo:ShowerCalorimetry           14.2696       31.073        48.0278       33.7151       11.0916        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      14.2896       29.0246       46.8339       29.0599       10.9354        10     
[art]:TriggerResults:TriggerResultInserter        5.9453e-05    0.000128925   0.000324171   0.000102273   7.75583e-05      10     
end_path:out1:RootOutput                           9.358e-06    1.4313e-05    2.7361e-05     1.318e-05    5.07236e-06      10     
end_path:out1:RootOutput(write)                     4.05724       5.52494       6.99708       5.51079       0.9325         10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5006.07 MB
  Peak resident set size usage (VmHWM): 4161.65 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.52 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.16 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9546308
-rw-r--r-- 1 dune01 dune01 2259188249 Nov 19 13:56 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2147129893 Nov 19 13:52 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1928879288 Nov 19 16:45 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1724422607 Nov 19 10:13 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1643969590 Nov 19 09:47 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   71200930 Nov 19 16:37 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242772 Nov 19 16:46 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 19 16:45 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 19 16:45 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 19 16:45 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 19 13:56 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 19 10:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 19 16:45 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 19 09:35 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 19 16:46 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 19 14:11 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 19 13:56 debugprod.log
justIN time: 2026-02-09 05:01:06 UTC       justIN version: 01.06.00