justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 281932.1@dunegpschedd01.fnal.gov

Jobsub ID281932.1@dunegpschedd01.fnal.gov
Workflow ID12088
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-19 08:24:55
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-19 12:25:39
From worker nodeHostnameheplnc170.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-19 08:26:15
Input filesmonte-carlo-012088-000001
JobscriptExit code0
Real time3h (14284s)
CPU time3h (13705s = 95%)
Max RSS bytes4103757824 (3913 MiB)
Outputting started2026-01-19 12:24:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12088/1/001/reco_stage1.root
Finished2026-01-19 12:25:39
Saved logsjustin-logs:281932.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120437 traces, 0 summary
Retagger: tagging trace set: gauss with 89789 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 89789 traces tagged "gauss"
FrameSaver: q=3.20413e+06 n=1544479 tag=gauss
wclsFrameSaver: saving 120437 traces tagged "wiener"
FrameSaver: q=3.39587e+06 n=1357724 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 48812 XUs and 40788 XVs -> 1715 XUVs
C:0 T:1 105242 XUs and 85350 XVs -> 15509 XUVs
C:0 T:2 79012 XUs and 112107 XVs -> 7684 XUVs
C:0 T:3 41203 XUs and 36152 XVs -> 404 XUVs
C:0 T:4 17086 XUs and 15064 XVs -> 267 XUVs
C:0 T:5 62603 XUs and 55574 XVs -> 6006 XUVs
C:0 T:6 51624 XUs and 70077 XVs -> 2873 XUVs
C:0 T:7 23039 XUs and 30842 XVs -> 171 XUVs
C:0 T:8 5846 XUs and 6723 XVs -> 168 XUVs
C:0 T:9 47776 XUs and 63963 XVs -> 9439 XUVs
C:0 T:10 21132 XUs and 5664 XVs -> 2509 XUVs
C:0 T:11 89 XUs and 73 XVs -> 12 XUVs
46757 XUVs total
10166 collection wire objects
42360 potential space points
Neighbour search...
6780608 tests to find 3206066 neighbours
Iterating with no regularization...
Begin: 7.61046e+07
0 6.67207e+07
1 6.61397e+07
2 6.60563e+07
3 6.60287e+07
Now with regularization...
Begin: 5.60561e+07
0 5.60296e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411833395
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411833395
fLifetime: 30000 [us]
19-Jan-2026 12:24:18 GMT  Closed output file "reco_stage1.root"
19-Jan-2026 12:24:18 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          393.532       450.876       530.392       441.663       46.3584        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00125387    0.00479284    0.00764085    0.00504937    0.00203606       10     
reco:ophit:OpHitFinder                             0.381176      0.416588      0.504024      0.406777      0.0360663       10     
reco:opflash:OpFlashFinder                         0.189786      0.222041      0.301287      0.210354      0.0322538       10     
reco:opslicer:OpSlicer                             0.738267       0.95902       1.18159      0.977976      0.132131        10     
reco:caldata:DataPrepModule                         7.40432       8.0151        8.81496       7.99338      0.373767        10     
reco:wclsdatasp:WireCellToolkit                     42.8776       43.7951       45.2521       43.7338      0.719818        10     
reco:gaushit:GausHitFinder                          0.88194       1.61241       1.85625       1.67775       0.26545        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00039048    0.000691349   0.00104638    0.000694808   0.00015563       10     
reco:reco3d:SpacePointSolver                        12.8796       20.8813       30.5586       19.5967       5.14209        10     
reco:hitpdune:DisambigFromSpacePoints               1.7131        2.22588       3.09193       2.17971      0.410282        10     
reco:pandora:StandardPandora                        29.8461       42.9415       56.9306       41.4731       8.5566         10     
reco:pandoraWriter:StandardPandora                 0.528567      0.634688       0.76146      0.625395      0.0702994       10     
reco:pandoraTrack:LArPandoraTrackCreation            4.262        5.41705       7.30802        5.248       0.806751        10     
reco:pandoraShower:LArPandoraShowerCreation         3.68827       5.13897       7.03428       5.17471       1.04844        10     
reco:pandoracalo:Calorimetry                        2.6472        3.14538       4.02985       2.9766       0.429501        10     
reco:pandoracalonosce:Calorimetry                   2.35495       2.85874       3.74468       2.59032      0.458456        10     
reco:pandorapid:Chi2ParticleID                    0.00673324    0.00886201     0.0112989    0.00867441    0.00128357       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0751421     0.0919533     0.144081      0.0856658     0.0190432       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00752461    0.00870815     0.0109407    0.00861598    0.000916059      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.12787       5.73005       7.87946       5.41781       1.15121        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.39508       5.70346       7.83695       5.33969       1.13432        10     
reco:emtrkmichelid:EmTrackMichelId                  218.926       243.442       281.267       240.229       20.7715        10     
reco:crttag:SingleCRTMatchingProducer              0.0155794     0.0203947     0.0350977     0.0184827     0.0053336       10     
reco:crtreco:TwoCRTMatchingProducer                0.0101117     0.0118948     0.0140248     0.011729     0.00151776       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.926177       1.14858       1.29097       1.15523       0.10899        10     
reco:pandora2Track:LArPandoraTrackCreation          12.2522       16.5684       23.4199       15.6647       3.40598        10     
reco:pandora2calo:Calorimetry                       6.53397       8.40733       11.5523       7.9765        1.49262        10     
reco:pandora2calonosce:Calorimetry                  6.00097        8.107        10.8897       7.9638        1.54158        10     
reco:pandora2pid:Chi2ParticleID                    0.0172724     0.0205974     0.0286222     0.0199954     0.0031182       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0929486     0.104514      0.133137      0.0971276     0.0125524       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.014426      0.0197381     0.0244721     0.0200865    0.00290142       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.44876       5.69779       7.45135       5.55998       1.04204        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.37559       7.12097       9.58108       6.78936       1.39988        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.09238       6.48604       8.9419        6.19538       1.27678        10     
[art]:TriggerResults:TriggerResultInserter        7.3477e-05    0.000101903   0.000197389   8.6241e-05    3.67209e-05      10     
end_path:out1:RootOutput                           1.042e-05    1.37416e-05   2.3214e-05    1.27585e-05   3.4212e-06       10     
end_path:out1:RootOutput(write)                     3.22608       3.76734       4.61835       3.62493       0.44349        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5028 MB
  Peak resident set size usage (VmHWM): 4103.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10052532
-rw-r--r-- 1 pltdune04 pltdune 2371792683 Jan 19 11:06 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2252830570 Jan 19 11:03 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2042617818 Jan 19 12:24 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1814866407 Jan 19 08:48 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1739346352 Jan 19 08:34 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   71728490 Jan 19 12:19 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242764 Jan 19 12:24 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 19 12:24 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 19 12:24 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 19 12:24 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 19 11:06 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 19 08:48 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 19 12:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 19 08:26 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 19 12:24 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 19 11:14 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 19 11:06 debugprod.log
justIN time: 2026-02-05 18:10:07 UTC       justIN version: 01.06.00