justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301992.0@dunegpschedd01.fnal.gov

Jobsub ID301992.0@dunegpschedd01.fnal.gov
Workflow ID12494
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-28 16:18:52
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-01-28 20:23:31
From worker nodeHostnameheplnc169.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-28 16:19:36
Input filesmonte-carlo-012494-000001
JobscriptExit code0
Real time3h (14322s)
CPU time3h (13541s = 94%)
Max RSS bytes3939074048 (3756 MiB)
Outputting started2026-01-28 20:18:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12494/1/001/reco_stage1.root
Finished2026-01-28 20:23:31
Saved logsjustin-logs:301992.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 121725 traces, 0 summary
Retagger: tagging trace set: gauss with 89882 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 89882 traces tagged "gauss"
FrameSaver: q=3.33727e+06 n=1590676 tag=gauss
wclsFrameSaver: saving 121725 traces tagged "wiener"
FrameSaver: q=3.5521e+06 n=1395234 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30466 XUs and 31015 XVs -> 981 XUVs
C:0 T:1 111792 XUs and 188872 XVs -> 30102 XUVs
C:0 T:2 124871 XUs and 133849 XVs -> 9775 XUVs
C:0 T:3 51371 XUs and 52570 XVs -> 629 XUVs
C:0 T:4 13834 XUs and 12974 XVs -> 206 XUVs
C:0 T:5 99710 XUs and 118905 XVs -> 19515 XUVs
C:0 T:6 64379 XUs and 38428 XVs -> 4612 XUVs
C:0 T:7 23884 XUs and 15223 XVs -> 227 XUVs
C:0 T:8 8265 XUs and 8582 XVs -> 113 XUVs
C:0 T:9 65196 XUs and 78844 XVs -> 6028 XUVs
C:0 T:10 13338 XUs and 10575 XVs -> 1884 XUVs
C:0 T:11 26 XUs and 17 XVs -> 3 XUVs
74075 XUVs total
11378 collection wire objects
66803 potential space points
Neighbour search...
17033037 tests to find 6919024 neighbours
Iterating with no regularization...
Begin: 7.985e+07
0 6.21712e+07
1 6.09475e+07
2 6.07791e+07
3 6.07244e+07
Now with regularization...
Begin: 5.01455e+07
0 5.00464e+07
1 4.99793e+07
2 4.99296e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 412020962
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412020962
fLifetime: 30000 [us]
28-Jan-2026 20:18:18 GMT  Closed output file "reco_stage1.root"
28-Jan-2026 20:18:18 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          300.227       431.844       593.854       436.489       85.0099        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00159481     0.0051819     0.0112515    0.00479149    0.00282098       10     
reco:ophit:OpHitFinder                              0.30952       0.37067      0.435896      0.366552      0.0354278       10     
reco:opflash:OpFlashFinder                         0.148887      0.187011      0.223414      0.180932      0.0236852       10     
reco:opslicer:OpSlicer                             0.774886      0.886801        1.033       0.869156      0.0835527       10     
reco:caldata:DataPrepModule                         7.17665       7.9657        8.85718       7.88131      0.447026        10     
reco:wclsdatasp:WireCellToolkit                     39.5832       43.8602       48.0959       44.2018       2.11961        10     
reco:gaushit:GausHitFinder                         0.894366       1.53035       1.97907       1.60006      0.295871        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000435686   0.000702532   0.000866882   0.000727577   0.00013863       10     
reco:reco3d:SpacePointSolver                        9.48471       23.2909       43.9699       20.9876       10.0273        10     
reco:hitpdune:DisambigFromSpacePoints               1.16451       2.0367        3.66006       1.93286      0.679767        10     
reco:pandora:StandardPandora                        20.5432       41.5495       65.6678       41.9815       15.9861        10     
reco:pandoraWriter:StandardPandora                 0.369741      0.547432      0.739211      0.569311      0.135425        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.46786       4.52039       7.76561       4.02906       1.68697        10     
reco:pandoraShower:LArPandoraShowerCreation         2.25704       4.76716       9.27042       4.42811       1.9883         10     
reco:pandoracalo:Calorimetry                        1.64946       2.62594       4.01469       2.67871      0.665981        10     
reco:pandoracalonosce:Calorimetry                   1.24484       2.28597       3.87165       2.13224      0.786862        10     
reco:pandorapid:Chi2ParticleID                    0.00492535    0.00757664     0.009242     0.00735654    0.00134981       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0482652     0.0753726     0.102678      0.0778094     0.0168221       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00471713    0.00728337    0.00929792    0.00727679    0.00121979       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.18686       5.20023       9.62517       4.86186       2.26475        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.43202       4.84431       7.88639       4.18969       1.93911        10     
reco:emtrkmichelid:EmTrackMichelId                  174.258       231.607       308.194       232.288       38.0586        10     
reco:crttag:SingleCRTMatchingProducer              0.0134397      0.01933      0.0310934     0.0182299    0.00547443       10     
reco:crtreco:TwoCRTMatchingProducer               0.00901909     0.0118568     0.0178902     0.0108616    0.00283706       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.804832       1.06682       1.44086       1.00971      0.189401        10     
reco:pandora2Track:LArPandoraTrackCreation          8.29595       15.5428       24.669        15.4687       5.32903        10     
reco:pandora2calo:Calorimetry                       3.22602       7.76683       11.9028       7.80079       2.88806        10     
reco:pandora2calonosce:Calorimetry                  3.53557       7.38811       12.4128       6.9987        2.81905        10     
reco:pandora2pid:Chi2ParticleID                    0.0154413     0.0193192     0.0265779     0.0176555    0.00394818       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0810533     0.100634      0.138391      0.0970081     0.0170274       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0155764     0.0186675     0.0252457     0.0176809    0.00321455       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.81903       5.31063       9.13085       4.82096       1.74271        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.34961       6.63903       12.4559       6.13094       2.54176        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.1524        5.92106       10.8106       5.5974        2.1795         10     
[art]:TriggerResults:TriggerResultInserter        6.5153e-05    9.60489e-05   0.000236142   8.04355e-05   4.78527e-05      10     
end_path:out1:RootOutput                           1.097e-05    1.35282e-05   2.0849e-05    1.26185e-05   2.82567e-06      10     
end_path:out1:RootOutput(write)                     2.99702       3.71871       5.12115       3.61915      0.667682        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4844.4 MB
  Peak resident set size usage (VmHWM): 3939.07 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.97 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9323420
-rw-r--r-- 1 pltdune04 pltdune 2220107243 Jan 28 19:03 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2110884706 Jan 28 19:01 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1888915711 Jan 28 20:18 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1660688440 Jan 28 16:44 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1595181531 Jan 28 16:29 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   70783480 Jan 28 20:13 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243435 Jan 28 20:18 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 28 20:18 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 28 20:18 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 28 20:18 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 19:03 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 16:44 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 20:18 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 28 16:19 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 28 20:18 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 28 19:11 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 28 19:03 debugprod.log
justIN time: 2026-02-04 18:08:27 UTC       justIN version: 01.06.00