justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275408.1@dunegpschedd02.fnal.gov

Jobsub ID275408.1@dunegpschedd02.fnal.gov
Workflow ID12077
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-18 15:13:52
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce04
Last heartbeat2026-01-18 19:20:57
From worker nodeHostnameccwcondor0629
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-18 15:15:42
Input filesmonte-carlo-012077-000001
JobscriptExit code0
Real time4h (14600s)
CPU time3h (13912s = 95%)
Max RSS bytes4085878784 (3896 MiB)
Outputting started2026-01-18 19:19:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12077/1/001/reco_stage1.root
Finished2026-01-18 19:20:57
Saved logsjustin-logs:275408.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 154410 traces, 0 summary
Retagger: tagging trace set: gauss with 117011 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 117011 traces tagged "gauss"
FrameSaver: q=4.06077e+06 n=2051557 tag=gauss
wclsFrameSaver: saving 154410 traces tagged "wiener"
FrameSaver: q=4.28869e+06 n=1803491 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 49800 XUs and 46076 XVs -> 769 XUVs
C:0 T:1 90888 XUs and 85261 XVs -> 5514 XUVs
C:0 T:2 114558 XUs and 88918 XVs -> 7874 XUVs
C:0 T:3 51459 XUs and 33498 XVs -> 257 XUVs
C:0 T:4 13395 XUs and 18228 XVs -> 195 XUVs
C:0 T:5 68989 XUs and 85442 XVs -> 6818 XUVs
C:0 T:6 83481 XUs and 54796 XVs -> 2902 XUVs
C:0 T:7 34319 XUs and 19885 XVs -> 169 XUVs
C:0 T:8 10679 XUs and 11731 XVs -> 173 XUVs
C:0 T:9 75018 XUs and 101553 XVs -> 6856 XUVs
C:0 T:10 36546 XUs and 19058 XVs -> 2653 XUVs
C:0 T:11 74 XUs and 50 XVs -> 6 XUVs
34186 XUVs total
13725 collection wire objects
30856 potential space points
Neighbour search...
1734730 tests to find 902532 neighbours
Iterating with no regularization...
Begin: 9.79919e+07
0 8.42521e+07
1 8.33147e+07
2 8.31858e+07
3 8.31494e+07
Now with regularization...
Begin: 7.12173e+07
0 7.11936e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411818953
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411818953
fLifetime: 30000 [us]
18-Jan-2026 20:19:01 CET  Closed output file "reco_stage1.root"
18-Jan-2026 20:19:01 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          384.92        536.639       737.569       513.967       104.317        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00168988    0.00688295     0.0129613    0.00692387    0.00365461       10     
reco:ophit:OpHitFinder                             0.290008       0.35375      0.411269      0.373803      0.0425498       10     
reco:opflash:OpFlashFinder                         0.131267      0.204158      0.261762      0.219731      0.0439025       10     
reco:opslicer:OpSlicer                             0.658359      0.833424       1.07583      0.818043      0.124234        10     
reco:caldata:DataPrepModule                         7.01343       7.20464       7.46227       7.17117      0.170358        10     
reco:wclsdatasp:WireCellToolkit                     41.2649       43.5245       45.8625       43.3053       1.53637        10     
reco:gaushit:GausHitFinder                          1.21756       1.67801       2.09564       1.66009      0.297551        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000631495   0.000933442   0.00144689    0.000950061   0.000241027      10     
reco:reco3d:SpacePointSolver                        10.5323       20.2271       37.3039       20.0616       6.98488        10     
reco:hitpdune:DisambigFromSpacePoints               1.44919       2.23171       3.23764       2.09772      0.575885        10     
reco:pandora:StandardPandora                        37.018        64.2838       144.33        57.846        30.1277        10     
reco:pandoraWriter:StandardPandora                 0.431035      0.634542      0.859425      0.610631      0.129132        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.54723       8.8536        12.2756        9.037        1.87603        10     
reco:pandoraShower:LArPandoraShowerCreation         6.56074       11.3398       17.5835       10.5208       3.41392        10     
reco:pandoracalo:Calorimetry                        3.1254        5.6288        7.49282       5.86924       1.22969        10     
reco:pandoracalonosce:Calorimetry                   2.39601       5.28923       7.79023       5.34806       1.42512        10     
reco:pandorapid:Chi2ParticleID                    0.00605959    0.00856035     0.0112554    0.00852125    0.00150792       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0717354     0.0824976     0.102343      0.0806675     0.0101941       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0057984    0.00803735    0.00925117     0.0080514    0.00100866       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.0136        13.2258       21.6822       12.9298       4.72022        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.1699        12.5288       19.3793       12.4405       4.21042        10     
reco:emtrkmichelid:EmTrackMichelId                  187.842       234.322       302.986       223.659       35.9966        10     
reco:crttag:SingleCRTMatchingProducer              0.0163523     0.036319      0.0679125     0.0322007     0.0148803       10     
reco:crtreco:TwoCRTMatchingProducer                0.0139311     0.0209043     0.0283423     0.0215123    0.00450985       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.804495       1.03248       1.40878      0.961718      0.185118        10     
reco:pandora2Track:LArPandoraTrackCreation          19.0285       28.8798       46.1117       27.5103       7.88632        10     
reco:pandora2calo:Calorimetry                       9.92452       17.6109       30.3224       16.3082       6.15319        10     
reco:pandora2calonosce:Calorimetry                  8.9387        15.865        27.4506       14.9616       5.24564        10     
reco:pandora2pid:Chi2ParticleID                    0.0177477     0.0201702     0.0242165     0.0187232    0.00248641       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0862656     0.108844      0.163999      0.104625      0.022363        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0171853     0.020537      0.0286832     0.0189252    0.00363768       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.08525       10.4896       16.9954       10.1349       2.86438        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.13159       14.092        26.3505       12.661        5.24195        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.87489       12.3723       23.1731       11.6285       4.45723        10     
[art]:TriggerResults:TriggerResultInserter        7.7756e-05    0.00010897    0.000224922   9.0655e-05    4.2018e-05       10     
end_path:out1:RootOutput                          1.1772e-05    1.62004e-05    2.699e-05    1.4953e-05    3.97866e-06      10     
end_path:out1:RootOutput(write)                     2.64414       3.37269       4.47525       3.18303      0.547121        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4918.06 MB
  Peak resident set size usage (VmHWM): 4085.88 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.54 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9825460
-rw-r--r-- 1 dune001 lbno 2329270184 Jan 18 18:46 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2212819124 Jan 18 18:43 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2011468339 Jan 18 20:19 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1753753611 Jan 18 16:37 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1679199121 Jan 18 16:24 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   74144970 Jan 18 20:10 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242802 Jan 18 20:19 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 18 20:19 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 18 20:19 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 18 20:19 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 18 18:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 16:37 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 20:19 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 18 16:15 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 18 20:19 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 18 18:56 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 18 18:46 debugprod.log
justIN time: 2026-02-05 18:28:46 UTC       justIN version: 01.06.00