justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275790.0@dunegpschedd01.fnal.gov

Jobsub ID275790.0@dunegpschedd01.fnal.gov
Workflow ID11739
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 08:41:54
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-12 13:16:06
From worker nodeHostnameccwcondor0583
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 08:42:35
Input filesmonte-carlo-011739-000001
JobscriptExit code0
Real time4h (16297s)
CPU time4h (15638s = 95%)
Max RSS bytes5124415488 (4887 MiB)
Outputting started2026-01-12 13:14:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11739/1/001/reco_stage1.root
Finished2026-01-12 13:16:06
Saved logsjustin-logs:275790.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123646 traces, 0 summary
Retagger: tagging trace set: gauss with 94076 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94076 traces tagged "gauss"
FrameSaver: q=3.30994e+06 n=1662840 tag=gauss
wclsFrameSaver: saving 123646 traces tagged "wiener"
FrameSaver: q=3.49791e+06 n=1470621 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 35364 XUs and 36991 XVs -> 564 XUVs
C:0 T:1 77028 XUs and 93722 XVs -> 10156 XUVs
C:0 T:2 91277 XUs and 106229 XVs -> 7541 XUVs
C:0 T:3 37296 XUs and 43296 XVs -> 210 XUVs
C:0 T:4 6797 XUs and 9827 XVs -> 92 XUVs
C:0 T:5 23569 XUs and 35324 XVs -> 4338 XUVs
C:0 T:6 97293 XUs and 87484 XVs -> 4137 XUVs
C:0 T:7 43010 XUs and 36170 XVs -> 284 XUVs
C:0 T:8 9717 XUs and 8145 XVs -> 151 XUVs
C:0 T:9 50824 XUs and 54804 XVs -> 3589 XUVs
C:0 T:10 41327 XUs and 52146 XVs -> 10654 XUVs
C:0 T:11 65 XUs and 45 XVs -> 33 XUVs
41749 XUVs total
10521 collection wire objects
38651 potential space points
Neighbour search...
4753547 tests to find 2399118 neighbours
Iterating with no regularization...
Begin: 6.85072e+07
0 5.56582e+07
1 5.49284e+07
2 5.4815e+07
3 5.47787e+07
Now with regularization...
Begin: 4.54268e+07
0 4.5376e+07
1 4.53353e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411692808
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411692808
fLifetime: 30000 [us]
12-Jan-2026 14:14:10 CET  Closed output file "reco_stage1.root"
12-Jan-2026 14:14:10 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          401.553       564.46        851.216       534.858       126.866        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00203503    0.00916813     0.0288352     0.005206     0.00927193       10     
reco:ophit:OpHitFinder                             0.355639      0.395594      0.471418      0.386916      0.0305071       10     
reco:opflash:OpFlashFinder                         0.220613      0.249808      0.333381      0.246252      0.0310846       10     
reco:opslicer:OpSlicer                             0.843063      0.971394       1.23579      0.940054      0.118051        10     
reco:caldata:DataPrepModule                         6.61755       7.06389       7.26984       7.0685       0.182663        10     
reco:wclsdatasp:WireCellToolkit                     38.6435       44.9701       51.8823       44.213        3.62413        10     
reco:gaushit:GausHitFinder                          1.37756       1.83903       2.28962       1.88943      0.260369        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000576367    0.0014584    0.00776248    0.000796484   0.00210545       10     
reco:reco3d:SpacePointSolver                        14.2129       25.5264       40.634        23.7952       7.88751        10     
reco:hitpdune:DisambigFromSpacePoints               1.78955       2.68407       4.31124       2.49852      0.708313        10     
reco:pandora:StandardPandora                        28.9142       58.9185       93.2373       59.8322       19.6184        10     
reco:pandoraWriter:StandardPandora                 0.520247       0.66699      0.856792      0.672843      0.110802        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.19043       9.05299       12.0511       8.91209       1.80163        10     
reco:pandoraShower:LArPandoraShowerCreation         6.1537        11.3925       20.8377       10.5702       4.21788        10     
reco:pandoracalo:Calorimetry                        3.80286       5.18563       6.70083       5.23785      0.873432        10     
reco:pandoracalonosce:Calorimetry                   3.20842       4.79963       6.32027       4.68503      0.843964        10     
reco:pandorapid:Chi2ParticleID                    0.00686661    0.00822104     0.0101874    0.00830613    0.000967623      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0680472     0.0887818     0.106469      0.0870458     0.0112811       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00697903    0.00860384     0.0140217    0.00806277    0.00187012       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.49121       12.0667       22.8046       11.6198       4.42403        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.08175       11.7309       24.0871       10.5177       4.89995        10     
reco:emtrkmichelid:EmTrackMichelId                  196.784       261.206       375.514       252.659       47.8441        10     
reco:crttag:SingleCRTMatchingProducer              0.014922      0.0273551     0.0469497     0.0250238     0.0092198       10     
reco:crtreco:TwoCRTMatchingProducer                0.0120031     0.0194132     0.0269376     0.0189321     0.0042598       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.855578       1.19154       1.60688       1.19164      0.223667        10     
reco:pandora2Track:LArPandoraTrackCreation          18.9702       29.6937       51.4689       25.7813       9.79102        10     
reco:pandora2calo:Calorimetry                       10.3733       17.0447       31.3708       14.7323       6.41803        10     
reco:pandora2calonosce:Calorimetry                  9.28088       15.8286       27.6451       14.2904       5.46874        10     
reco:pandora2pid:Chi2ParticleID                    0.0156669     0.0215163     0.0305806     0.0213419    0.00397478       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0781206     0.113639      0.147356      0.107932      0.0204856       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0155304     0.0209343     0.0299214     0.0204806    0.00382792       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.22781       10.5991       19.2743       9.31047       3.46034        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.90116       14.5281       28.0213       12.6357       5.65098        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.53139       12.4616       21.9489       11.4018       3.98485        10     
[art]:TriggerResults:TriggerResultInserter        6.5413e-05    9.84201e-05   0.00019256    8.63215e-05   3.52632e-05      10     
end_path:out1:RootOutput                          1.2504e-05    1.40504e-05   1.8194e-05    1.37755e-05   1.57226e-06      10     
end_path:out1:RootOutput(write)                     2.95487       3.86308       5.40854       3.71932      0.660532        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5951.55 MB
  Peak resident set size usage (VmHWM): 5124.42 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.44 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11819692
-rw-r--r-- 1 dune001 lbno 2713233976 Jan 12 12:37 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2575922461 Jan 12 12:34 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2447534394 Jan 12 14:14 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2189708237 Jan 12 10:09 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2090631198 Jan 12 09:51 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   85720160 Jan 12 14:08 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242935 Jan 12 14:14 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 12 14:14 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 12 14:14 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 12 14:14 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 12 12:37 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 10:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 14:14 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 12 09:42 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 12 14:14 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:45 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:37 debugprod.log
justIN time: 2026-02-06 10:21:19 UTC       justIN version: 01.06.00