justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270057.0@dunegpschedd02.fnal.gov

Jobsub ID270057.0@dunegpschedd02.fnal.gov
Workflow ID11744
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 08:41:55
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2026-01-12 12:24:15
From worker nodeHostnameccwcondor0552
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 08:42:47
Input filesmonte-carlo-011744-000001
JobscriptExit code0
Real time3h (13175s)
CPU time3h (12713s = 96%)
Max RSS bytes4278177792 (4079 MiB)
Outputting started2026-01-12 12:22:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11744/1/001/reco_stage1.root
Finished2026-01-12 12:24:15
Saved logsjustin-logs:270057.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112406 traces, 0 summary
Retagger: tagging trace set: gauss with 86047 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 86047 traces tagged "gauss"
FrameSaver: q=2.83657e+06 n=1502603 tag=gauss
wclsFrameSaver: saving 112406 traces tagged "wiener"
FrameSaver: q=2.99331e+06 n=1315857 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 12354 XUs and 16358 XVs -> 95 XUVs
C:0 T:1 30645 XUs and 28262 XVs -> 3401 XUVs
C:0 T:2 117000 XUs and 120023 XVs -> 10624 XUVs
C:0 T:3 39445 XUs and 45364 XVs -> 364 XUVs
C:0 T:4 12216 XUs and 12022 XVs -> 54 XUVs
C:0 T:5 50870 XUs and 35983 XVs -> 4519 XUVs
C:0 T:6 134007 XUs and 119158 XVs -> 6919 XUVs
C:0 T:7 46242 XUs and 45864 XVs -> 408 XUVs
C:0 T:8 4638 XUs and 3496 XVs -> 56 XUVs
C:0 T:9 28971 XUs and 28076 XVs -> 1522 XUVs
C:0 T:10 39819 XUs and 21590 XVs -> 3378 XUVs
C:0 T:11 127 XUs and 89 XVs -> 9 XUVs
31349 XUVs total
10201 collection wire objects
29046 potential space points
Neighbour search...
3742588 tests to find 1662490 neighbours
Iterating with no regularization...
Begin: 4.97726e+07
0 4.07403e+07
1 4.0133e+07
2 4.00443e+07
3 4.00191e+07
Now with regularization...
Begin: 3.24903e+07
0 3.24602e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411692749
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411692749
fLifetime: 30000 [us]
12-Jan-2026 13:22:21 CET  Closed output file "reco_stage1.root"
12-Jan-2026 13:22:21 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          323.104       421.096       538.364       421.574       59.8264        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131869    0.00437122    0.00793166    0.00441362    0.00215079       10     
reco:ophit:OpHitFinder                             0.244234      0.356569      0.411304      0.370378      0.0422096       10     
reco:opflash:OpFlashFinder                         0.193446      0.227011      0.287287      0.218971      0.0292564       10     
reco:opslicer:OpSlicer                             0.785251      0.834153      0.890603      0.825977      0.0395899       10     
reco:caldata:DataPrepModule                         5.9182        6.60889       7.22216       6.61133      0.377978        10     
reco:wclsdatasp:WireCellToolkit                     33.9992        39.25        41.6856       40.187        2.42993        10     
reco:gaushit:GausHitFinder                          1.12677       1.3508        1.59614       1.36032      0.155539        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000498434   0.000588572   0.000667251   0.000593467   6.04158e-05      10     
reco:reco3d:SpacePointSolver                        12.9343       19.7935       38.6073       18.4114       7.42921        10     
reco:hitpdune:DisambigFromSpacePoints               1.33703       1.7982        2.18696       1.84339      0.278668        10     
reco:pandora:StandardPandora                        22.3517       37.212        61.4235       33.0295       10.869         10     
reco:pandoraWriter:StandardPandora                 0.299088      0.535324      0.679635      0.562777      0.0977269       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.54944       7.29441       10.1005       7.58641       1.57208        10     
reco:pandoraShower:LArPandoraShowerCreation         3.62038       7.6846        10.2954       8.18285       2.07502        10     
reco:pandoracalo:Calorimetry                        2.94693       4.22547       5.5405        4.48653      0.905136        10     
reco:pandoracalonosce:Calorimetry                   2.48519       3.9519        5.47196       4.02381       1.04563        10     
reco:pandorapid:Chi2ParticleID                    0.00406183    0.00745521     0.0113563    0.00730222    0.00210763       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0333172     0.0636911     0.0985261     0.0662118     0.0212778       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00387511    0.00658576    0.00863709     0.0069384     0.0017329       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.65987       8.2125        12.7848       8.20581        2.392         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.59069       8.17221       13.4082       8.22914       2.53986        10     
reco:emtrkmichelid:EmTrackMichelId                  162.89        201.148       242.932       205.187       22.9716        10     
reco:crttag:SingleCRTMatchingProducer              0.0161822     0.019889      0.0247711     0.0197991    0.00304262       10     
reco:crtreco:TwoCRTMatchingProducer               0.00975231     0.0164748     0.0277866     0.0152394    0.00481874       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.840086      0.993088       1.19499      0.995214      0.130255        10     
reco:pandora2Track:LArPandoraTrackCreation          13.8213       21.1748       29.4352       21.596        4.62922        10     
reco:pandora2calo:Calorimetry                       5.12476       10.8619       15.5419       12.0479       3.06546        10     
reco:pandora2calonosce:Calorimetry                  4.64667       10.6349       16.3467       11.4901       3.19968        10     
reco:pandora2pid:Chi2ParticleID                   0.00931283     0.0173916     0.022858      0.0182076    0.00390316       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0473249     0.0849244     0.111647      0.0913473     0.0208759       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00787075     0.0166154     0.0226245     0.0181032    0.00448012       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.38574       7.74352       10.6925       8.25172       1.74614        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.53577       9.42474       13.5671       9.4211        2.71995        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.50076       8.19611       11.2352       8.7951        2.05885        10     
[art]:TriggerResults:TriggerResultInserter        6.1075e-05    8.61345e-05   0.000162044   7.8973e-05    2.63187e-05      10     
end_path:out1:RootOutput                           9.678e-06    1.29983e-05   2.0468e-05    1.23335e-05   2.82851e-06      10     
end_path:out1:RootOutput(write)                     1.90643       2.98748       4.03797       3.07082      0.609616        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5103.6 MB
  Peak resident set size usage (VmHWM): 4278.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.13 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10678368
-rw-r--r-- 1 dune001 lbno 2489458814 Jan 12 12:09 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2362335633 Jan 12 12:08 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2169855737 Jan 12 13:22 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1964017741 Jan 12 10:07 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1875130370 Jan 12 09:51 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   73236260 Jan 12 13:18 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242885 Jan 12 13:22 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 12 13:22 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 12 13:22 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 12 13:22 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 12 12:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 10:07 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 13:22 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 12 09:42 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 12 13:22 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:16 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:09 debugprod.log
justIN time: 2026-02-06 10:21:39 UTC       justIN version: 01.06.00