justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275405.0@dunegpschedd02.fnal.gov

Jobsub ID275405.0@dunegpschedd02.fnal.gov
Workflow ID12074
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-18 15:13:51
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-18 19:36:44
From worker nodeHostnameccwcondor0605
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-18 15:15:05
Input filesmonte-carlo-012074-000001
JobscriptExit code0
Real time4h (15590s)
CPU time4h (14972s = 96%)
Max RSS bytes4206690304 (4011 MiB)
Outputting started2026-01-18 19:34:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12074/1/001/reco_stage1.root
Finished2026-01-18 19:36:44
Saved logsjustin-logs:275405.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142590 traces, 0 summary
Retagger: tagging trace set: gauss with 107902 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107902 traces tagged "gauss"
FrameSaver: q=3.75653e+06 n=1903870 tag=gauss
wclsFrameSaver: saving 142590 traces tagged "wiener"
FrameSaver: q=3.97067e+06 n=1679721 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 47076 XUs and 50050 XVs -> 1211 XUVs
C:0 T:1 121393 XUs and 106818 XVs -> 14251 XUVs
C:0 T:2 79033 XUs and 84446 XVs -> 2873 XUVs
C:0 T:3 42188 XUs and 39602 XVs -> 235 XUVs
C:0 T:4 4867 XUs and 4589 XVs -> 51 XUVs
C:0 T:5 20715 XUs and 11969 XVs -> 1574 XUVs
C:0 T:6 216668 XUs and 224567 XVs -> 9567 XUVs
C:0 T:7 74888 XUs and 68984 XVs -> 773 XUVs
C:0 T:8 4478 XUs and 5503 XVs -> 46 XUVs
C:0 T:9 23526 XUs and 32236 XVs -> 1629 XUVs
C:0 T:10 63993 XUs and 42009 XVs -> 7401 XUVs
C:0 T:11 293 XUs and 212 XVs -> 28 XUVs
39639 XUVs total
13335 collection wire objects
35586 potential space points
Neighbour search...
3771802 tests to find 1761184 neighbours
Iterating with no regularization...
Begin: 9.14915e+07
0 7.81382e+07
1 7.7219e+07
2 7.7067e+07
3 7.70205e+07
Now with regularization...
Begin: 6.59522e+07
0 6.59204e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411818988
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411818988
fLifetime: 30000 [us]
18-Jan-2026 20:34:53 CET  Closed output file "reco_stage1.root"
18-Jan-2026 20:34:53 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          377.318       577.463       759.81        581.289       100.803        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00181284    0.00539201     0.0105675    0.00495768    0.00235643       10     
reco:ophit:OpHitFinder                             0.282636      0.352801      0.427123      0.341791      0.0479092       10     
reco:opflash:OpFlashFinder                         0.162704      0.213631      0.313228      0.199686      0.0448759       10     
reco:opslicer:OpSlicer                             0.658984      0.867177       1.11574      0.857492      0.150741        10     
reco:caldata:DataPrepModule                         6.4817        7.32275       7.80349       7.48375      0.421606        10     
reco:wclsdatasp:WireCellToolkit                     40.2654       48.3566       51.9271       48.605        3.03169        10     
reco:gaushit:GausHitFinder                          1.20805       1.85861       2.48958       1.83034      0.374546        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000649785   0.00174844    0.00354405    0.00149454    0.00100309       10     
reco:reco3d:SpacePointSolver                        10.5868       22.3135       33.3744       23.0677       6.31679        10     
reco:hitpdune:DisambigFromSpacePoints               1.3419        2.45171       3.30652       2.41891      0.600717        10     
reco:pandora:StandardPandora                        37.5377       67.1587       127.999       61.1181       24.5188        10     
reco:pandoraWriter:StandardPandora                 0.582447       0.67751      0.805484      0.685281      0.0616172       10     
reco:pandoraTrack:LArPandoraTrackCreation           7.5525        10.4489       15.3928       9.41964       2.74486        10     
reco:pandoraShower:LArPandoraShowerCreation         7.91555       13.3785        21.47        12.171        4.08325        10     
reco:pandoracalo:Calorimetry                        3.50013       6.38224       9.51217       6.32203       1.57065        10     
reco:pandoracalonosce:Calorimetry                   3.68741       6.13066       9.66971       5.88133       1.64753        10     
reco:pandorapid:Chi2ParticleID                     0.0055773    0.00896206     0.0121121    0.00938589    0.00173505       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0488131     0.0793731     0.0868385     0.082761      0.0105269       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00451788     0.0082169    0.00943896    0.00884501    0.00141382       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.04422       15.7716       24.1154       15.1152       5.52755        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.49727       14.6529       21.8231       13.7156       4.97219        10     
reco:emtrkmichelid:EmTrackMichelId                  179.939       242.18        302.697       241.709       31.7996        10     
reco:crttag:SingleCRTMatchingProducer              0.0234714     0.0335777     0.052425      0.030876     0.00907527       10     
reco:crtreco:TwoCRTMatchingProducer                0.013192      0.0245145     0.0365444     0.0240466    0.00619035       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.899514       1.04564       1.18906       1.04537      0.0832975       10     
reco:pandora2Track:LArPandoraTrackCreation          19.6675       33.7958       47.7027       32.8409       8.16145        10     
reco:pandora2calo:Calorimetry                       7.4378        19.972        26.3675       21.4387       5.02464        10     
reco:pandora2calonosce:Calorimetry                  7.35997       18.9808       26.0453       19.2749       5.26421        10     
reco:pandora2pid:Chi2ParticleID                    0.016038      0.022047      0.0276993     0.0216085    0.00407312       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0784324     0.112076      0.155167      0.109862      0.0201326       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0157324     0.0204273     0.027006      0.0201921     0.0036273       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.18563       11.365        15.6529       11.9494       2.46629        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.52391       15.3207       20.4429       16.3465       4.02503        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.37407       12.511        15.7035       13.2011       2.97917        10     
[art]:TriggerResults:TriggerResultInserter        8.0851e-05    0.00010772    0.000214742   9.7367e-05    3.70346e-05      10     
end_path:out1:RootOutput                           1.037e-05    1.69258e-05   4.3531e-05    1.2924e-05    9.39812e-06      10     
end_path:out1:RootOutput(write)                     2.37427       3.3805        3.94152       3.45974      0.440173        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5045.97 MB
  Peak resident set size usage (VmHWM): 4206.69 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.56 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10279848
-rw-r--r-- 1 dune001 lbno 2422344244 Jan 18 18:55 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2303476747 Jan 18 18:53 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2112836210 Jan 18 20:34 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1847239566 Jan 18 16:38 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1763738706 Jan 18 16:23 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   76306580 Jan 18 20:27 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242788 Jan 18 20:34 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 18 20:34 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 18 20:34 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 18 20:34 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 18 18:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 16:38 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 20:34 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 18 16:15 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 18 20:34 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 18 19:05 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 18 18:55 debugprod.log
justIN time: 2026-02-05 18:28:46 UTC       justIN version: 01.06.00