justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252556.0@dunegpschedd01.fnal.gov

Jobsub ID252556.0@dunegpschedd01.fnal.gov
Workflow ID10241
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 04:28:31
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-11-16 06:28:29
From worker nodeHostnamedunegli-47944-0-cmswn4057.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 04:29:43
Input filesmonte-carlo-010241-000001
JobscriptExit code0
Real time1h (7104s)
CPU time1h (7062s = 99%)
Max RSS bytes4088438784 (3899 MiB)
Outputting started2025-11-16 06:28:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10241/1/001/reco_stage1.root
Finished2025-11-16 06:28:29
Saved logsjustin-logs:252556.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

epare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 118313 traces, 0 summary
Retagger: tagging trace set: gauss with 91288 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91288 traces tagged "gauss"
FrameSaver: q=3.3197e+06 n=1642309 tag=gauss
wclsFrameSaver: saving 118313 traces tagged "wiener"
FrameSaver: q=3.52764e+06 n=1469188 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31124 XUs and 34737 XVs -> 277 XUVs
C:0 T:1 81464 XUs and 77910 XVs -> 6549 XUVs
C:0 T:2 101537 XUs and 124815 XVs -> 5245 XUVs
C:0 T:3 53209 XUs and 52696 XVs -> 580 XUVs
C:0 T:4 9145 XUs and 11142 XVs -> 88 XUVs
C:0 T:5 34115 XUs and 48339 XVs -> 7915 XUVs
C:0 T:6 62390 XUs and 77941 XVs -> 3130 XUVs
C:0 T:7 28117 XUs and 33768 XVs -> 301 XUVs
C:0 T:8 9574 XUs and 9125 XVs -> 138 XUVs
C:0 T:9 60096 XUs and 83755 XVs -> 8830 XUVs
C:0 T:10 30986 XUs and 32458 XVs -> 5134 XUVs
C:0 T:11 76 XUs and 32 XVs -> 3 XUVs
38190 XUVs total
10825 collection wire objects
35939 potential space points
Neighbour search...
5152627 tests to find 2100662 neighbours
Iterating with no regularization...
Begin: 9.47369e+07
0 8.01018e+07
1 7.87449e+07
2 7.85205e+07
3 7.84561e+07
Now with regularization...
Begin: 6.53983e+07
0 6.52672e+07
1 6.5224e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410542605
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410542605
fLifetime: 30000 [us]
16-Nov-2025 06:28:07 UTC  Closed output file "reco_stage1.root"
16-Nov-2025 06:28:07 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          137.985       204.967       337.162       194.067       57.2746        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00102274    0.00283095    0.00430151    0.00301651    0.00112048       10     
reco:ophit:OpHitFinder                             0.172509       0.19725       0.23193      0.184898      0.0228286       10     
reco:opflash:OpFlashFinder                         0.0920335     0.122089      0.170339      0.114486      0.0256837       10     
reco:opslicer:OpSlicer                              0.34472      0.434629      0.554135      0.414947      0.0709056       10     
reco:caldata:DataPrepModule                         3.31468       3.41033       3.66886       3.40561      0.101264        10     
reco:wclsdatasp:WireCellToolkit                     21.5659       22.2974       23.6323       22.0836      0.566716        10     
reco:gaushit:GausHitFinder                         0.520542      0.761148       1.17847      0.726784      0.180435        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000247557   0.000411551   0.000659736   0.000409754   0.00012235       10     
reco:reco3d:SpacePointSolver                        5.43056       8.94969       14.3262       7.68665       3.09164        10     
reco:hitpdune:DisambigFromSpacePoints              0.477091       0.97846       1.82615      0.859891      0.396028        10     
reco:pandora:StandardPandora                        8.46661       22.5022       71.1059       19.9215       17.349         10     
reco:pandoraWriter:StandardPandora                 0.162381      0.251409      0.365223       0.2403       0.0602575       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.17104       2.00266       2.92785        1.858       0.662123        10     
reco:pandoraShower:LArPandoraShowerCreation         1.10663       2.24599       4.44886       2.07351       1.02686        10     
reco:pandoracalo:Calorimetry                       0.690144       1.19909       1.72725       1.16009      0.389202        10     
reco:pandoracalonosce:Calorimetry                  0.619835       1.10607       1.60734       1.06389      0.369898        10     
reco:pandorapid:Chi2ParticleID                    0.00263257    0.00344719    0.00517432    0.00330341    0.000762753      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0218641     0.0314171     0.0430137     0.0296262    0.00620602       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00248629    0.00305786     0.0038418    0.00288449    0.000506928      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.12002       2.51644       5.02228       2.3858        1.21382        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.08612       2.4608        4.92391       2.33197       1.19325        10     
reco:emtrkmichelid:EmTrackMichelId                  78.6558       109.229       156.324        103.5        23.2007        10     
reco:crttag:SingleCRTMatchingProducer             0.00527452    0.00832053     0.0151682    0.00813339    0.00282556       10     
reco:crtreco:TwoCRTMatchingProducer                0.0033295    0.00462943    0.00626278    0.00427757    0.00100056       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.352811      0.533215      0.770528      0.514696      0.149061        10     
reco:pandora2Track:LArPandoraTrackCreation          3.37814       6.5383        12.3248       6.2215        2.78885        10     
reco:pandora2calo:Calorimetry                       1.73983       3.53992       6.44735       3.46414       1.4978         10     
reco:pandora2calonosce:Calorimetry                  1.64203       3.37239       6.14168       3.29095       1.43595        10     
reco:pandora2pid:Chi2ParticleID                   0.00586459    0.00788997     0.0108559    0.00779903    0.00181498       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0283717     0.0385105     0.0530432     0.0367672    0.00821609       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00576094     0.0076321     0.0105126    0.00759642    0.00172178       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.31296       2.39847       4.20745       2.26353      0.909904        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.52509       3.09058       5.54341       2.99451       1.28409        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.41471       2.86043       5.11717       2.76462       1.1808         10     
[art]:TriggerResults:TriggerResultInserter        3.2632e-05    5.03902e-05   0.000179068   3.4891e-05    4.30577e-05      10     
end_path:out1:RootOutput                           7.303e-06    1.13501e-05   1.9487e-05    1.15765e-05   3.24136e-06      10     
end_path:out1:RootOutput(write)                     1.27161       1.81154       2.58538       1.71539      0.395941        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4903.8 MB
  Peak resident set size usage (VmHWM): 4088.44 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9860312
-rw-r--r-- 1 dunegli fnalgrid 2323598839 Nov 16 05:52 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2206611394 Nov 16 05:51 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2010038088 Nov 16 06:28 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1776468173 Nov 16 04:39 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1704166426 Nov 16 04:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   75266310 Nov 16 06:25 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242711 Nov 16 06:28 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Nov 16 06:28 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Nov 16 06:28 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Nov 16 06:28 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 05:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 04:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 06:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Nov 16 04:29 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Nov 16 06:28 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:56 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:52 debugprod.log
justIN time: 2026-02-09 10:52:27 UTC       justIN version: 01.06.00