justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275792.1@dunegpschedd01.fnal.gov

Jobsub ID275792.1@dunegpschedd01.fnal.gov
Workflow ID11743
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 08:41:55
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2026-01-12 12:17:46
From worker nodeHostnamewn-da-17.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 08:42:46
Input filesmonte-carlo-011743-000001
JobscriptExit code0
Real time3h (12810s)
CPU time3h (12674s = 98%)
Max RSS bytes4352217088 (4150 MiB)
Outputting started2026-01-12 12:16:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11743/1/001/reco_stage1.root
Finished2026-01-12 12:17:46
Saved logsjustin-logs:275792.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 184121 traces, 0 summary
Retagger: tagging trace set: gauss with 133199 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 133199 traces tagged "gauss"
FrameSaver: q=4.67526e+06 n=2318115 tag=gauss
wclsFrameSaver: saving 184121 traces tagged "wiener"
FrameSaver: q=4.98108e+06 n=2019819 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26398 XUs and 32383 XVs -> 373 XUVs
C:0 T:1 61176 XUs and 66195 XVs -> 7632 XUVs
C:0 T:2 163530 XUs and 207165 XVs -> 10955 XUVs
C:0 T:3 70426 XUs and 75448 XVs -> 742 XUVs
C:0 T:4 26021 XUs and 19242 XVs -> 224 XUVs
C:0 T:5 135319 XUs and 92615 XVs -> 13217 XUVs
C:0 T:6 179447 XUs and 165028 XVs -> 11382 XUVs
C:0 T:7 60271 XUs and 54060 XVs -> 881 XUVs
C:0 T:8 9482 XUs and 9093 XVs -> 109 XUVs
C:0 T:9 73656 XUs and 105377 XVs -> 10817 XUVs
C:0 T:10 35147 XUs and 46140 XVs -> 7488 XUVs
C:0 T:11 106 XUs and 96 XVs -> 14 XUVs
63834 XUVs total
17829 collection wire objects
57815 potential space points
Neighbour search...
6343539 tests to find 3097224 neighbours
Iterating with no regularization...
Begin: 8.87417e+07
0 7.35036e+07
1 7.22549e+07
2 7.20609e+07
3 7.19977e+07
Now with regularization...
Begin: 5.84496e+07
0 5.83787e+07
1 5.83332e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411692761
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411692761
fLifetime: 30000 [us]
12-Jan-2026 13:16:15 CET  Closed output file "reco_stage1.root"
12-Jan-2026 13:16:15 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          338.011       442.269       601.48        429.037       76.2007        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000781548   0.00506917    0.00868555    0.00550792    0.00264362       10     
reco:ophit:OpHitFinder                             0.236069      0.281286      0.304564      0.285821      0.0200267       10     
reco:opflash:OpFlashFinder                         0.141416       0.19659      0.220121      0.206974      0.0267478       10     
reco:opslicer:OpSlicer                             0.581131      0.676922      0.774836      0.684006      0.074402        10     
reco:caldata:DataPrepModule                         4.52367       4.68592       4.81922       4.70676      0.0863265       10     
reco:wclsdatasp:WireCellToolkit                     33.0969       60.5681       72.8342       65.5249       13.9935        10     
reco:gaushit:GausHitFinder                         0.974354       1.24674       1.61192       1.21166      0.209378        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000569279   0.00078409    0.000898454   0.000807544    9.526e-05       10     
reco:reco3d:SpacePointSolver                        8.31874       14.4912       23.3337       14.4015       4.22003        10     
reco:hitpdune:DisambigFromSpacePoints               1.06397       1.5986        2.2681         1.594       0.402939        10     
reco:pandora:StandardPandora                        22.6801       47.0395       105.908       44.4587       21.518         10     
reco:pandoraWriter:StandardPandora                 0.360122       0.45776      0.551578      0.454128      0.0563311       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.84312       6.41087       9.45516       5.98296       1.4432         10     
reco:pandoraShower:LArPandoraShowerCreation         3.87159       7.51823       10.8482       7.47941       2.50022        10     
reco:pandoracalo:Calorimetry                        2.15708       3.35401       5.39097       2.89765      0.898441        10     
reco:pandoracalonosce:Calorimetry                   2.00026       3.18699       5.32525       2.73819      0.912747        10     
reco:pandorapid:Chi2ParticleID                    0.00533634    0.00633046    0.00839942    0.00627576    0.000813904      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.038592      0.0504017     0.0628832     0.0496848    0.00773075       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00504725    0.00617576    0.00697269    0.00620113    0.000559402      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.39005       7.19581       11.4287       6.01265       2.72341        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.38521       7.07545       11.3536       5.97217       2.60504        10     
reco:emtrkmichelid:EmTrackMichelId                  171.142       211.535       271.819       210.711       31.1153        10     
reco:crttag:SingleCRTMatchingProducer              0.0097897     0.0145673     0.0267336     0.0118765    0.00579714       10     
reco:crtreco:TwoCRTMatchingProducer               0.00674612     0.0126028     0.0246428     0.0103834    0.00613628       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.571541      0.830219       1.16751      0.803597      0.179434        10     
reco:pandora2Track:LArPandoraTrackCreation          10.197        19.6171       37.6474       17.1271       8.54806        10     
reco:pandora2calo:Calorimetry                       4.4894        9.7259        19.9875       8.07018        4.792         10     
reco:pandora2calonosce:Calorimetry                  4.17685       9.22601       19.0635       7.50448       4.67582        10     
reco:pandora2pid:Chi2ParticleID                    0.0123075     0.0152352     0.0180477     0.0151697    0.00175675       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0538861     0.0666372     0.0873402     0.0653483     0.0108533       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0121462     0.015134      0.0180394     0.0150449     0.0018239       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.74084       7.12373       13.8487       5.75908       3.23621        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.66479       7.9425        16.4211       6.23436       4.14371        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.11198       6.89461       14.3197       5.28597       3.65504        10     
[art]:TriggerResults:TriggerResultInserter        3.1481e-05    4.33307e-05   9.4983e-05    3.5208e-05    1.83432e-05      10     
end_path:out1:RootOutput                           5.862e-06    6.9453e-06     9.919e-06     6.487e-06    1.14412e-06      10     
end_path:out1:RootOutput(write)                     2.32181       3.01839       3.98044       3.02805      0.505929        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5154.32 MB
  Peak resident set size usage (VmHWM): 4352.22 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.91 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11027044
-rw-r--r--. 1 dune009 dune 2567343213 Jan 12 12:00 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2437202248 Jan 12 11:58 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2282352273 Jan 12 13:16 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1999156065 Jan 12 10:01 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1923017977 Jan 12 09:49 g4_stage1.root
-rw-r--r--. 1 dune009 dune   82007770 Jan 12 13:08 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243141 Jan 12 13:16 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 12 13:16 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 12 13:16 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 12 13:16 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 12 12:00 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 12 10:01 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 12 13:16 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 12 09:42 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 12 13:16 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 12 12:07 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 12 12:00 debugprod.log
justIN time: 2026-02-06 10:22:30 UTC       justIN version: 01.06.00