justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271483.1@dunegpschedd02.fnal.gov

Jobsub ID271483.1@dunegpschedd02.fnal.gov
Workflow ID11829
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-13 15:41:29
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2026-01-13 20:09:47
From worker nodeHostnamewn-da-12.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-13 15:42:49
Input filesmonte-carlo-011829-000001
JobscriptExit code0
Real time4h (15919s)
CPU time4h (15728s = 98%)
Max RSS bytes4466548736 (4259 MiB)
Outputting started2026-01-13 20:08:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11829/1/001/reco_stage1.root
Finished2026-01-13 20:09:47
Saved logsjustin-logs:271483.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 181073 traces, 0 summary
Retagger: tagging trace set: gauss with 138492 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 138492 traces tagged "gauss"
FrameSaver: q=4.84491e+06 n=2495367 tag=gauss
wclsFrameSaver: saving 181073 traces tagged "wiener"
FrameSaver: q=5.12249e+06 n=2215946 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 58667 XUs and 63915 XVs -> 1018 XUVs
C:0 T:1 146430 XUs and 168726 XVs -> 11095 XUVs
C:0 T:2 79820 XUs and 138379 XVs -> 5172 XUVs
C:0 T:3 36666 XUs and 51840 XVs -> 362 XUVs
C:0 T:4 14349 XUs and 15694 XVs -> 170 XUVs
C:0 T:5 70957 XUs and 81884 XVs -> 6250 XUVs
C:0 T:6 125554 XUs and 144367 XVs -> 5385 XUVs
C:0 T:7 49244 XUs and 57227 XVs -> 467 XUVs
C:0 T:8 12189 XUs and 13753 XVs -> 255 XUVs
C:0 T:9 95143 XUs and 126890 XVs -> 8806 XUVs
C:0 T:10 50269 XUs and 45092 XVs -> 5922 XUVs
C:0 T:11 115 XUs and 85 XVs -> 12 XUVs
44914 XUVs total
17138 collection wire objects
40074 potential space points
Neighbour search...
2370902 tests to find 1177648 neighbours
Iterating with no regularization...
Begin: 1.22738e+08
0 8.91638e+07
1 8.74094e+07
2 8.71949e+07
3 8.71302e+07
Now with regularization...
Begin: 7.3441e+07
0 7.33866e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411718750
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411718750
fLifetime: 30000 [us]
13-Jan-2026 21:08:07 CET  Closed output file "reco_stage1.root"
13-Jan-2026 21:08:07 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          268.752       503.168       643.343       521.609       105.892        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000880619   0.00516653     0.0081201    0.00551511    0.00249931       10     
reco:ophit:OpHitFinder                             0.308278      0.350381      0.408985      0.349635      0.0336568       10     
reco:opflash:OpFlashFinder                         0.208188      0.268887      0.350115      0.263617      0.0425528       10     
reco:opslicer:OpSlicer                             0.764243      0.916362       1.15198      0.915002      0.123576        10     
reco:caldata:DataPrepModule                         4.97183       5.15677       5.56018       5.11744      0.154443        10     
reco:wclsdatasp:WireCellToolkit                     33.557        45.7494       53.3772       47.5551       5.6963         10     
reco:gaushit:GausHitFinder                         0.983925       1.6518        1.98718       1.75989      0.300146        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000453264   0.000789834    0.0010278    0.000814533   0.000176815      10     
reco:reco3d:SpacePointSolver                        5.66286       17.2744       24.0748       16.8962       5.44897        10     
reco:hitpdune:DisambigFromSpacePoints              0.880532       2.30743       3.06609       2.48348       0.65445        10     
reco:pandora:StandardPandora                        22.9816       60.7207       116.373       60.4621       25.6374        10     
reco:pandoraWriter:StandardPandora                 0.341355      0.527825      0.610704      0.540063      0.0809412       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.07715       6.59611       8.72723       6.95383       1.67468        10     
reco:pandoraShower:LArPandoraShowerCreation         2.43111       8.18749       11.9929       8.70575       2.87061        10     
reco:pandoracalo:Calorimetry                        1.50949       3.2023        4.01711       3.45507      0.733108        10     
reco:pandoracalonosce:Calorimetry                   1.29959       2.98201       3.75242       3.19878      0.747055        10     
reco:pandorapid:Chi2ParticleID                    0.00586246    0.00752505    0.00835956    0.00784156    0.000866496      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0364795     0.0606533     0.0703676     0.0629815     0.0102077       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00573865    0.00713492    0.00821454    0.00754198    0.000840357      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.98992       7.42701       10.0247       8.09662       2.4591         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.91042       7.29457       10.5709       7.89483       2.51078        10     
reco:emtrkmichelid:EmTrackMichelId                  161.684       265.596       309.794       269.862       45.3124        10     
reco:crttag:SingleCRTMatchingProducer              0.0075748     0.0201187     0.0530651     0.017515      0.0118353       10     
reco:crtreco:TwoCRTMatchingProducer               0.00554245     0.0118905     0.0163783     0.0122717    0.00292657       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.548418       1.05974       1.36545       1.07793      0.222144        10     
reco:pandora2Track:LArPandoraTrackCreation          7.18528       21.9262       29.0491       24.9017       6.90409        10     
reco:pandora2calo:Calorimetry                       3.10139       9.42931       12.6031       10.7629       2.78289        10     
reco:pandora2calonosce:Calorimetry                  3.00842       8.96902       12.557        10.0839       2.64616        10     
reco:pandora2pid:Chi2ParticleID                    0.0142648     0.0198088     0.0231636     0.0206883    0.00278604       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0473319     0.0790336     0.0925439     0.0817037     0.0140429       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0131396     0.018722      0.0222156     0.0200237    0.00297687       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.75313       7.41366       9.87709       8.19938       2.17961        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.67458       7.44867       9.61481       8.27354       2.00609        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.36361       6.61908       8.69404       7.19907       1.7919         10     
[art]:TriggerResults:TriggerResultInserter        2.8504e-05    4.21023e-05   0.00013624    3.00425e-05   3.18019e-05      10     
end_path:out1:RootOutput                           5.972e-06    4.10304e-05   0.000345439   6.4975e-06    0.000101484      10     
end_path:out1:RootOutput(write)                     2.21084       3.71161       4.54466       3.82058      0.648612        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5477.83 MB
  Peak resident set size usage (VmHWM): 4466.55 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.27 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12250512
-rw-r--r--. 1 dune009 dune 2795546497 Jan 13 19:41 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2649051790 Jan 13 19:38 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2559915937 Jan 13 21:08 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2262481072 Jan 13 17:07 g4_stage2.root
-rw-r--r--. 1 dune009 dune 2182666731 Jan 13 16:50 g4_stage1.root
-rw-r--r--. 1 dune009 dune   94246040 Jan 13 21:00 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243080 Jan 13 21:08 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 13 21:08 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 13 21:08 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 13 21:08 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 13 19:41 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 17:07 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 13 21:08 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 13 16:42 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 13 21:08 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 13 19:48 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 13 19:41 debugprod.log
justIN time: 2026-02-06 05:56:44 UTC       justIN version: 01.06.00