justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265609.0@dunegpschedd02.fnal.gov

Jobsub ID265609.0@dunegpschedd02.fnal.gov
Workflow ID11480
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 04:01:15
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2026-01-03 08:08:01
From worker nodeHostnameccwcondor0008
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 04:28:03
Input filesmonte-carlo-011480-000001
JobscriptExit code0
Real time3h (13106s)
CPU time3h (12728s = 97%)
Max RSS bytes4171673600 (3978 MiB)
Outputting started2026-01-03 08:06:30
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11480/1/001/reco_stage1.root
Finished2026-01-03 08:08:01
Saved logsjustin-logs:265609.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 124344 traces, 0 summary
Retagger: tagging trace set: gauss with 93861 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93861 traces tagged "gauss"
FrameSaver: q=3.23422e+06 n=1629774 tag=gauss
wclsFrameSaver: saving 124344 traces tagged "wiener"
FrameSaver: q=3.42375e+06 n=1435640 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27888 XUs and 22580 XVs -> 320 XUVs
C:0 T:1 53226 XUs and 41425 XVs -> 4939 XUVs
C:0 T:2 74438 XUs and 76901 XVs -> 2508 XUVs
C:0 T:3 43054 XUs and 36446 XVs -> 293 XUVs
C:0 T:4 14888 XUs and 15063 XVs -> 172 XUVs
C:0 T:5 58100 XUs and 52696 XVs -> 5239 XUVs
C:0 T:6 65323 XUs and 76140 XVs -> 2869 XUVs
C:0 T:7 30117 XUs and 35050 XVs -> 422 XUVs
C:0 T:8 11166 XUs and 8363 XVs -> 103 XUVs
C:0 T:9 79602 XUs and 82788 XVs -> 7317 XUVs
C:0 T:10 23212 XUs and 12795 XVs -> 4501 XUVs
C:0 T:11 119 XUs and 132 XVs -> 46 XUVs
28729 XUVs total
10607 collection wire objects
26491 potential space points
Neighbour search...
1545651 tests to find 816410 neighbours
Iterating with no regularization...
Begin: 5.57163e+07
0 4.56251e+07
1 4.48301e+07
2 4.47175e+07
3 4.46883e+07
Now with regularization...
Begin: 3.52116e+07
0 3.518e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411507848
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411507848
fLifetime: 30000 [us]
03-Jan-2026 09:06:28 CET  Closed output file "reco_stage1.root"
03-Jan-2026 09:06:28 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          335.869       441.483       565.544       424.612       77.7752        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145084    0.00443189    0.00615715    0.00505174    0.00145562       10     
reco:ophit:OpHitFinder                              0.23787      0.312917      0.407817       0.29853      0.0455846       10     
reco:opflash:OpFlashFinder                         0.133363      0.179413      0.221712      0.183721      0.0292327       10     
reco:opslicer:OpSlicer                             0.487067      0.704472      0.964442      0.719118       0.15151        10     
reco:caldata:DataPrepModule                         5.78651       6.08869       6.49096       6.02343      0.195229        10     
reco:wclsdatasp:WireCellToolkit                     40.0916       42.5333       47.1151       42.1469       2.06799        10     
reco:gaushit:GausHitFinder                          1.13193       1.46122       1.78021       1.39292      0.215365        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000448346   0.000633461   0.000884746   0.000680332   0.000132009      10     
reco:reco3d:SpacePointSolver                        11.8527       21.6832       28.0973       22.5467       5.63472        10     
reco:hitpdune:DisambigFromSpacePoints               1.34714       1.98351       2.71284       1.73105      0.476842        10     
reco:pandora:StandardPandora                        21.955        49.381        89.3704       45.1308       21.3204        10     
reco:pandoraWriter:StandardPandora                 0.457511      0.592278      0.832227      0.562579      0.103926        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.55896       5.8226        9.41392       5.17569       2.01317        10     
reco:pandoraShower:LArPandoraShowerCreation         3.6408        6.89504       11.9016       6.7183        2.61633        10     
reco:pandoracalo:Calorimetry                        2.27192       3.0562        4.4048        2.77881      0.746439        10     
reco:pandoracalonosce:Calorimetry                   1.90611       2.75934       3.98235       2.55608      0.706264        10     
reco:pandorapid:Chi2ParticleID                    0.00490546    0.00649552     0.0103206    0.00610128    0.00144496       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0517135     0.0770432     0.122757      0.0739874     0.0184069       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00386746    0.00587231     0.0073337    0.00576122    0.00111986       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.82519       6.76454       13.1984       5.91009       2.7691         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.95267       6.44193       10.9327       6.00281       2.25761        10     
reco:emtrkmichelid:EmTrackMichelId                  185.878       224.894       272.631       218.666       28.4853        10     
reco:crttag:SingleCRTMatchingProducer              0.012601      0.0205546     0.0331213     0.0192656    0.00647602       10     
reco:crtreco:TwoCRTMatchingProducer               0.00928087     0.0115746     0.0163868     0.0107467    0.00202979       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.832508       1.00882       1.28194       1.00857      0.130718        10     
reco:pandora2Track:LArPandoraTrackCreation          11.9642       18.4534       29.3285       16.6564       5.72451        10     
reco:pandora2calo:Calorimetry                       4.90135       8.83939       13.6083       8.6379        2.59674        10     
reco:pandora2calonosce:Calorimetry                  4.80693       8.0377        12.6046       8.23868       2.36161        10     
reco:pandora2pid:Chi2ParticleID                   0.00717819     0.0142375     0.0197674     0.0152249    0.00396278       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0422378     0.0868278     0.112247      0.0946472     0.0211655       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00709963     0.0142495     0.0180347     0.0147085    0.00320283       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.78406       6.2263        9.15206       6.22953       1.76729        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.92268       7.27803       9.90437       7.2184        1.57304        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.16888       6.34615       9.38493       6.30371       1.69207        10     
[art]:TriggerResults:TriggerResultInserter        4.7101e-05    7.76921e-05   0.000187713    6.955e-05    3.76982e-05      10     
end_path:out1:RootOutput                           7.701e-06    1.12642e-05   2.3456e-05    1.00305e-05   4.28966e-06      10     
end_path:out1:RootOutput(write)                     2.61332       3.30124       4.13197       3.27709      0.519513        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5013.1 MB
  Peak resident set size usage (VmHWM): 4171.67 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10684108
-rw-r--r-- 1 dune001 lbno 2479330982 Jan  3 07:50 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2352996973 Jan  3 07:48 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2205290182 Jan  3 09:06 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1953157695 Jan  3 05:49 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1864108873 Jan  3 05:35 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   85024630 Jan  3 09:02 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243394 Jan  3 09:06 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan  3 09:06 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan  3 09:06 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan  3 09:06 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan  3 07:50 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 05:49 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 09:06 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan  3 05:28 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan  3 09:06 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:59 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan  3 07:50 debugprod.log
justIN time: 2026-02-06 22:22:20 UTC       justIN version: 01.06.00