justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 282174.0@dunegpschedd01.fnal.gov

Jobsub ID282174.0@dunegpschedd01.fnal.gov
Workflow ID12099
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-19 14:19:12
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-19 20:25:06
From worker nodeHostnametds413.pic.es
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-19 14:20:28
Input filesmonte-carlo-012099-000001
JobscriptExit code0
Real time6h (21821s)
CPU time5h (20819s = 95%)
Max RSS bytes4505505792 (4296 MiB)
Outputting started2026-01-19 20:24:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12099/1/001/reco_stage1.root
Finished2026-01-19 20:25:06
Saved logsjustin-logs:282174.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150901 traces, 0 summary
Retagger: tagging trace set: gauss with 119718 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119718 traces tagged "gauss"
FrameSaver: q=4.11378e+06 n=2231051 tag=gauss
wclsFrameSaver: saving 150901 traces tagged "wiener"
FrameSaver: q=4.34594e+06 n=2020191 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41132 XUs and 40469 XVs -> 417 XUVs
C:0 T:1 83260 XUs and 79382 XVs -> 8398 XUVs
C:0 T:2 91830 XUs and 104262 XVs -> 4308 XUVs
C:0 T:3 46969 XUs and 44043 XVs -> 372 XUVs
C:0 T:4 12575 XUs and 16601 XVs -> 120 XUVs
C:0 T:5 43950 XUs and 43463 XVs -> 4381 XUVs
C:0 T:6 127848 XUs and 132390 XVs -> 9532 XUVs
C:0 T:7 41283 XUs and 44202 XVs -> 516 XUVs
C:0 T:8 11661 XUs and 13507 XVs -> 193 XUVs
C:0 T:9 67999 XUs and 90214 XVs -> 6054 XUVs
C:0 T:10 66804 XUs and 63261 XVs -> 8251 XUVs
C:0 T:11 238 XUs and 192 XVs -> 21 XUVs
42563 XUVs total
13920 collection wire objects
39349 potential space points
Neighbour search...
3341793 tests to find 1744410 neighbours
Iterating with no regularization...
Begin: 7.1736e+07
0 6.06027e+07
1 5.96049e+07
2 5.94362e+07
3 5.93822e+07
Now with regularization...
Begin: 4.86016e+07
0 4.85618e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411838264
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411838264
fLifetime: 30000 [us]
19-Jan-2026 21:24:06 CET  Closed output file "reco_stage1.root"
19-Jan-2026 21:24:06 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          676.621       810.451       961.948       796.204       95.4125        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0015887    0.00488328    0.00934055    0.00486294    0.00236866       10     
reco:ophit:OpHitFinder                             0.348429      0.465879      0.538623      0.467622      0.0487336       10     
reco:opflash:OpFlashFinder                         0.244138      0.307135      0.420498      0.304452      0.0448778       10     
reco:opslicer:OpSlicer                             0.915658       1.35167       1.64006       1.37966      0.187621        10     
reco:caldata:DataPrepModule                         5.60135       7.82085       8.44423       8.05473      0.766186        10     
reco:wclsdatasp:WireCellToolkit                     45.5144       52.8286       57.7914       54.4081       4.12456        10     
reco:gaushit:GausHitFinder                          1.51394       2.48706       2.90364       2.53508      0.410859        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000687791   0.00104151    0.00139047    0.00107504    0.000215178      10     
reco:reco3d:SpacePointSolver                        14.5625       25.3751       38.5608       24.7469       6.75391        10     
reco:hitpdune:DisambigFromSpacePoints               2.20514       3.76813       5.17278       3.8388       0.896825        10     
reco:pandora:StandardPandora                        66.7845       101.879       202.268       91.4774       38.5859        10     
reco:pandoraWriter:StandardPandora                 0.744318      0.876376      0.971584      0.876116      0.0619869       10     
reco:pandoraTrack:LArPandoraTrackCreation           8.40216        11.87        14.9482       12.0853       2.10926        10     
reco:pandoraShower:LArPandoraShowerCreation         8.77095       13.5209       17.2862       13.1673       2.90104        10     
reco:pandoracalo:Calorimetry                        4.40109       6.16992       8.49506       6.27011       1.42349        10     
reco:pandoracalonosce:Calorimetry                   4.18805       5.77034       7.12471       5.71321       1.01298        10     
reco:pandorapid:Chi2ParticleID                    0.00857738     0.0120598     0.014427      0.0126767    0.00173103       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0662345     0.115113      0.138105      0.119926      0.0197835       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00823987     0.0115079     0.0140733     0.0116342    0.00153264       10     
reco:pandoraShowercalo:ShowerCalorimetry            9.3381        14.6078       21.2457        15.51        3.46492        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.1143        14.3039       19.8483       14.849        3.08441        10     
reco:emtrkmichelid:EmTrackMichelId                  377.544       430.274       500.969       433.59        33.6379        10     
reco:crttag:SingleCRTMatchingProducer              0.0225401     0.030771      0.0396966     0.0311777    0.00555815       10     
reco:crtreco:TwoCRTMatchingProducer                0.0148202     0.0219349     0.0281248     0.0220638    0.00374119       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.14699       1.51002       1.77141       1.50292      0.156151        10     
reco:pandora2Track:LArPandoraTrackCreation          24.0888       36.5005       51.1853       35.0557       7.72993        10     
reco:pandora2calo:Calorimetry                       13.8361       17.9615       23.2447       18.3104       2.8705         10     
reco:pandora2calonosce:Calorimetry                  11.0391       16.4548       23.5036       15.9333       3.68945        10     
reco:pandora2pid:Chi2ParticleID                    0.0193128     0.027772      0.0391869     0.0266797    0.00633863       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.089887      0.135917      0.185758      0.136181      0.030383        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0179906     0.0277481     0.0377603     0.0284734    0.00630644       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.22241       12.1234       16.1486       11.6656       2.02475        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.56002       13.9352       18.2135       13.2876       2.68397        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.76549       12.2613       15.8844       12.0873       2.16176        10     
[art]:TriggerResults:TriggerResultInserter        6.3921e-05    8.44926e-05   0.000170433   7.64665e-05   2.91949e-05      10     
end_path:out1:RootOutput                           1.422e-05    1.99182e-05   5.3181e-05     1.668e-05    1.11708e-05      10     
end_path:out1:RootOutput(write)                     4.42014       5.4262        6.15518       5.61505      0.539148        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5341.75 MB
  Peak resident set size usage (VmHWM): 4505.51 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.96 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11658256
-rw-r--r-- 1 dune001 dune 2680786324 Jan 19 19:05 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2539990284 Jan 19 19:01 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2446654228 Jan 19 21:24 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2123396834 Jan 19 15:56 g4_stage2.root
-rw-r--r-- 1 dune001 dune 2052210376 Jan 19 15:31 g4_stage1.root
-rw-r--r-- 1 dune001 dune   94381450 Jan 19 21:13 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243001 Jan 19 21:24 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 19 21:24 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 19 21:24 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 19 21:24 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 19 19:05 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 19 15:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 19 21:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 19 15:20 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 19 21:24 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 19 19:17 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 19 19:05 debugprod.log
justIN time: 2026-02-05 17:08:42 UTC       justIN version: 01.06.00