justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294237.0@dunegpschedd01.fnal.gov

Jobsub ID294237.0@dunegpschedd01.fnal.gov
Workflow ID12108
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-20 14:48:54
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-01-20 19:10:22
From worker nodeHostnamehnode68.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-20 14:50:13
Input filesmonte-carlo-012108-000001
JobscriptExit code0
Real time4h (15537s)
CPU time3h (12272s = 78%)
Max RSS bytes4131332096 (3939 MiB)
Outputting started2026-01-20 19:09:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12108/1/001/reco_stage1.root
Finished2026-01-20 19:10:22
Saved logsjustin-logs:294237.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 126357 traces, 0 summary
Retagger: tagging trace set: gauss with 96769 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96769 traces tagged "gauss"
FrameSaver: q=3.38702e+06 n=1730096 tag=gauss
wclsFrameSaver: saving 126357 traces tagged "wiener"
FrameSaver: q=3.58758e+06 n=1546022 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39343 XUs and 55877 XVs -> 552 XUVs
C:0 T:1 80701 XUs and 99254 XVs -> 6159 XUVs
C:0 T:2 47154 XUs and 104678 XVs -> 5995 XUVs
C:0 T:3 23360 XUs and 44443 XVs -> 201 XUVs
C:0 T:4 11361 XUs and 12329 XVs -> 67 XUVs
C:0 T:5 36714 XUs and 33634 XVs -> 3027 XUVs
C:0 T:6 165062 XUs and 176464 XVs -> 16790 XUVs
C:0 T:7 53288 XUs and 64452 XVs -> 664 XUVs
C:0 T:8 5483 XUs and 4216 XVs -> 54 XUVs
C:0 T:9 26649 XUs and 31807 XVs -> 2311 XUVs
C:0 T:10 16790 XUs and 10857 XVs -> 2115 XUVs
C:0 T:11 91 XUs and 41 XVs -> 11 XUVs
37946 XUVs total
10254 collection wire objects
35100 potential space points
Neighbour search...
4921352 tests to find 2226720 neighbours
Iterating with no regularization...
Begin: 6.4442e+07
0 5.33706e+07
1 5.25505e+07
2 5.24334e+07
3 5.23996e+07
Now with regularization...
Begin: 4.29996e+07
0 4.29564e+07
1 4.29279e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411853275
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411853275
fLifetime: 30000 [us]
20-Jan-2026 20:09:09 CET  Closed output file "reco_stage1.root"
20-Jan-2026 20:09:09 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          300.631       474.263       734.995       478.169       116.359        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00108091    0.00342809    0.00506425    0.00391662    0.00132448       10     
reco:ophit:OpHitFinder                             0.264657      0.338441      0.459346      0.351661      0.0540537       10     
reco:opflash:OpFlashFinder                         0.137392      0.190306      0.218491      0.196994      0.0226224       10     
reco:opslicer:OpSlicer                             0.618362      0.877352       1.15266      0.882361      0.172036        10     
reco:caldata:DataPrepModule                         4.52536       6.77997       10.8123       6.61966       1.51548        10     
reco:wclsdatasp:WireCellToolkit                     36.5243       54.9987       174.779       41.9032       40.0578        10     
reco:gaushit:GausHitFinder                         0.954835       1.57669       2.0596        1.62914      0.302579        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000422322   0.000704947    0.0010571    0.00067996    0.000192461      10     
reco:reco3d:SpacePointSolver                        8.22663       15.054        27.8826       14.2811       5.44374        10     
reco:hitpdune:DisambigFromSpacePoints              0.702685       1.80445       2.72589       1.88098      0.554417        10     
reco:pandora:StandardPandora                        18.5836       45.7541       131.98        38.6925       30.338         10     
reco:pandoraWriter:StandardPandora                 0.264868      0.517106      0.862212      0.480411      0.159867        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.91635       5.91816       12.753        5.68939       2.65266        10     
reco:pandoraShower:LArPandoraShowerCreation         1.30953       5.63316       12.0647       5.52205       2.77857        10     
reco:pandoracalo:Calorimetry                       0.992664       2.84129       6.56693       2.57695       1.4125         10     
reco:pandoracalonosce:Calorimetry                  0.925858       2.63408       6.25182       2.20141       1.36279        10     
reco:pandorapid:Chi2ParticleID                    0.00551492    0.00740391     0.0117547    0.00724758    0.00175282       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0375752     0.0620054     0.0979833     0.0616821     0.017708        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00528199    0.00709463     0.0117148    0.00693519    0.00174528       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.94196       5.4161        13.589        4.6637        3.13528        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.85694       5.19421       12.6009       4.55457       2.84014        10     
reco:emtrkmichelid:EmTrackMichelId                  192.571       264.043       357.561       266.378       44.7949        10     
reco:crttag:SingleCRTMatchingProducer             0.00938752     0.0155987     0.028389      0.0146257    0.00492451       10     
reco:crtreco:TwoCRTMatchingProducer               0.00830819     0.0124492     0.0166046     0.0124714    0.00317685       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.456344      0.945802       1.3221       0.948546      0.222649        10     
reco:pandora2Track:LArPandoraTrackCreation          6.60909       16.3847       29.6261       14.701        6.47047        10     
reco:pandora2calo:Calorimetry                       3.21776       8.6801        20.2303       7.57652       4.64099        10     
reco:pandora2calonosce:Calorimetry                  2.93197       7.84609       15.6809       7.22406       3.6713         10     
reco:pandora2pid:Chi2ParticleID                    0.012523      0.0186961     0.0263823     0.0187725    0.00440672       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0496092     0.087693       0.12623      0.0921108     0.0218057       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.012736      0.0188006     0.0265576     0.0189686    0.00427634       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.59151       5.60715       7.8561        5.2026        1.67479        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.82746       6.08406       10.1879       5.13956       2.29727        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.15177       5.56354       8.50338       5.25156       1.90512        10     
[art]:TriggerResults:TriggerResultInserter        2.4151e-05    4.64463e-05   0.000124213   3.8066e-05    2.77486e-05      10     
end_path:out1:RootOutput                           4.47e-06      6.207e-06     1.021e-05     6.28e-06     1.54839e-06      10     
end_path:out1:RootOutput(write)                      2.373        3.2085        4.09857       3.17583       0.50553        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4949.54 MB
  Peak resident set size usage (VmHWM): 4131.33 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.41 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9912584
-rw-r--r-- 1 dune001 dune 2350892875 Jan 20 18:47 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2234706718 Jan 20 18:44 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2018583650 Jan 20 20:09 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1767270385 Jan 20 16:11 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1707781646 Jan 20 15:58 g4_stage1.root
-rw-r--r-- 1 dune001 dune   70622330 Jan 20 20:02 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242768 Jan 20 20:09 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 20 20:09 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 20 20:09 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 20 20:09 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 20 18:47 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 20 16:11 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 20 20:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 20 15:50 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 20 20:09 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 20 18:55 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 20 18:47 debugprod.log
justIN time: 2026-02-04 07:13:36 UTC       justIN version: 01.06.00