justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255200.1@dunegpschedd01.fnal.gov

Jobsub ID255200.1@dunegpschedd01.fnal.gov
Workflow ID10507
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-22 03:26:07
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-11-22 07:10:16
From worker nodeHostnametds428.pic.es
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 03:27:09
Input filesmonte-carlo-010507-000001
JobscriptExit code0
Real time3h (13342s)
CPU time3h (12779s = 95%)
Max RSS bytes3978735616 (3794 MiB)
Outputting started2025-11-22 07:09:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10507/1/001/reco_stage1.root
Finished2025-11-22 07:10:16
Saved logsjustin-logs:255200.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 147081 traces, 0 summary
Retagger: tagging trace set: gauss with 107528 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107528 traces tagged "gauss"
FrameSaver: q=3.62216e+06 n=1817660 tag=gauss
wclsFrameSaver: saving 147081 traces tagged "wiener"
FrameSaver: q=3.84946e+06 n=1572189 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 45100 XUs and 32134 XVs -> 881 XUVs
C:0 T:1 132346 XUs and 96170 XVs -> 17709 XUVs
C:0 T:2 67848 XUs and 67180 XVs -> 5706 XUVs
C:0 T:3 26880 XUs and 24098 XVs -> 107 XUVs
C:0 T:4 18860 XUs and 14375 XVs -> 146 XUVs
C:0 T:5 76353 XUs and 48908 XVs -> 4961 XUVs
C:0 T:6 71654 XUs and 65171 XVs -> 6372 XUVs
C:0 T:7 24265 XUs and 23470 XVs -> 260 XUVs
C:0 T:8 8844 XUs and 8823 XVs -> 89 XUVs
C:0 T:9 61535 XUs and 78412 XVs -> 6561 XUVs
C:0 T:10 65985 XUs and 42824 XVs -> 5999 XUVs
C:0 T:11 717 XUs and 428 XVs -> 61 XUVs
48852 XUVs total
15066 collection wire objects
45149 potential space points
Neighbour search...
6968149 tests to find 3200520 neighbours
Iterating with no regularization...
Begin: 7.17125e+07
0 6.10915e+07
1 6.04119e+07
2 6.03037e+07
3 6.027e+07
Now with regularization...
Begin: 4.91309e+07
0 4.90825e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410662432
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410662432
fLifetime: 30000 [us]
22-Nov-2025 08:09:30 CET  Closed output file "reco_stage1.root"
22-Nov-2025 08:09:30 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          349.143       439.003       545.666       421.615        71.99         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00151549    0.00397217    0.00704668    0.00392078    0.00175447       10     
reco:ophit:OpHitFinder                             0.246683      0.308869      0.418736       0.29542      0.0486441       10     
reco:opflash:OpFlashFinder                         0.123034      0.166709      0.232455      0.155409      0.0328671       10     
reco:opslicer:OpSlicer                             0.516678      0.709488      0.992713       0.7056       0.130431        10     
reco:caldata:DataPrepModule                         5.6226        6.41251       7.45256       6.20177      0.596996        10     
reco:wclsdatasp:WireCellToolkit                     39.2211       42.9635       49.7848       41.0683       3.46684        10     
reco:gaushit:GausHitFinder                          1.01401       1.33902       1.72245       1.29399      0.218816        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000505825   0.000680097   0.00118121    0.000613426   0.000181814      10     
reco:reco3d:SpacePointSolver                        5.88823       16.7848       35.0303       12.9117       9.30802        10     
reco:hitpdune:DisambigFromSpacePoints              0.937933       1.69154       2.70697       1.65081      0.516965        10     
reco:pandora:StandardPandora                        21.6903       31.5093       44.194        30.7264       6.42863        10     
reco:pandoraWriter:StandardPandora                 0.385232      0.500033      0.728133       0.4947       0.0991697       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.84653       4.84155       7.28811       4.8124        1.4097         10     
reco:pandoraShower:LArPandoraShowerCreation         2.46197       5.51798       9.37315       5.02737       2.10723        10     
reco:pandoracalo:Calorimetry                        1.36555       2.36308       3.37654       2.42588      0.696361        10     
reco:pandoracalonosce:Calorimetry                   1.21957       2.24785       3.71449       1.98403      0.827456        10     
reco:pandorapid:Chi2ParticleID                    0.00477724    0.00666674     0.0109692    0.00554433    0.00226992       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0454084     0.0621862     0.0958476     0.0531858     0.0178419       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00479465    0.00650683    0.00975665    0.00564709    0.00184912       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.00613       5.1522        7.86282       5.38657       1.6945         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.98996       4.90482       8.8605        4.70277       1.95197        10     
reco:emtrkmichelid:EmTrackMichelId                  212.349       258.419       319.134       247.629       37.7785        10     
reco:crttag:SingleCRTMatchingProducer              0.0110134     0.0182785     0.029538      0.0174971    0.00506123       10     
reco:crtreco:TwoCRTMatchingProducer               0.00838265     0.0134859     0.0180048     0.0132324    0.00279036       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.772657       1.05458       1.48197       1.0291        0.20147        10     
reco:pandora2Track:LArPandoraTrackCreation          8.6243        15.7152       24.1241       14.1694       4.79305        10     
reco:pandora2calo:Calorimetry                       4.5404        7.5981        15.5388       6.33264       3.35513        10     
reco:pandora2calonosce:Calorimetry                  3.60556       6.80081       10.2282       6.65851       2.18357        10     
reco:pandora2pid:Chi2ParticleID                    0.0102445     0.0150289     0.0195668     0.0148195    0.00245606       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0547843     0.0720218      0.10009      0.070122      0.0140596       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0103697     0.0148068     0.0193834     0.0145175     0.0024259       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.14307       5.91316       8.80343       5.36133       1.66373        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.39507       6.54224       10.0402       5.6189        2.39244        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.25636       5.86396       9.40199       5.71704       2.09918        10     
[art]:TriggerResults:TriggerResultInserter         4.801e-05    6.80867e-05   0.000157041   6.0336e-05    3.04476e-05      10     
end_path:out1:RootOutput                           1.022e-05    1.23733e-05   1.8381e-05    1.1915e-05    2.29658e-06      10     
end_path:out1:RootOutput(write)                     2.47673       3.29852       4.01832       3.42176      0.490944        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4834.53 MB
  Peak resident set size usage (VmHWM): 3978.74 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9336676
-rw-r--r-- 1 dune001 dune 2215392975 Nov 22 06:53 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2106973371 Nov 22 06:50 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1876294806 Nov 22 08:09 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1674021538 Nov 22 04:47 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1618597619 Nov 22 04:35 g4_stage1.root
-rw-r--r-- 1 dune001 dune   68846710 Nov 22 08:02 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242843 Nov 22 08:09 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Nov 22 08:09 mem.db
-rw-r--r-- 1 dune001 dune      62376 Nov 22 08:09 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Nov 22 08:09 time.db
-rw-r--r-- 1 dune001 dune        519 Nov 22 06:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov 22 04:47 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov 22 08:09 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Nov 22 04:27 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Nov 22 08:09 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Nov 22 07:01 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Nov 22 06:53 debugprod.log
justIN time: 2026-02-08 22:54:31 UTC       justIN version: 01.06.00