justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 272044.1@dunegpschedd02.fnal.gov

Jobsub ID272044.1@dunegpschedd02.fnal.gov
Workflow ID11901
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 14:28:39
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-14 20:09:26
From worker nodeHostnamehnode74.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 14:29:15
Input filesmonte-carlo-011901-000001
JobscriptExit code0
Real time5h (20214s)
CPU time3h (11156s = 55%)
Max RSS bytes4455227392 (4248 MiB)
Outputting started2026-01-14 20:06:10
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11901/1/001/reco_stage1.root
Finished2026-01-14 20:09:26
Saved logsjustin-logs:272044.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 137674 traces, 0 summary
Retagger: tagging trace set: gauss with 106156 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106156 traces tagged "gauss"
FrameSaver: q=3.59712e+06 n=1911376 tag=gauss
wclsFrameSaver: saving 137674 traces tagged "wiener"
FrameSaver: q=3.82046e+06 n=1710463 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 35251 XUs and 37970 XVs -> 468 XUVs
C:0 T:1 90254 XUs and 76915 XVs -> 6203 XUVs
C:0 T:2 148381 XUs and 154846 XVs -> 8857 XUVs
C:0 T:3 70793 XUs and 54367 XVs -> 697 XUVs
C:0 T:4 6003 XUs and 5682 XVs -> 58 XUVs
C:0 T:5 15911 XUs and 16211 XVs -> 4693 XUVs
C:0 T:6 119669 XUs and 138216 XVs -> 7417 XUVs
C:0 T:7 42665 XUs and 45433 XVs -> 525 XUVs
C:0 T:8 10388 XUs and 8487 XVs -> 63 XUVs
C:0 T:9 53165 XUs and 63001 XVs -> 3000 XUVs
C:0 T:10 54168 XUs and 41903 XVs -> 4679 XUVs
C:0 T:11 157 XUs and 162 XVs -> 16 XUVs
36676 XUVs total
12147 collection wire objects
33189 potential space points
Neighbour search...
3192599 tests to find 1446652 neighbours
Iterating with no regularization...
Begin: 6.93273e+07
0 5.64809e+07
1 5.56043e+07
2 5.54769e+07
3 5.54388e+07
Now with regularization...
Begin: 4.60565e+07
0 4.60253e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411737773
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411737773
fLifetime: 30000 [us]
14-Jan-2026 21:06:08 CET  Closed output file "reco_stage1.root"
14-Jan-2026 21:06:08 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          289.426       627.532       1077.47       578.913       201.543        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00109099    0.00355735    0.00618963    0.00335432    0.00181251       10     
reco:ophit:OpHitFinder                             0.244553      0.285376      0.398079      0.265287      0.050351        10     
reco:opflash:OpFlashFinder                         0.164566      0.206412      0.253089      0.201329       0.02417        10     
reco:opslicer:OpSlicer                              0.51633      0.703749       1.1661       0.640463      0.190193        10     
reco:caldata:DataPrepModule                         5.68866       6.20234       7.20841       5.81509      0.552676        10     
reco:wclsdatasp:WireCellToolkit                     27.3016       56.9684       161.448       31.3418       51.9135        10     
reco:gaushit:GausHitFinder                         0.868514       1.24785       1.69077       1.18885      0.296643        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000383109   0.000545513   0.000713367   0.000544902   0.000119301      10     
reco:reco3d:SpacePointSolver                        9.57086       28.5415       149.81        16.3355       40.6597        10     
reco:hitpdune:DisambigFromSpacePoints               1.03852       1.8816        3.34992       1.69473       0.65778        10     
reco:pandora:StandardPandora                        21.6036       54.5064       176.154       40.7434       42.0397        10     
reco:pandoraWriter:StandardPandora                 0.313302      0.401306      0.586369      0.386098      0.0917179       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.66885       6.07816       9.2842        5.63617       1.87185        10     
reco:pandoraShower:LArPandoraShowerCreation         3.24487       7.06847       11.1155       7.1794        2.81975        10     
reco:pandoracalo:Calorimetry                        2.05773       3.65827       5.18862       3.56037       1.17781        10     
reco:pandoracalonosce:Calorimetry                   1.94384       15.8075       129.092       3.23953       37.7735        10     
reco:pandorapid:Chi2ParticleID                    0.00422987    0.00585939    0.00980266    0.00516545    0.00171529       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0363443     0.0507848     0.0813175     0.0429855     0.016125        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00376176    0.00555709    0.00943915    0.00485359     0.0017797       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.2129        7.24935       11.8586       7.07578       2.38487        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.31156       6.92563       10.3111       6.80101       1.89799        10     
reco:emtrkmichelid:EmTrackMichelId                  156.235       317.829       573.193       266.055       141.27         10     
reco:crttag:SingleCRTMatchingProducer              0.0128253     0.0234172     0.0487139     0.0193572     0.0106463       10     
reco:crtreco:TwoCRTMatchingProducer               0.00590633     0.0167477     0.031742      0.0159459    0.00741114       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.784653      0.943415       1.13915      0.932281      0.126182        10     
reco:pandora2Track:LArPandoraTrackCreation          7.72627       20.2221       37.9116       19.3488       7.27735        10     
reco:pandora2calo:Calorimetry                       4.65921       22.0102       130.711       9.70231       36.4119        10     
reco:pandora2calonosce:Calorimetry                  4.46606       21.8527       135.274       9.60696       37.9536        10     
reco:pandora2pid:Chi2ParticleID                   0.00893115     0.0139231     0.0214445      0.01324     0.00395726       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0441601     0.0761317     0.175627      0.0631077     0.0367079       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00898123     0.0140497     0.0229024     0.0130615    0.00431639       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.97295       30.4221       129.312       7.34588       48.7728        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.29457       7.13486       10.2399       7.56165       1.93133        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.00475       6.24538       8.7179        6.54007       1.63837        10     
[art]:TriggerResults:TriggerResultInserter        2.7661e-05    4.55701e-05   0.000123873   3.82705e-05   2.70278e-05      10     
end_path:out1:RootOutput                           4.74e-06     6.4151e-06    1.2301e-05     5.735e-06    2.10679e-06      10     
end_path:out1:RootOutput(write)                     1.98702       2.77305       3.49707       2.82794      0.441959        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5274.69 MB
  Peak resident set size usage (VmHWM): 4455.23 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11435304
-rw-r--r-- 1 dune001 dune 2635042323 Jan 14 19:19 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2499150692 Jan 14 19:17 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2360292135 Jan 14 21:06 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2109798073 Jan 14 15:58 g4_stage2.root
-rw-r--r-- 1 dune001 dune 2020469961 Jan 14 15:40 g4_stage1.root
-rw-r--r-- 1 dune001 dune   84369360 Jan 14 20:58 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242929 Jan 14 21:06 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 14 21:06 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 14 21:06 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 14 21:06 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 14 19:19 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 14 15:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 14 21:06 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 14 15:29 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 14 21:06 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 14 19:24 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 14 19:19 debugprod.log
justIN time: 2026-02-06 03:25:00 UTC       justIN version: 01.06.00