justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266070.0@dunegpschedd02.fnal.gov

Jobsub ID266070.0@dunegpschedd02.fnal.gov
Workflow ID11526
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-05 08:00:05
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2026-01-05 13:28:39
From worker nodeHostnamehd71.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-05 08:01:27
Input filesmonte-carlo-011526-000001
JobscriptExit code0
Real time5h (19539s)
CPU time5h (19019s = 97%)
Max RSS bytes4153380864 (3960 MiB)
Outputting started2026-01-05 13:27:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11526/1/001/reco_stage1.root
Finished2026-01-05 13:28:39
Saved logsjustin-logs:266070.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

SignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112294 traces, 0 summary
Retagger: tagging trace set: gauss with 86778 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 86778 traces tagged "gauss"
FrameSaver: q=2.96637e+06 n=1588226 tag=gauss
wclsFrameSaver: saving 112294 traces tagged "wiener"
FrameSaver: q=3.14125e+06 n=1424206 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29564 XUs and 39214 XVs -> 398 XUVs
C:0 T:1 53512 XUs and 68634 XVs -> 5601 XUVs
C:0 T:2 137119 XUs and 106347 XVs -> 5255 XUVs
C:0 T:3 67549 XUs and 42997 XVs -> 418 XUVs
C:0 T:4 6483 XUs and 7380 XVs -> 64 XUVs
C:0 T:5 19246 XUs and 18808 XVs -> 1800 XUVs
C:0 T:6 97727 XUs and 68584 XVs -> 5240 XUVs
C:0 T:7 38249 XUs and 25958 XVs -> 327 XUVs
C:0 T:8 6915 XUs and 5845 XVs -> 32 XUVs
C:0 T:9 44818 XUs and 47194 XVs -> 3362 XUVs
C:0 T:10 80296 XUs and 39097 XVs -> 10789 XUVs
C:0 T:11 942 XUs and 538 XVs -> 121 XUVs
33407 XUVs total
10653 collection wire objects
31190 potential space points
Neighbour search...
4136644 tests to find 1893456 neighbours
Iterating with no regularization...
Begin: 5.74626e+07
0 4.84442e+07
1 4.79e+07
2 4.78255e+07
3 4.78026e+07
Now with regularization...
Begin: 4.03682e+07
0 4.03362e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411551397
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411551397
fLifetime: 30000 [us]
05-Jan-2026 13:27:05 UTC  Closed output file "reco_stage1.root"
05-Jan-2026 13:27:05 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          390.454       544.842       616.512       573.015       65.4248        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00129733    0.00467952    0.00724943    0.00479867    0.00193033       10     
reco:ophit:OpHitFinder                              0.34411      0.367858      0.423313      0.360058      0.0231406       10     
reco:opflash:OpFlashFinder                         0.195271      0.230201      0.264144      0.239079      0.0227408       10     
reco:opslicer:OpSlicer                             0.918365       1.09264       1.35082       1.08106      0.116075        10     
reco:caldata:DataPrepModule                         7.17951       7.43281       7.91036       7.36939      0.222432        10     
reco:wclsdatasp:WireCellToolkit                     51.833        55.2237       61.7583       55.102        2.76162        10     
reco:gaushit:GausHitFinder                          1.49292       1.96276       2.2014        1.9795       0.198965        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000561871   0.000734791   0.00111606    0.000649793   0.000174679      10     
reco:reco3d:SpacePointSolver                        14.6641       18.518        27.0877       17.7958       3.20961        10     
reco:hitpdune:DisambigFromSpacePoints               1.85887       2.52856       2.97928       2.67779      0.348927        10     
reco:pandora:StandardPandora                        27.061        54.5331       81.3734       57.2622       14.3193        10     
reco:pandoraWriter:StandardPandora                 0.501581       0.62346      0.692146      0.627929      0.0515846       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.42743        6.613        8.03625        6.944        1.26683        10     
reco:pandoraShower:LArPandoraShowerCreation         4.15741       8.32041       10.7338       8.71981       2.03185        10     
reco:pandoracalo:Calorimetry                        2.01977       3.69238       4.56152       3.95192      0.736167        10     
reco:pandoracalonosce:Calorimetry                   1.85197       3.39659       4.04527       3.66202      0.666418        10     
reco:pandorapid:Chi2ParticleID                    0.00613791    0.00854402     0.0104203    0.00866804    0.00117413       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0721798     0.0874543     0.0945586     0.0873909    0.00611855       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00627414    0.00799608    0.00884407    0.00821498    0.000727146      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.95832       8.46788       12.3793       8.27861       2.39216        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.65769       8.00853       11.678        8.34518       2.25824        10     
reco:emtrkmichelid:EmTrackMichelId                  223.858       288.088       329.103       294.044       28.8268        10     
reco:crttag:SingleCRTMatchingProducer              0.0153388     0.0207384     0.0257608     0.020618     0.00272261       10     
reco:crtreco:TwoCRTMatchingProducer                0.0119337     0.0153857     0.0188487     0.0150875    0.00193042       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.1044        1.26327       1.55285       1.22631      0.138134        10     
reco:pandora2Track:LArPandoraTrackCreation          12.0546       22.4415       28.5825       23.7174       4.57389        10     
reco:pandora2calo:Calorimetry                       5.75407       11.1666       13.8525       11.9307       2.46262        10     
reco:pandora2calonosce:Calorimetry                  4.96863       10.5325       13.3916       11.4232       2.48369        10     
reco:pandora2pid:Chi2ParticleID                    0.015497      0.0220108     0.0264222     0.0220668    0.00284926       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0942501     0.115117      0.142865      0.117506      0.0124098       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0152931     0.0217477     0.0267091     0.0219873    0.00303478       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.46839       8.27743       10.0269       8.87283       1.81802        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.60366       9.23739       12.2405       9.54433       2.26663        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.15938       8.35053       12.7934       8.51226       2.4019         10     
[art]:TriggerResults:TriggerResultInserter        6.5765e-05    9.44824e-05   0.000196821   7.81175e-05   3.8874e-05       10     
end_path:out1:RootOutput                           1.088e-05    1.27549e-05   1.7894e-05    1.2213e-05    1.93562e-06      10     
end_path:out1:RootOutput(write)                     3.28856       4.00809       4.44615       4.10797      0.352127        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4968.14 MB
  Peak resident set size usage (VmHWM): 4153.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.65 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10249804
-rw-r--r-- 1 dnpil001 users 2403918161 Jan  5 11:53 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2283403903 Jan  5 11:50 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2093797153 Jan  5 13:27 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1856910776 Jan  5 08:27 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1781251132 Jan  5 08:09 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   75840290 Jan  5 13:20 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     242632 Jan  5 13:27 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Jan  5 13:27 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Jan  5 13:27 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Jan  5 13:27 time.db
-rw-r--r-- 1 dnpil001 users        519 Jan  5 11:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  5 08:27 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Jan  5 13:27 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Jan  5 08:01 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Jan  5 13:27 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Jan  5 12:01 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Jan  5 11:53 debugprod.log
justIN time: 2026-02-06 19:43:53 UTC       justIN version: 01.06.00