justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 262271.0@dunegpschedd02.fnal.gov

Jobsub ID262271.0@dunegpschedd02.fnal.gov
Workflow ID11208
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-13 17:10:51
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-12-13 23:19:33
From worker nodeHostnamewn1208240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-13 17:11:43
Input filesmonte-carlo-011208-000001
JobscriptExit code0
Real time6h (21969s)
CPU time5h (21171s = 96%)
Max RSS bytes4205400064 (4010 MiB)
Outputting started2025-12-13 23:17:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11208/1/001/reco_stage1.root
Finished2025-12-13 23:19:33
Saved logsjustin-logs:262271.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

der
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142514 traces, 0 summary
Retagger: tagging trace set: gauss with 108424 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 108424 traces tagged "gauss"
FrameSaver: q=3.69818e+06 n=1926360 tag=gauss
wclsFrameSaver: saving 142514 traces tagged "wiener"
FrameSaver: q=3.91437e+06 n=1700531 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31555 XUs and 31365 XVs -> 735 XUVs
C:0 T:1 54552 XUs and 48344 XVs -> 3923 XUVs
C:0 T:2 123459 XUs and 174950 XVs -> 5552 XUVs
C:0 T:3 56817 XUs and 66301 XVs -> 351 XUVs
C:0 T:4 14343 XUs and 16331 XVs -> 187 XUVs
C:0 T:5 107223 XUs and 65491 XVs -> 7017 XUVs
C:0 T:6 117811 XUs and 141739 XVs -> 8306 XUVs
C:0 T:7 43240 XUs and 52680 XVs -> 709 XUVs
C:0 T:8 8814 XUs and 9671 XVs -> 191 XUVs
C:0 T:9 116037 XUs and 114540 XVs -> 13142 XUVs
C:0 T:10 117091 XUs and 26891 XVs -> 12283 XUVs
C:0 T:11 91 XUs and 47 XVs -> 12 XUVs
52408 XUVs total
13740 collection wire objects
48182 potential space points
Neighbour search...
6725390 tests to find 3129750 neighbours
Iterating with no regularization...
Begin: 7.57459e+07
0 5.20963e+07
1 5.06709e+07
2 5.05057e+07
3 5.04504e+07
4 5.04218e+07
Now with regularization...
Begin: 3.96712e+07
0 3.96349e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411096428
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411096428
fLifetime: 30000 [us]
13-Dec-2025 23:17:49 GMT  Closed output file "reco_stage1.root"
13-Dec-2025 23:17:49 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          500.798       713.292       920.159       726.477       121.294        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00177378    0.00488923    0.00778612    0.00495664    0.00218911       10     
reco:ophit:OpHitFinder                             0.356121      0.454736      0.565966      0.476934      0.0646694       10     
reco:opflash:OpFlashFinder                         0.236809      0.292471       0.36143      0.285518      0.0372268       10     
reco:opslicer:OpSlicer                             0.901895       1.17739       1.45363       1.20292      0.187693        10     
reco:caldata:DataPrepModule                         7.00917       8.68399       9.8119        8.91901      0.784687        10     
reco:wclsdatasp:WireCellToolkit                     49.9686       62.7199       76.1628        62.21        8.07601        10     
reco:gaushit:GausHitFinder                          1.58086       2.10201       2.52801       2.14075      0.274266        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000783345   0.00108092     0.0015532    0.00105392    0.000228229      10     
reco:reco3d:SpacePointSolver                        17.1924       27.0688       39.1376       26.9451       6.84738        10     
reco:hitpdune:DisambigFromSpacePoints               2.15701       2.99279       3.9219        2.93396      0.608133        10     
reco:pandora:StandardPandora                        28.2056       65.7684       83.1259       69.6198       15.2945        10     
reco:pandoraWriter:StandardPandora                 0.514204       0.81106       1.06211       0.83571      0.146271        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.32796       8.91947       12.6385       9.39275       2.19753        10     
reco:pandoraShower:LArPandoraShowerCreation         4.7352        10.8188       15.0342       11.7947       3.4514         10     
reco:pandoracalo:Calorimetry                        3.00513       5.18414       7.12333       5.38328       1.42083        10     
reco:pandoracalonosce:Calorimetry                   2.49731       4.96876       7.71872       5.23451       1.74945        10     
reco:pandorapid:Chi2ParticleID                     0.0069657     0.011424      0.0157589     0.0114379    0.00268483       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0581455      0.12199       0.29555      0.114372      0.0631232       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00658502     0.010159      0.0139232     0.010176     0.00222385       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.23531       12.5132       20.0782       12.5036       5.35045        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.74687       11.5658       19.9262       12.6999       5.02071        10     
reco:emtrkmichelid:EmTrackMichelId                  281.432       385.889       489.292       392.975       57.6485        10     
reco:crttag:SingleCRTMatchingProducer              0.0151878     0.0266822     0.0394144     0.0276746     0.0065505       10     
reco:crtreco:TwoCRTMatchingProducer                0.0134597     0.0222584     0.0319409     0.0202026    0.00619068       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.05183       1.51458       2.08146       1.48663      0.275741        10     
reco:pandora2Track:LArPandoraTrackCreation          12.8313       29.9838       47.3453       30.896         8.627         10     
reco:pandora2calo:Calorimetry                       6.69844       16.1686       24.957        16.4103       4.9455         10     
reco:pandora2calonosce:Calorimetry                  6.09952       14.2134        24.36        14.5932       5.30665        10     
reco:pandora2pid:Chi2ParticleID                    0.0195805     0.0274343     0.0364378     0.0282701    0.00501485       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.089345      0.133187      0.175557      0.135786      0.0264805       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0193712     0.0276221     0.0352977     0.0284441    0.00517282       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.68189       10.4311       15.6756       10.1087       3.16804        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.90479       12.1738       19.4126       11.8843       4.15889        10     
reco:pandora2Showercalonosce:ShowerCalorimetry       4.993        11.4391       18.1868       11.8025       3.93927        10     
[art]:TriggerResults:TriggerResultInserter        3.5066e-05    6.60181e-05   0.000198774   5.08855e-05   4.57204e-05      10     
end_path:out1:RootOutput                           6.302e-06    8.1904e-06    1.5199e-05     7.509e-06    2.43121e-06      10     
end_path:out1:RootOutput(write)                     4.02823       4.82734       6.24928       4.74328      0.645307        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5041.59 MB
  Peak resident set size usage (VmHWM): 4205.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.62 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10520144
-rw-r--r-- 1 dune01 dune01 2456971867 Dec 13 21:15 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2332910505 Dec 13 21:11 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2149755959 Dec 13 23:17 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1920046792 Dec 13 17:44 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1835784614 Dec 13 17:23 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   76525700 Dec 13 23:07 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243043 Dec 13 23:17 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec 13 23:17 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec 13 23:17 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec 13 23:17 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec 13 21:15 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 13 17:44 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 13 23:17 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec 13 17:11 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec 13 23:17 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec 13 21:27 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec 13 21:15 debugprod.log
justIN time: 2026-02-07 10:52:31 UTC       justIN version: 01.06.00