justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 224261.0@dunegpschedd02.fnal.gov

Jobsub ID224261.0@dunegpschedd02.fnal.gov
Workflow ID8539
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-04 08:20:11
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-10-04 10:45:07
From worker nodeHostnamen245.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-04 08:21:05
Input filesmonte-carlo-008539-000001
JobscriptExit code0
Real time2h (8548s)
CPU time2h (8324s = 97%)
Max RSS bytes4137345024 (3945 MiB)
Outputting started2025-10-04 10:43:34
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/08539/1/001/reco_stage1.root
Finished2025-10-04 10:45:07
Saved logsjustin-logs:224261.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122460 traces, 0 summary
Retagger: tagging trace set: gauss with 93223 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93223 traces tagged "gauss"
FrameSaver: q=3.33968e+06 n=1663042 tag=gauss
wclsFrameSaver: saving 122460 traces tagged "wiener"
FrameSaver: q=3.53801e+06 n=1477136 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29684 XUs and 34024 XVs -> 351 XUVs
C:0 T:1 48813 XUs and 56154 XVs -> 5470 XUVs
C:0 T:2 50373 XUs and 64258 XVs -> 4094 XUVs
C:0 T:3 25505 XUs and 24272 XVs -> 145 XUVs
C:0 T:4 17041 XUs and 18274 XVs -> 135 XUVs
C:0 T:5 77061 XUs and 65448 XVs -> 6164 XUVs
C:0 T:6 79261 XUs and 96160 XVs -> 2483 XUVs
C:0 T:7 36202 XUs and 45508 XVs -> 221 XUVs
C:0 T:8 9050 XUs and 6562 XVs -> 174 XUVs
C:0 T:9 163461 XUs and 82889 XVs -> 16584 XUVs
C:0 T:10 34443 XUs and 12990 XVs -> 3832 XUVs
C:0 T:11 182 XUs and 138 XVs -> 15 XUVs
39668 XUVs total
10192 collection wire objects
37082 potential space points
Neighbour search...
4800190 tests to find 2258644 neighbours
Iterating with no regularization...
Begin: 9.85978e+07
0 7.85015e+07
1 7.74199e+07
2 7.72486e+07
3 7.71953e+07
Now with regularization...
Begin: 6.71246e+07
0 6.70556e+07
1 6.70222e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 409680825
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409680825
fLifetime: 30000 [us]
04-Oct-2025 11:43:32 BST  Closed output file "reco_stage1.root"
04-Oct-2025 11:43:32 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          167.935       269.553       370.961       265.828       70.0196        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000925173   0.00347117    0.00558922    0.00356061    0.00144877       10     
reco:ophit:OpHitFinder                             0.212369      0.247622      0.287048      0.240865      0.0264606       10     
reco:opflash:OpFlashFinder                         0.114396      0.157935       0.1966       0.162768      0.0285225       10     
reco:opslicer:OpSlicer                             0.383473      0.527136      0.648517      0.553745      0.0900534       10     
reco:caldata:DataPrepModule                         4.18772       4.34981       4.55876       4.31915      0.120609        10     
reco:wclsdatasp:WireCellToolkit                     29.2285       30.9027       32.6965       30.7197      0.997234        10     
reco:gaushit:GausHitFinder                         0.593704      0.934185       1.21268       0.95813      0.209681        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000320975   0.000614485   0.00112438    0.00054161    0.000276632      10     
reco:reco3d:SpacePointSolver                        5.16016       11.2096       19.6894       10.5428       4.83189        10     
reco:hitpdune:DisambigFromSpacePoints               0.54581       1.18237       1.72041       1.15831      0.388504        10     
reco:pandora:StandardPandora                        11.6294       25.4822       48.3871       25.2106       11.7183        10     
reco:pandoraWriter:StandardPandora                 0.222853      0.380697      0.553041      0.364786      0.0936476       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.68734       3.90667       7.74663       3.57521       1.77813        10     
reco:pandoraShower:LArPandoraShowerCreation         1.3022        4.0179        7.85853       3.63731       2.17849        10     
reco:pandoracalo:Calorimetry                       0.955071       2.1063        4.64056       1.97238       1.05939        10     
reco:pandoracalonosce:Calorimetry                  0.845817       1.97465       3.92789       2.01125      0.917133        10     
reco:pandorapid:Chi2ParticleID                     0.0033877    0.00459781    0.00666566    0.00444989    0.000990985      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0292852     0.0407366     0.0519625     0.0408363    0.00784008       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00368927    0.00463213    0.00617847    0.00451581    0.000756596      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.3215        4.42942       8.76418       4.33403       2.28141        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.28243       4.10371       8.28333       4.13848       2.10551        10     
reco:emtrkmichelid:EmTrackMichelId                  90.9508       131.666       165.948       131.907       25.7542        10     
reco:crttag:SingleCRTMatchingProducer             0.00783945     0.0119438     0.0186153     0.0103076    0.00347096       10     
reco:crtreco:TwoCRTMatchingProducer               0.00553006    0.00809356     0.0149986    0.00681494     0.0029073       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.486414      0.689096      0.934797      0.699265      0.156801        10     
reco:pandora2Track:LArPandoraTrackCreation          4.66568       12.8291       23.025        12.1874       5.92002        10     
reco:pandora2calo:Calorimetry                       2.66842       6.48203       11.7522       6.19603       3.10147        10     
reco:pandora2calonosce:Calorimetry                  2.10421       6.05883       10.3833       6.22033       2.85277        10     
reco:pandora2pid:Chi2ParticleID                   0.00708683    0.00989955     0.0133128     0.0102594    0.00199845       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0338794     0.0499807     0.063192      0.0512428    0.00997646       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00691277     0.0097643     0.0125907    0.00991298    0.00187188       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.67131       4.2406        8.41893       3.84211       2.13885        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.04992       4.9127        8.93103       4.83954       2.22567        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.78348       4.17911       6.71664       4.10859       1.77476        10     
[art]:TriggerResults:TriggerResultInserter        5.5665e-05    7.70073e-05   0.000184608   6.28385e-05   3.72724e-05      10     
end_path:out1:RootOutput                           7.905e-06    1.01701e-05   1.9066e-05    9.3475e-06    3.06266e-06      10     
end_path:out1:RootOutput(write)                     1.53128       2.28277       3.02038       2.22881      0.491037        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4941.93 MB
  Peak resident set size usage (VmHWM): 4137.35 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.69 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9806204
-rw-r--r--. 1 dune004 dune 2315569460 Oct  4 10:57 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2198398975 Oct  4 10:55 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 1987655237 Oct  4 11:43 reco_stage1.root
-rw-r--r--. 1 dune004 dune 1773218063 Oct  4 09:34 g4_stage2.root
-rw-r--r--. 1 dune004 dune 1694665370 Oct  4 09:25 g4_stage1.root
-rw-r--r--. 1 dune004 dune   71416640 Oct  4 11:40 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     242519 Oct  4 11:43 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Oct  4 11:43 mem.db
-rw-r--r--. 1 dune004 dune      62376 Oct  4 11:43 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Oct  4 11:43 time.db
-rw-r--r--. 1 dune004 dune        519 Oct  4 10:57 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct  4 09:34 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Oct  4 11:43 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Oct  4 09:21 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Oct  4 11:43 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Oct  4 11:02 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Oct  4 10:57 debugprod.log
justIN time: 2025-11-03 18:40:59 UTC       justIN version: 01.05.01