justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275211.1@dunegpschedd01.fnal.gov

Jobsub ID275211.1@dunegpschedd01.fnal.gov
Workflow ID11738
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 22:53:22
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-12 03:11:33
From worker nodeHostnamewn2206251.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 22:55:27
Input filesmonte-carlo-011738-000001
JobscriptExit code0
Real time4h (15289s)
CPU time4h (15065s = 98%)
Max RSS bytes4622315520 (4408 MiB)
Outputting started2026-01-12 03:10:17
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11738/1/001/reco_stage1.root
Finished2026-01-12 03:11:33
Saved logsjustin-logs:275211.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

KeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 152463 traces, 0 summary
Retagger: tagging trace set: gauss with 111600 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111600 traces tagged "gauss"
FrameSaver: q=3.96987e+06 n=1922638 tag=gauss
wclsFrameSaver: saving 152463 traces tagged "wiener"
FrameSaver: q=4.23382e+06 n=1689696 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38703 XUs and 27228 XVs -> 345 XUVs
C:0 T:1 79689 XUs and 70347 XVs -> 8793 XUVs
C:0 T:2 109235 XUs and 181379 XVs -> 6522 XUVs
C:0 T:3 47511 XUs and 70471 XVs -> 533 XUVs
C:0 T:4 15403 XUs and 14364 XVs -> 200 XUVs
C:0 T:5 81882 XUs and 61866 XVs -> 11752 XUVs
C:0 T:6 93421 XUs and 97353 XVs -> 3971 XUVs
C:0 T:7 42345 XUs and 41826 XVs -> 296 XUVs
C:0 T:8 10081 XUs and 12773 XVs -> 201 XUVs
C:0 T:9 90350 XUs and 145278 XVs -> 12561 XUVs
C:0 T:10 26084 XUs and 22308 XVs -> 5655 XUVs
C:0 T:11 294 XUs and 171 XVs -> 34 XUVs
50863 XUVs total
13556 collection wire objects
47254 potential space points
Neighbour search...
7055838 tests to find 3278166 neighbours
Iterating with no regularization...
Begin: 9.13675e+07
0 7.56558e+07
1 7.4564e+07
2 7.44399e+07
3 7.44071e+07
Now with regularization...
Begin: 6.16981e+07
0 6.1654e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411684474
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411684474
fLifetime: 30000 [us]
12-Jan-2026 03:10:14 GMT  Closed output file "reco_stage1.root"
12-Jan-2026 03:10:14 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          265.741       448.219       620.177       441.11        109.803        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000858564   0.00356704    0.00561315    0.00376937    0.00163865       10     
reco:ophit:OpHitFinder                             0.279997      0.330854      0.385358      0.329898      0.0286584       10     
reco:opflash:OpFlashFinder                         0.222931       0.28387      0.354295      0.292292      0.0411376       10     
reco:opslicer:OpSlicer                              0.60668      0.770622      0.951761      0.762207      0.111265        10     
reco:caldata:DataPrepModule                         6.0984        6.25701       6.37489       6.27739      0.0890348       10     
reco:wclsdatasp:WireCellToolkit                     44.679        48.8606       52.502        49.6294       2.50073        10     
reco:gaushit:GausHitFinder                          1.15065       1.62494       2.13463       1.57363      0.293412        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000501523   0.000726309   0.000908514   0.000728818   0.00010834       10     
reco:reco3d:SpacePointSolver                        7.93581       18.1754       32.1432       16.6295       6.7543         10     
reco:hitpdune:DisambigFromSpacePoints               1.31711       2.75915       4.49389       2.80419      0.920884        10     
reco:pandora:StandardPandora                        18.2188       49.7561       84.5934       41.1015       22.9817        10     
reco:pandoraWriter:StandardPandora                 0.331956       0.53093      0.688628       0.53672      0.106812        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.50502       5.9165        9.49044       5.67173       2.0661         10     
reco:pandoraShower:LArPandoraShowerCreation         2.27478       6.21433       10.9926       5.80454       2.66563        10     
reco:pandoracalo:Calorimetry                        1.35135       3.00499       4.62996       2.95798      0.989349        10     
reco:pandoracalonosce:Calorimetry                   1.18978       2.76235       4.30481       2.7104       0.941111        10     
reco:pandorapid:Chi2ParticleID                    0.00443473     0.0067208    0.00842341    0.00692414    0.00110689       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0555938     0.0763153     0.0969906     0.0734096     0.0127207       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00430922    0.00638359    0.00817309    0.00638756    0.00108915       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.16448       6.31679       11.2379       5.97064       2.83543        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.11771       6.19874       11.0131       5.89542       2.78171        10     
reco:emtrkmichelid:EmTrackMichelId                  147.013       223.844       293.026       228.291       42.6505        10     
reco:crttag:SingleCRTMatchingProducer              0.0127622     0.0219322     0.0267834     0.0230678    0.00409764       10     
reco:crtreco:TwoCRTMatchingProducer               0.00797424     0.0117097     0.014692      0.0119758    0.00200651       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.05441       1.49763       1.91828       1.46803      0.261242        10     
reco:pandora2Track:LArPandoraTrackCreation          7.5611        19.6999       33.6595       18.8713       8.06731        10     
reco:pandora2calo:Calorimetry                       3.40073       9.00276       15.253        8.75019       3.58476        10     
reco:pandora2calonosce:Calorimetry                  3.18596       8.57448       14.4963       8.37116       3.43012        10     
reco:pandora2pid:Chi2ParticleID                    0.010056      0.0165543     0.023663      0.0160286    0.00389178       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0641116     0.0943067     0.124387      0.0918663     0.0179334       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00996337     0.0160823     0.0228694     0.0157093    0.00362961       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.83644       6.92977       11.2358       6.80931       2.55166        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.98767       7.89054       12.9113       7.79623       3.04178        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.78181       7.33602       11.8941       7.32079       2.81576        10     
[art]:TriggerResults:TriggerResultInserter        4.7899e-05    6.67015e-05   0.000134993   5.68625e-05   2.43063e-05      10     
end_path:out1:RootOutput                           7.379e-06    8.3812e-06    1.2694e-05    7.8845e-06    1.47501e-06      10     
end_path:out1:RootOutput(write)                     2.22007       3.3257        4.43452       3.30868      0.644518        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5427.17 MB
  Peak resident set size usage (VmHWM): 4622.32 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.95 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11461404
-rw-r--r-- 1 dune01 dune01 2635531344 Jan 12 01:53 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2502275939 Jan 12 01:50 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2362220682 Jan 12 03:10 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2117337037 Jan 11 23:18 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2033690562 Jan 11 23:02 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   84788240 Jan 12 03:04 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243038 Jan 12 03:10 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 12 03:10 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 12 03:10 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 12 03:10 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 12 01:53 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 23:18 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 12 03:10 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 11 22:55 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 12 03:10 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 12 01:58 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 12 01:53 debugprod.log
justIN time: 2026-02-06 10:22:28 UTC       justIN version: 01.06.00