justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 273624.0@dunegpschedd01.fnal.gov

Jobsub ID273624.0@dunegpschedd01.fnal.gov
Workflow ID11686
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-10 17:45:48
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2026-01-11 00:59:23
From worker nodeHostnamewn2206291.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-10 17:47:22
Input filesmonte-carlo-011686-000001
JobscriptExit code0
Real time7h (25775s)
CPU time7h (25210s = 97%)
Max RSS bytes4457951232 (4251 MiB)
Outputting started2026-01-11 00:56:58
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11686/1/001/reco_stage1.root
Finished2026-01-11 00:59:23
Saved logsjustin-logs:273624.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 172639 traces, 0 summary
Retagger: tagging trace set: gauss with 125982 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125982 traces tagged "gauss"
FrameSaver: q=4.42573e+06 n=2153397 tag=gauss
wclsFrameSaver: saving 172639 traces tagged "wiener"
FrameSaver: q=4.72292e+06 n=1876700 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25120 XUs and 33081 XVs -> 375 XUVs
C:0 T:1 66302 XUs and 74490 XVs -> 6743 XUVs
C:0 T:2 135994 XUs and 138268 XVs -> 6577 XUVs
C:0 T:3 55625 XUs and 49222 XVs -> 796 XUVs
C:0 T:4 17873 XUs and 14123 XVs -> 319 XUVs
C:0 T:5 90524 XUs and 51618 XVs -> 7092 XUVs
C:0 T:6 174388 XUs and 174777 XVs -> 9651 XUVs
C:0 T:7 62155 XUs and 61322 XVs -> 598 XUVs
C:0 T:8 10282 XUs and 12792 XVs -> 968 XUVs
C:0 T:9 84818 XUs and 114737 XVs -> 19470 XUVs
C:0 T:10 42921 XUs and 27566 XVs -> 4432 XUVs
C:0 T:11 222 XUs and 153 XVs -> 26 XUVs
57047 XUVs total
16001 collection wire objects
50040 potential space points
Neighbour search...
4748548 tests to find 2329610 neighbours
Iterating with no regularization...
Begin: 7.79049e+07
0 6.10994e+07
1 5.968e+07
2 5.94667e+07
3 5.94029e+07
4 5.93747e+07
Now with regularization...
Begin: 4.63891e+07
0 4.63407e+07
1 4.63108e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411660170
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411660170
fLifetime: 30000 [us]
11-Jan-2026 00:56:54 GMT  Closed output file "reco_stage1.root"
11-Jan-2026 00:56:54 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          780.926       991.979       1295.14       1000.66       148.39         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00221662    0.00604407     0.0111001    0.00626638     0.0023858       10     
reco:ophit:OpHitFinder                             0.490277      0.541121      0.622826      0.528074      0.0478225       10     
reco:opflash:OpFlashFinder                         0.304246      0.392082      0.461632      0.395018      0.0492587       10     
reco:opslicer:OpSlicer                              1.12068       1.33901       1.6433        1.29763      0.174419        10     
reco:caldata:DataPrepModule                         9.05942       10.4778       11.2163       10.529       0.652764        10     
reco:wclsdatasp:WireCellToolkit                     70.0494       75.6839       84.9345       74.9132       4.24817        10     
reco:gaushit:GausHitFinder                          2.24144       2.91767       3.76481       2.85785      0.472946        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000959938   0.00115509    0.00160359    0.00111031    0.000170671      10     
reco:reco3d:SpacePointSolver                        20.9324       35.8623       64.0338       33.373        12.5853        10     
reco:hitpdune:DisambigFromSpacePoints               3.71514       5.39024       7.77025       5.4102        1.22919        10     
reco:pandora:StandardPandora                        54.4624       108.488       216.649       98.2192       44.8759        10     
reco:pandoraWriter:StandardPandora                 0.850512       1.02478       1.26396       1.02604      0.114127        10     
reco:pandoraTrack:LArPandoraTrackCreation           14.7079       19.1889       25.5819       18.0835       3.06874        10     
reco:pandoraShower:LArPandoraShowerCreation         15.7878       23.1226       29.1336       23.0185       4.24165        10     
reco:pandoracalo:Calorimetry                        7.98189       9.57314       12.5218       9.12158       1.35426        10     
reco:pandoracalonosce:Calorimetry                   7.42393       9.12122       12.5874       8.5761        1.54337        10     
reco:pandorapid:Chi2ParticleID                     0.0115317     0.013225      0.0154092     0.0127641    0.00126598       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.103934       0.13308      0.156101      0.137125      0.0170522       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0107665     0.0122451     0.0147144     0.0121087    0.00106345       10     
reco:pandoraShowercalo:ShowerCalorimetry            17.5911       24.9171       30.8965       25.9343       4.31382        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       16.4828       24.5877       30.559        25.3521       4.59538        10     
reco:emtrkmichelid:EmTrackMichelId                  356.476       425.795       508.831       429.108       45.7643        10     
reco:crttag:SingleCRTMatchingProducer              0.0399697     0.050449      0.0592635     0.050106     0.00558749       10     
reco:crtreco:TwoCRTMatchingProducer                0.0309282     0.0389198     0.048578      0.0396946    0.00488954       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.4677        2.18591       2.7828        2.33858       0.42948        10     
reco:pandora2Track:LArPandoraTrackCreation          51.082        68.3557       86.0876       69.1841       11.3156        10     
reco:pandora2calo:Calorimetry                       24.5993       32.3655       41.2115       31.9313       5.69131        10     
reco:pandora2calonosce:Calorimetry                  23.5061       30.8292       38.8679       31.062        5.16717        10     
reco:pandora2pid:Chi2ParticleID                    0.0299106     0.0342968     0.0395876     0.0342248    0.00288435       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.140194      0.173724      0.213177      0.176601      0.0239212       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0275468     0.0322675     0.0372064     0.0321185    0.00281527       10     
reco:pandora2Shower:LArPandoraShowerCreation        15.881        21.0022       28.0628       21.0061       3.5316         10     
reco:pandora2Showercalo:ShowerCalorimetry           20.4717       27.7021       35.545        27.7977       4.83531        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      15.5592       24.7802       33.4264       24.761        5.2593         10     
[art]:TriggerResults:TriggerResultInserter        9.9227e-05    0.000126323   0.000197635   0.000124172   2.7933e-05       10     
end_path:out1:RootOutput                          1.3261e-05    1.48312e-05   1.8781e-05    1.40025e-05   1.71174e-06      10     
end_path:out1:RootOutput(write)                     4.11667       5.59415       7.26465       5.73153      0.879283        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5268.4 MB
  Peak resident set size usage (VmHWM): 4457.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.42 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.55 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11487884
-rw-r--r-- 1 dune01 dune01 2653313572 Jan 10 22:07 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2514840046 Jan 10 22:03 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2394246498 Jan 11 00:56 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2111217425 Jan 10 18:22 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2000825127 Jan 10 17:57 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   88522960 Jan 11 00:43 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242938 Jan 11 00:56 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 11 00:56 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 11 00:56 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 11 00:56 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 10 22:07 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 10 18:22 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 11 00:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 10 17:47 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 11 00:56 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 10 22:20 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 10 22:07 debugprod.log
justIN time: 2026-02-06 11:40:58 UTC       justIN version: 01.06.00