justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296768.0@dunegpschedd02.fnal.gov

Jobsub ID296768.0@dunegpschedd02.fnal.gov
Workflow ID12571
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 13:59:12
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2026-01-30 19:23:00
From worker nodeHostnamewn5917301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 14:00:42
Input filesmonte-carlo-012571-000001
JobscriptExit code0
Real time5h (19240s)
CPU time5h (18498s = 96%)
Max RSS bytes4154028032 (3961 MiB)
Outputting started2026-01-30 19:21:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12571/1/001/reco_stage1.root
Finished2026-01-30 19:23:00
Saved logsjustin-logs:296768.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 110358 traces, 0 summary
Retagger: tagging trace set: gauss with 84937 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 84937 traces tagged "gauss"
FrameSaver: q=3.01394e+06 n=1519507 tag=gauss
wclsFrameSaver: saving 110358 traces tagged "wiener"
FrameSaver: q=3.18391e+06 n=1354010 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34389 XUs and 34806 XVs -> 281 XUVs
C:0 T:1 65029 XUs and 59235 XVs -> 4245 XUVs
C:0 T:2 34784 XUs and 41952 XVs -> 2389 XUVs
C:0 T:3 18720 XUs and 19073 XVs -> 104 XUVs
C:0 T:4 9539 XUs and 10352 XVs -> 87 XUVs
C:0 T:5 34738 XUs and 34537 XVs -> 2764 XUVs
C:0 T:6 70708 XUs and 63145 XVs -> 2462 XUVs
C:0 T:7 31522 XUs and 26171 XVs -> 207 XUVs
C:0 T:8 9389 XUs and 8211 XVs -> 108 XUVs
C:0 T:9 58393 XUs and 62134 XVs -> 6609 XUVs
C:0 T:10 51596 XUs and 32306 XVs -> 3679 XUVs
C:0 T:11 515 XUs and 368 XVs -> 48 XUVs
22983 XUVs total
9440 collection wire objects
21198 potential space points
Neighbour search...
836598 tests to find 469302 neighbours
Iterating with no regularization...
Begin: 5.96374e+07
0 4.90359e+07
1 4.82666e+07
2 4.81695e+07
3 4.81451e+07
Now with regularization...
Begin: 3.96941e+07
0 3.96708e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412059357
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412059357
fLifetime: 30000 [us]
30-Jan-2026 19:21:21 GMT  Closed output file "reco_stage1.root"
30-Jan-2026 19:21:21 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          347.795       619.58        1067.06       586.362       202.286        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131927    0.00425316    0.00808164    0.00444183    0.00172896       10     
reco:ophit:OpHitFinder                             0.291231      0.399699      0.514356      0.396832      0.0564056       10     
reco:opflash:OpFlashFinder                         0.178674      0.223104      0.271203      0.217677      0.0319355       10     
reco:opslicer:OpSlicer                             0.658533      0.998789       1.16982       1.02454      0.133869        10     
reco:caldata:DataPrepModule                         6.07498       7.77362       8.9376        8.17866      0.980785        10     
reco:wclsdatasp:WireCellToolkit                     39.3377       50.7027       59.5176       51.7268       5.49125        10     
reco:gaushit:GausHitFinder                          1.1767        1.96369       3.17758       1.9395        0.55132        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000516264   0.000886691   0.00137767    0.000843477   0.00027839       10     
reco:reco3d:SpacePointSolver                        8.04695       25.3699       47.5592       28.2997       13.706         10     
reco:hitpdune:DisambigFromSpacePoints                1.164        2.4403        4.75137       2.17171       1.05667        10     
reco:pandora:StandardPandora                        19.4882       64.373        205.428       49.8909       50.9371        10     
reco:pandoraWriter:StandardPandora                 0.421488      0.683774      0.912496      0.707493      0.160531        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.96371       7.3455        17.7235       6.62873       3.7841         10     
reco:pandoraShower:LArPandoraShowerCreation         3.03888       8.34997       18.1467       6.6179        4.27458        10     
reco:pandoracalo:Calorimetry                        2.19157       4.01409       7.89798       3.96538       1.68322        10     
reco:pandoracalonosce:Calorimetry                   1.89652       3.66158       7.44803       3.54492       1.64344        10     
reco:pandorapid:Chi2ParticleID                    0.00578899    0.00941349     0.0119839    0.00991911    0.00184852       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0501431     0.0904298     0.115963      0.0941588     0.0202771       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00558935    0.00921604     0.0113179    0.00973877    0.00167568       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.47548       8.01486       12.9101       7.46087       3.72386        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.65875       7.55399       12.6958       7.13679       3.69627        10     
reco:emtrkmichelid:EmTrackMichelId                  210.082       348.219       531.959       340.658       89.2222        10     
reco:crttag:SingleCRTMatchingProducer              0.0110161     0.0224076     0.0403886     0.0179396    0.00946068       10     
reco:crtreco:TwoCRTMatchingProducer                0.0100237     0.0179646     0.0345605     0.0156544    0.00721907       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.804068       1.43036       2.11337       1.48098      0.358721        10     
reco:pandora2Track:LArPandoraTrackCreation          8.15957       24.5208       49.2094       20.1228       11.8989        10     
reco:pandora2calo:Calorimetry                       4.21986       11.5596       23.9995       9.40853       6.09141        10     
reco:pandora2calonosce:Calorimetry                  3.89172       10.199        20.6506       8.43407       5.28034        10     
reco:pandora2pid:Chi2ParticleID                    0.0127401     0.0212943     0.0399567     0.0187342    0.00809083       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0562925     0.105894      0.181289      0.096578      0.038366        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0125313     0.0221076     0.0409291     0.0220502     0.0081526       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.19971       8.24368       16.106        7.40304       4.12598        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.16127       8.96002       17.0055       8.32105       4.57941        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.26109       7.52061       16.8731       6.20746       4.39957        10     
[art]:TriggerResults:TriggerResultInserter        3.0337e-05    5.60087e-05   0.000149372   5.06255e-05   3.2809e-05       10     
end_path:out1:RootOutput                           5.55e-06     1.04777e-05   3.5306e-05    7.7095e-06    8.62387e-06      10     
end_path:out1:RootOutput(write)                     2.17575       4.57754       7.01633       4.65816       1.4975         10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5001.39 MB
  Peak resident set size usage (VmHWM): 4154.03 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9710904
-rw-r--r-- 1 dune01 dune01 2295992863 Jan 30 17:34 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2183211339 Jan 30 17:31 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1976509348 Jan 30 19:21 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1740212276 Jan 30 14:29 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1673860305 Jan 30 14:10 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   73546240 Jan 30 19:15 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243233 Jan 30 19:21 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Jan 30 19:21 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Jan 30 19:21 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Jan 30 19:21 time.db
-rw-r--r-- 1 dune01 dune01        519 Jan 30 17:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 30 14:29 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Jan 30 19:21 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Jan 30 14:00 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Jan 30 19:21 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Jan 30 17:47 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Jan 30 17:35 debugprod.log
justIN time: 2026-02-04 13:58:37 UTC       justIN version: 01.06.00