justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 245817.1@dunegpschedd02.fnal.gov

Jobsub ID245817.1@dunegpschedd02.fnal.gov
Workflow ID10210
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-15 08:06:32
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-15 12:26:19
From worker nodeHostnamewn2204250.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-15 08:07:13
Input filesmonte-carlo-010210-000001
JobscriptExit code0
Real time4h (15445s)
CPU time4h (14962s = 96%)
Max RSS bytes4249997312 (4053 MiB)
Outputting started2025-11-15 12:24:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10210/1/001/reco_stage1.root
Finished2025-11-15 12:26:19
Saved logsjustin-logs:245817.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

leKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169525 traces, 0 summary
Retagger: tagging trace set: gauss with 128602 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 128602 traces tagged "gauss"
FrameSaver: q=4.36044e+06 n=2336763 tag=gauss
wclsFrameSaver: saving 169525 traces tagged "wiener"
FrameSaver: q=4.60469e+06 n=2069945 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41529 XUs and 45735 XVs -> 576 XUVs
C:0 T:1 104278 XUs and 100278 XVs -> 6288 XUVs
C:0 T:2 114705 XUs and 131957 XVs -> 7056 XUVs
C:0 T:3 48976 XUs and 48144 XVs -> 347 XUVs
C:0 T:4 21872 XUs and 22164 XVs -> 303 XUVs
C:0 T:5 143495 XUs and 101358 XVs -> 18396 XUVs
C:0 T:6 127062 XUs and 122016 XVs -> 5827 XUVs
C:0 T:7 56308 XUs and 49847 XVs -> 631 XUVs
C:0 T:8 7765 XUs and 8632 XVs -> 81 XUVs
C:0 T:9 55377 XUs and 118046 XVs -> 6235 XUVs
C:0 T:10 71302 XUs and 54926 XVs -> 5112 XUVs
C:0 T:11 247 XUs and 187 XVs -> 27 XUVs
50879 XUVs total
15766 collection wire objects
46849 potential space points
Neighbour search...
5858451 tests to find 2593812 neighbours
Iterating with no regularization...
Begin: 9.07043e+07
0 7.56883e+07
1 7.46005e+07
2 7.44561e+07
3 7.44167e+07
Now with regularization...
Begin: 6.31707e+07
0 6.31143e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410525538
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410525538
fLifetime: 30000 [us]
15-Nov-2025 12:24:37 GMT  Closed output file "reco_stage1.root"
15-Nov-2025 12:24:37 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          319.645       461.548       683.541       445.24        107.03         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00216401    0.00605156    0.00979188    0.00542211    0.00261317       10     
reco:ophit:OpHitFinder                             0.272948      0.336624      0.447514      0.341733      0.0473604       10     
reco:opflash:OpFlashFinder                         0.141171      0.183995      0.259254      0.177586      0.0362956       10     
reco:opslicer:OpSlicer                             0.551326      0.796409       1.23735      0.800441      0.187846        10     
reco:caldata:DataPrepModule                         5.41255       6.93405       8.4818        7.04099      0.960175        10     
reco:wclsdatasp:WireCellToolkit                     37.6114       45.0496       54.4845       44.0761       5.21416        10     
reco:gaushit:GausHitFinder                          1.10734       1.64251       2.83993       1.50336      0.467978        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000482527   0.00075496    0.00114618    0.000708558   0.000181286      10     
reco:reco3d:SpacePointSolver                        8.27115       18.1059       30.3489       19.3353       6.25406        10     
reco:hitpdune:DisambigFromSpacePoints               1.04553       1.97284       3.90703       1.91268      0.718768        10     
reco:pandora:StandardPandora                        30.0393       42.6857       75.6118       39.2528       12.9969        10     
reco:pandoraWriter:StandardPandora                 0.436926      0.608952      0.862459      0.570516      0.121904        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.41908       6.29651       10.427        5.36684       2.13941        10     
reco:pandoraShower:LArPandoraShowerCreation         4.52654       7.73096       13.3509       7.6648        2.5045         10     
reco:pandoracalo:Calorimetry                        1.83228       3.62843       6.06688       3.0786        1.32366        10     
reco:pandoracalonosce:Calorimetry                   1.49511       3.41393       6.21621       3.06733       1.2748         10     
reco:pandorapid:Chi2ParticleID                    0.00367709    0.00742991     0.0137972    0.00662146    0.00286144       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0366605     0.0738959     0.124155      0.0681862     0.0290706       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00369883    0.00702572     0.0104112    0.00644365    0.00216102       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.14135       7.67907       15.7449       6.94574       3.47029        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.77279       7.2628        14.3088       6.41959       2.99779        10     
reco:emtrkmichelid:EmTrackMichelId                  151.736       236.097       324.107       225.633       52.5817        10     
reco:crttag:SingleCRTMatchingProducer              0.0117235     0.0260777     0.0625692     0.0218154     0.0149719       10     
reco:crtreco:TwoCRTMatchingProducer               0.00598601     0.0158095     0.0399703     0.0142478    0.00947339       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.597189      0.987114       1.87943      0.867589      0.344752        10     
reco:pandora2Track:LArPandoraTrackCreation          9.3644        20.1477       36.5532       18.4137       8.20718        10     
reco:pandora2calo:Calorimetry                       5.60208       10.9279       19.2987       9.89798       4.45007        10     
reco:pandora2calonosce:Calorimetry                  4.05428       10.1795       18.9074       9.10791       4.19845        10     
reco:pandora2pid:Chi2ParticleID                   0.00953276     0.0171691     0.0277155     0.0161288    0.00543021       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.055693      0.0897979     0.151085      0.0830068     0.0269274       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00966384     0.0168093      0.02521      0.0162594    0.00467952       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.81192       7.3652        14.5557       6.63843       3.54107        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.09928       9.30081       17.7194       8.08674       4.64278        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.66392       8.10119       13.5989       7.79053       3.5143         10     
[art]:TriggerResults:TriggerResultInserter        5.6857e-05    0.000109026   0.000256943   8.59165e-05   5.51859e-05      10     
end_path:out1:RootOutput                          1.2273e-05    1.98653e-05   3.7601e-05    1.81145e-05   7.46977e-06      10     
end_path:out1:RootOutput(write)                     2.34031       3.63504       5.77345       3.37907      0.917395        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5097.58 MB
  Peak resident set size usage (VmHWM): 4250 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.24 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9942620
-rw-r--r-- 1 dune01 dune01 2338281070 Nov 15 11:05 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2220721750 Nov 15 11:03 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2026671671 Nov 15 12:24 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1790283765 Nov 15 08:33 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1728963048 Nov 15 08:16 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   75693220 Nov 15 12:16 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242661 Nov 15 12:24 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 15 12:24 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 15 12:24 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 15 12:24 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 15 11:05 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 15 08:33 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 15 12:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 15 08:07 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 15 12:24 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 15 11:13 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 15 11:05 debugprod.log
justIN time: 2026-02-09 12:25:49 UTC       justIN version: 01.06.00