justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242130.1@dunegpschedd02.fnal.gov

Jobsub ID242130.1@dunegpschedd02.fnal.gov
Workflow ID9988
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-12 12:22:25
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-12 19:07:26
From worker nodeHostnamewn1907301.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-12 12:23:36
Input filesmonte-carlo-009988-000001
JobscriptExit code0
Real time6h (22877s)
CPU time6h (22242s = 97%)
Max RSS bytes4129644544 (3938 MiB)
Outputting started2025-11-12 18:44:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09988/1/001/reco_stage1.root
Finished2025-11-12 19:07:26
Saved logsjustin-logs:242130.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 114168 traces, 0 summary
Retagger: tagging trace set: gauss with 91217 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91217 traces tagged "gauss"
FrameSaver: q=3.30656e+06 n=1686285 tag=gauss
wclsFrameSaver: saving 114168 traces tagged "wiener"
FrameSaver: q=3.48926e+06 n=1528983 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25605 XUs and 22740 XVs -> 165 XUVs
C:0 T:1 41351 XUs and 37122 XVs -> 2672 XUVs
C:0 T:2 83058 XUs and 128534 XVs -> 4579 XUVs
C:0 T:3 41241 XUs and 44276 XVs -> 368 XUVs
C:0 T:4 9967 XUs and 10787 XVs -> 107 XUVs
C:0 T:5 31300 XUs and 22728 XVs -> 2134 XUVs
C:0 T:6 100452 XUs and 161108 XVs -> 12465 XUVs
C:0 T:7 33022 XUs and 39764 XVs -> 533 XUVs
C:0 T:8 10246 XUs and 9281 XVs -> 125 XUVs
C:0 T:9 56586 XUs and 64116 XVs -> 4538 XUVs
C:0 T:10 29519 XUs and 22454 XVs -> 2381 XUVs
C:0 T:11 181 XUs and 148 XVs -> 15 XUVs
30082 XUVs total
9647 collection wire objects
27185 potential space points
Neighbour search...
2655997 tests to find 1213754 neighbours
Iterating with no regularization...
Begin: 8.48695e+07
0 7.17519e+07
1 7.08782e+07
2 7.07681e+07
3 7.07355e+07
Now with regularization...
Begin: 6.06823e+07
0 6.06444e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410468641
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410468641
fLifetime: 30000 [us]
12-Nov-2025 18:44:50 GMT  Closed output file "reco_stage1.root"
12-Nov-2025 18:44:50 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          495.432       735.586       1118.01       746.99        171.75         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00204485    0.00425831     0.0072116    0.00388073    0.00166122       10     
reco:ophit:OpHitFinder                             0.382475      0.467082      0.635917      0.445499      0.0654208       10     
reco:opflash:OpFlashFinder                         0.191584      0.267957      0.440433      0.253258      0.0650826       10     
reco:opslicer:OpSlicer                             0.683933       1.11137       1.59666       1.04805      0.253759        10     
reco:caldata:DataPrepModule                         9.69848       10.3543       11.1805       10.3778      0.444901        10     
reco:wclsdatasp:WireCellToolkit                     59.4018       78.2925       88.8275       78.914        6.96768        10     
reco:gaushit:GausHitFinder                          1.66103       2.40691       3.63539       2.25602      0.578069        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000666643   0.000961064   0.00134947    0.000934864   0.000184673      10     
reco:reco3d:SpacePointSolver                        15.5839       25.5015       38.528        23.7736       7.12859        10     
reco:hitpdune:DisambigFromSpacePoints               1.6436        3.7103        6.82892       3.59062       1.39568        10     
reco:pandora:StandardPandora                        28.651        57.6457       115.374       53.5504       24.4476        10     
reco:pandoraWriter:StandardPandora                 0.598884      0.835513       1.13677      0.840992       0.14528        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.0844        13.0012       23.6553       12.5159       4.30374        10     
reco:pandoraShower:LArPandoraShowerCreation         6.12395       14.0515       28.8398       14.2738       6.36418        10     
reco:pandoracalo:Calorimetry                        3.63389       6.04762       10.9538       6.04312       2.09719        10     
reco:pandoracalonosce:Calorimetry                   3.02472       5.7848        10.767        5.69382       2.16334        10     
reco:pandorapid:Chi2ParticleID                    0.00575113      0.01036      0.0158472     0.0108136    0.00261666       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0679741     0.101495      0.163054      0.100881      0.0271141       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00646189    0.00979401     0.0144819     0.010252     0.00222195       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.39985       14.2033       28.3828       14.955        6.52386        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.17921       13.8971       28.0809       14.7404       6.49934        10     
reco:emtrkmichelid:EmTrackMichelId                  268.192       361.243       487.669       361.112       61.145         10     
reco:crttag:SingleCRTMatchingProducer              0.0283405     0.039442      0.0554563     0.037478     0.00897672       10     
reco:crtreco:TwoCRTMatchingProducer                0.0195384     0.0297516     0.043761      0.0282919    0.00706782       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.18354       1.68361       2.26236       1.66721       0.33446        10     
reco:pandora2Track:LArPandoraTrackCreation          17.2811       40.7692       78.2107       41.8349       16.6401        10     
reco:pandora2calo:Calorimetry                       6.8768        18.4315       35.8297       19.3229       8.11103        10     
reco:pandora2calonosce:Calorimetry                  6.85777       17.7099       34.6774       18.4393       7.81166        10     
reco:pandora2pid:Chi2ParticleID                    0.0169555      0.02798      0.0487398     0.0273582    0.00936392       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0846673     0.136926      0.197328      0.144616      0.0329402       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0118626     0.0259949     0.0389806     0.0268081     0.0078786       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.0842        13.008        22.4333       13.2742       4.71665        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.48948       16.0042       27.1057        16.27        6.32886        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.78535       13.9387       24.5757       14.1696       5.74499        10     
[art]:TriggerResults:TriggerResultInserter        9.1939e-05    0.000124047   0.000238548   0.000107215   4.21127e-05      10     
end_path:out1:RootOutput                          1.2605e-05    1.52622e-05    2.144e-05    1.34795e-05   3.20244e-06      10     
end_path:out1:RootOutput(write)                     3.36904       4.64438       6.33765       4.60155      0.888746        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4952.01 MB
  Peak resident set size usage (VmHWM): 4129.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.39 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9911232
-rw-r--r-- 1 dune01 dune01 2344696187 Nov 12 16:38 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2230807486 Nov 12 16:34 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2015868662 Nov 12 18:44 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1783495061 Nov 12 12:55 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1702493795 Nov 12 12:33 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   71114690 Nov 12 18:35 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242959 Nov 12 18:44 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 12 18:44 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 12 18:44 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 12 18:44 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 12 16:38 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 12:55 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 12 18:44 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 12 12:23 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 12 18:44 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 12 16:47 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 12 16:38 debugprod.log
justIN time: 2026-02-10 01:16:54 UTC       justIN version: 01.06.00