justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 240904.0@dunegpschedd02.fnal.gov

Jobsub ID240904.0@dunegpschedd02.fnal.gov
Workflow ID9895
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-10 13:27:02
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-10 19:26:28
From worker nodeHostnamewn1208081.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-10 13:27:53
Input filesmonte-carlo-009895-000001
JobscriptExit code0
Real time5h (21408s)
CPU time5h (20722s = 96%)
Max RSS bytes4241240064 (4044 MiB)
Outputting started2025-11-10 19:24:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09895/1/001/reco_stage1.root
Finished2025-11-10 19:26:28
Saved logsjustin-logs:240904.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 89388 traces, 0 summary
Retagger: tagging trace set: gauss with 70788 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 70788 traces tagged "gauss"
FrameSaver: q=2.55785e+06 n=1320638 tag=gauss
wclsFrameSaver: saving 89388 traces tagged "wiener"
FrameSaver: q=2.70741e+06 n=1196134 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23103 XUs and 23665 XVs -> 137 XUVs
C:0 T:1 43643 XUs and 40923 XVs -> 2295 XUVs
C:0 T:2 111196 XUs and 96718 XVs -> 15215 XUVs
C:0 T:3 51436 XUs and 36824 XVs -> 405 XUVs
C:0 T:4 7817 XUs and 8638 XVs -> 48 XUVs
C:0 T:5 28994 XUs and 29660 XVs -> 3507 XUVs
C:0 T:6 43690 XUs and 26072 XVs -> 2283 XUVs
C:0 T:7 14320 XUs and 11919 XVs -> 87 XUVs
C:0 T:8 8349 XUs and 10056 XVs -> 108 XUVs
C:0 T:9 50697 XUs and 72814 XVs -> 4574 XUVs
C:0 T:10 78208 XUs and 17581 XVs -> 5342 XUVs
C:0 T:11 61 XUs and 30 XVs -> 10 XUVs
34011 XUVs total
7936 collection wire objects
31133 potential space points
Neighbour search...
5744269 tests to find 2512628 neighbours
Iterating with no regularization...
Begin: 6.63098e+07
0 5.61589e+07
1 5.54879e+07
2 5.5403e+07
3 5.53804e+07
Now with regularization...
Begin: 4.79729e+07
0 4.79378e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
run: 1 ; subrun: 0 ; event: 10
evttime: 410403777
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410403777
fLifetime: 30000 [us]
11-Nov-2025 01:24:39 GMT  Closed output file "reco_stage1.root"
11-Nov-2025 01:24:39 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          526.753       778.733       1111.88       744.914       171.635        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00188722    0.00559083    0.00873824    0.00583596    0.00204927       10     
reco:ophit:OpHitFinder                             0.374233      0.447705      0.517371      0.454434      0.0472965       10     
reco:opflash:OpFlashFinder                         0.182561      0.259876      0.321872      0.267213      0.0418253       10     
reco:opslicer:OpSlicer                             0.869601       1.25674       1.67648       1.22727      0.224676        10     
reco:caldata:DataPrepModule                         7.24113       8.65282       10.0812       8.86854      0.953646        10     
reco:wclsdatasp:WireCellToolkit                     54.8092       61.0301       68.7746       59.9293       4.44948        10     
reco:gaushit:GausHitFinder                          1.42628       2.35469       3.26598       2.34614      0.489816        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000911147   0.00130462    0.00188058    0.00118831    0.000314174      10     
reco:reco3d:SpacePointSolver                        18.3779       27.4872       54.7977       22.9468       11.2791        10     
reco:hitpdune:DisambigFromSpacePoints               1.88262       3.19524       5.46322       3.08606       1.02196        10     
reco:pandora:StandardPandora                        41.3448       77.0817       134.033       70.6898       29.9501        10     
reco:pandoraWriter:StandardPandora                 0.567888      0.865642       1.30803      0.822755      0.202567        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.75635       10.5243       16.7596       9.67725       3.13209        10     
reco:pandoraShower:LArPandoraShowerCreation         6.23201       14.8645       30.1695       12.6251       6.94057        10     
reco:pandoracalo:Calorimetry                        3.6844        6.06046       9.59175       5.57568       1.82583        10     
reco:pandoracalonosce:Calorimetry                   3.35412       5.69001       9.49028       5.51723       1.99987        10     
reco:pandorapid:Chi2ParticleID                    0.00645743     0.0106612     0.0143855     0.0106003    0.00216166       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0648207     0.109353      0.158136      0.112473      0.0280456       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.006337      0.0105369     0.0128502     0.0107927    0.00192329       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.63035       16.8231       28.0672       16.1553       6.94066        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.30392       16.776        32.4361       15.9881       7.34114        10     
reco:emtrkmichelid:EmTrackMichelId                  305.498       403.395       531.411       395.783       68.6519        10     
reco:crttag:SingleCRTMatchingProducer              0.0171647     0.0340731     0.050735      0.0335855     0.0105552       10     
reco:crtreco:TwoCRTMatchingProducer                0.014407      0.0247005     0.0370087     0.0247533    0.00698627       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.949961       1.50135       2.09136       1.4694       0.356739        10     
reco:pandora2Track:LArPandoraTrackCreation          18.2538       36.7427       57.7206       35.2066       11.6632        10     
reco:pandora2calo:Calorimetry                       8.81059       18.5662       28.6392       19.1903       6.22598        10     
reco:pandora2calonosce:Calorimetry                  7.90541       18.4169       31.9351       17.7744       7.02078        10     
reco:pandora2pid:Chi2ParticleID                    0.0218402     0.0272006     0.0387482     0.025947     0.00493759       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0954872      0.13548      0.202022      0.132098      0.0290992       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0236018     0.0277774     0.0363666     0.027253     0.00382768       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.16713       12.046        17.5716       12.057        3.39617        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.38782       15.3158       28.8313       15.289        6.19525        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.9187        13.517        22.5562       13.2667       5.23037        10     
[art]:TriggerResults:TriggerResultInserter        4.5957e-05    6.8895e-05    0.000190589   5.4057e-05    4.14091e-05      10     
end_path:out1:RootOutput                           7.675e-06    9.2466e-06    1.7273e-05    8.3105e-06    2.73544e-06      10     
end_path:out1:RootOutput(write)                     3.88635       5.20441       6.2429        5.44893      0.771574        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5069.55 MB
  Peak resident set size usage (VmHWM): 4241.24 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.52 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10308352
-rw-r--r-- 1 dune01 dune01 2422927636 Nov 10 23:11 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2301325674 Nov 10 23:07 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2132343882 Nov 11 01:24 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1849368418 Nov 10 19:57 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1768713648 Nov 10 19:37 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   80443130 Nov 11 01:18 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242879 Nov 11 01:24 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 11 01:24 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 11 01:24 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 11 01:24 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 10 23:11 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 10 19:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 11 01:24 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 10 19:27 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 11 01:24 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 10 23:22 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 10 23:11 debugprod.log
justIN time: 2026-02-10 09:58:16 UTC       justIN version: 01.06.00