justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 258506.0@dunegpschedd02.fnal.gov

Jobsub ID258506.0@dunegpschedd02.fnal.gov
Workflow ID11062
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-06 11:33:57
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-12-06 15:45:50
From worker nodeHostnamewn1905240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 11:35:09
Input filesmonte-carlo-011062-000001
JobscriptExit code0
Real time4h (14860s)
CPU time3h (14334s = 96%)
Max RSS bytes4194308096 (4000 MiB)
Outputting started2025-12-06 15:42:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11062/1/001/reco_stage1.root
Finished2025-12-06 15:45:50
Saved logsjustin-logs:258506.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

edRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 155253 traces, 0 summary
Retagger: tagging trace set: gauss with 116595 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 116595 traces tagged "gauss"
FrameSaver: q=4.18641e+06 n=2073015 tag=gauss
wclsFrameSaver: saving 155253 traces tagged "wiener"
FrameSaver: q=4.44904e+06 n=1842505 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29834 XUs and 23339 XVs -> 257 XUVs
C:0 T:1 46524 XUs and 41071 XVs -> 3930 XUVs
C:0 T:2 154902 XUs and 219221 XVs -> 16012 XUVs
C:0 T:3 65067 XUs and 73119 XVs -> 971 XUVs
C:0 T:4 13472 XUs and 15176 XVs -> 150 XUVs
C:0 T:5 61469 XUs and 89949 XVs -> 17275 XUVs
C:0 T:6 98522 XUs and 104745 XVs -> 10457 XUVs
C:0 T:7 31803 XUs and 37427 XVs -> 330 XUVs
C:0 T:8 11143 XUs and 11647 XVs -> 215 XUVs
C:0 T:9 75767 XUs and 99920 XVs -> 7009 XUVs
C:0 T:10 53466 XUs and 48348 XVs -> 7281 XUVs
C:0 T:11 266 XUs and 216 XVs -> 30 XUVs
63917 XUVs total
12947 collection wire objects
59165 potential space points
Neighbour search...
11712567 tests to find 5048690 neighbours
Iterating with no regularization...
Begin: 8.23019e+07
0 6.76167e+07
1 6.64746e+07
2 6.6301e+07
3 6.62464e+07
Now with regularization...
Begin: 5.52216e+07
0 5.51537e+07
1 5.51025e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410949900
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410949900
fLifetime: 30000 [us]
06-Dec-2025 15:42:48 GMT  Closed output file "reco_stage1.root"
06-Dec-2025 15:42:48 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          367.47        445.117       528.696       436.12        50.8662        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00141193    0.00544797     0.0126312    0.00470682    0.00324248       10     
reco:ophit:OpHitFinder                             0.283233      0.405509      0.483769      0.413773      0.0546889       10     
reco:opflash:OpFlashFinder                         0.174582      0.234244      0.335688      0.218653      0.0464939       10     
reco:opslicer:OpSlicer                             0.504811      0.875454       1.1748       0.898188      0.197226        10     
reco:caldata:DataPrepModule                         6.4935        7.29518       7.76322       7.45176      0.444774        10     
reco:wclsdatasp:WireCellToolkit                     38.0954       40.0344       40.785        40.2909      0.817778        10     
reco:gaushit:GausHitFinder                          1.26213       1.61735       2.08037       1.54861      0.238148        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00044149    0.000696912   0.000985833   0.000691559   0.000148632      10     
reco:reco3d:SpacePointSolver                        18.7765       25.2636       38.6048       23.0562       6.66332        10     
reco:hitpdune:DisambigFromSpacePoints               1.79461       2.36599       2.99843       2.30963      0.395334        10     
reco:pandora:StandardPandora                        31.0477       41.9405       54.8738       39.7626       7.79778        10     
reco:pandoraWriter:StandardPandora                 0.448225      0.617955      0.756279      0.624269      0.0930654       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.95364       5.22755       6.82405       5.09258       0.96044        10     
reco:pandoraShower:LArPandoraShowerCreation         3.93857       5.77706       8.09534       5.39623       1.38009        10     
reco:pandoracalo:Calorimetry                        2.42214       3.25139       4.28299        3.112       0.546972        10     
reco:pandoracalonosce:Calorimetry                   2.18733       2.90602       3.70421       2.86461      0.434102        10     
reco:pandorapid:Chi2ParticleID                    0.00645865    0.00817437    0.00975453    0.00823232    0.00111947       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.059154      0.0804196     0.0926422     0.0794553     0.010116        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00623636    0.00794623    0.00957552    0.00751749    0.00111457       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.99502       6.42713       8.9995        5.74656       1.35079        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.72527       6.14238       8.42315       5.36273       1.33888        10     
reco:emtrkmichelid:EmTrackMichelId                  196.14        235.447       267.062       233.002       23.8689        10     
reco:crttag:SingleCRTMatchingProducer              0.0126617     0.0198902     0.0253535     0.0199682    0.00393291       10     
reco:crtreco:TwoCRTMatchingProducer               0.00808876     0.0120849     0.0137074     0.012415     0.00152441       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.801445       1.08728       1.25833       1.12139      0.140402        10     
reco:pandora2Track:LArPandoraTrackCreation          12.0725       16.7162       23.9038       15.0671       3.75141        10     
reco:pandora2calo:Calorimetry                       6.41602       8.62592       12.3375       7.92659       1.88398        10     
reco:pandora2calonosce:Calorimetry                  5.67501       8.33345       11.1378       7.8714        1.74257        10     
reco:pandora2pid:Chi2ParticleID                   0.00981818     0.018649      0.0241141      0.01912     0.00420942       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0520911     0.0942125     0.115568      0.0945724     0.0183008       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00930982     0.018308      0.0274349     0.0183344    0.00513557       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.78375       6.05902       8.03442       6.0957        1.29459        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.66178       7.43387       9.69437       7.22888       1.42976        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.16537       6.84568       8.65083       6.80498       1.13963        10     
[art]:TriggerResults:TriggerResultInserter        4.9873e-05    8.61628e-05   0.000214854   6.87445e-05   4.70142e-05      10     
end_path:out1:RootOutput                           1.082e-05    1.4409e-05    2.4116e-05    1.3265e-05    3.82868e-06      10     
end_path:out1:RootOutput(write)                     3.17881       3.77809       4.48149       3.67157      0.379804        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5022.03 MB
  Peak resident set size usage (VmHWM): 4194.31 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.61 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10390976
-rw-r--r-- 1 dune01 dune01 2433201912 Dec  6 14:26 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2304180026 Dec  6 14:23 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2139514488 Dec  6 15:42 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1882647937 Dec  6 11:58 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1800875539 Dec  6 11:43 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   79313650 Dec  6 15:36 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243104 Dec  6 15:42 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec  6 15:42 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec  6 15:42 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec  6 15:42 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec  6 14:26 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec  6 11:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec  6 15:42 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec  6 11:35 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec  6 15:42 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec  6 14:33 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec  6 14:26 debugprod.log
justIN time: 2026-02-07 17:42:08 UTC       justIN version: 01.06.00