justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232830.1@dunegpschedd02.fnal.gov

Jobsub ID232830.1@dunegpschedd02.fnal.gov
Workflow ID9328
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-25 07:27:02
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-10-25 11:05:23
From worker nodeHostnamewn2205240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-25 07:28:43
Input filesmonte-carlo-009328-000001
JobscriptExit code0
Real time3h (12911s)
CPU time3h (12303s = 95%)
Max RSS bytes4075573248 (3886 MiB)
Outputting started2025-10-25 11:03:54
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09328/1/001/reco_stage1.root
Finished2025-10-25 11:05:23
Saved logsjustin-logs:232830.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 102387 traces, 0 summary
Retagger: tagging trace set: gauss with 80220 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80220 traces tagged "gauss"
FrameSaver: q=2.7047e+06 n=1438952 tag=gauss
wclsFrameSaver: saving 102387 traces tagged "wiener"
FrameSaver: q=2.86531e+06 n=1286818 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32032 XUs and 28066 XVs -> 451 XUVs
C:0 T:1 63474 XUs and 60096 XVs -> 4197 XUVs
C:0 T:2 102190 XUs and 115408 XVs -> 5510 XUVs
C:0 T:3 44891 XUs and 46823 XVs -> 289 XUVs
C:0 T:4 21438 XUs and 17197 XVs -> 339 XUVs
C:0 T:5 97902 XUs and 68993 XVs -> 6912 XUVs
C:0 T:6 62124 XUs and 60578 XVs -> 2141 XUVs
C:0 T:7 26954 XUs and 25002 XVs -> 102 XUVs
C:0 T:8 3504 XUs and 1768 XVs -> 254 XUVs
C:0 T:9 8992 XUs and 8124 XVs -> 768 XUVs
C:0 T:10 21091 XUs and 8578 XVs -> 2545 XUVs
23508 XUVs total
9929 collection wire objects
21734 potential space points
Neighbour search...
1014614 tests to find 525282 neighbours
Iterating with no regularization...
Begin: 5.04416e+07
0 4.03112e+07
1 3.96316e+07
2 3.95368e+07
3 3.95055e+07
Now with regularization...
Begin: 3.22266e+07
0 3.21944e+07
1 3.2181e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410102381
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410102381
fLifetime: 30000 [us]
25-Oct-2025 12:03:52 BST  Closed output file "reco_stage1.root"
25-Oct-2025 12:03:52 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          240.136       377.542       465.109       385.61        59.596         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00147825    0.00469543    0.00884483    0.00424645    0.00222658       10     
reco:ophit:OpHitFinder                             0.307028      0.352457      0.389706      0.356495      0.0292705       10     
reco:opflash:OpFlashFinder                         0.142204      0.181585      0.220242      0.180766      0.0254995       10     
reco:opslicer:OpSlicer                             0.615548      0.779632      0.991268      0.798221      0.118971        10     
reco:caldata:DataPrepModule                         7.1839        7.43317       7.72096       7.4076       0.156116        10     
reco:wclsdatasp:WireCellToolkit                     39.0266       40.4798       42.2638       40.0886       1.10205        10     
reco:gaushit:GausHitFinder                         0.942036       1.50053       1.79872       1.58617      0.238758        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000445258   0.000631337   0.000914741   0.00061901    0.000117676      10     
reco:reco3d:SpacePointSolver                        10.1439       18.5688       25.997        18.8547       4.33687        10     
reco:hitpdune:DisambigFromSpacePoints              0.823497       1.77084       2.31758       1.84516      0.402921        10     
reco:pandora:StandardPandora                        14.555        36.4973       52.5429       36.0706       9.21469        10     
reco:pandoraWriter:StandardPandora                 0.349877      0.556731      0.737204      0.538951      0.0989348       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.02744       4.54874       6.34776       4.65467       1.04342        10     
reco:pandoraShower:LArPandoraShowerCreation         1.44965       4.37426       6.78167       4.42688       1.37402        10     
reco:pandoracalo:Calorimetry                        1.26816       2.61227       3.80607       2.60332      0.604057        10     
reco:pandoracalonosce:Calorimetry                   1.0972        2.44452       3.55135       2.3671       0.657341        10     
reco:pandorapid:Chi2ParticleID                    0.00487055    0.00758301    0.00908139    0.00762845    0.00112738       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0487921     0.0721858     0.0887047     0.0727461     0.0113048       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00504333    0.00741387    0.00907312    0.00746179    0.00103736       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.58443       4.82875       7.02157       4.99619       1.61397        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.54337       4.58232       6.61703       4.67341       1.47977        10     
reco:emtrkmichelid:EmTrackMichelId                  136.911       199.461       242.933       201.096       28.0183        10     
reco:crttag:SingleCRTMatchingProducer              0.010042      0.0175049     0.022445      0.0182639    0.00356026       10     
reco:crtreco:TwoCRTMatchingProducer                0.0061217     0.0101729     0.0137116    0.00956606    0.00228316       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.630606       0.89048       1.13149      0.897123      0.139247        10     
reco:pandora2Track:LArPandoraTrackCreation          4.74048       12.8976       20.1226       12.7992       3.88795        10     
reco:pandora2calo:Calorimetry                       2.66939       6.80219       10.2739       6.91179       2.0795         10     
reco:pandora2calonosce:Calorimetry                  2.42672       6.4388        9.75974       6.57172       1.97048        10     
reco:pandora2pid:Chi2ParticleID                    0.0114478     0.0170609     0.0224426     0.0165984    0.00269844       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0631401     0.0879442     0.125402      0.081893      0.0167206       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0116946     0.0170256     0.0220933     0.0168992    0.00277944       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.82588       4.66199       7.21561       4.63369       1.45428        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.2317        5.80516       9.61092       5.66597       1.88418        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.9575        5.24491       8.78517       4.96973       1.78519        10     
[art]:TriggerResults:TriggerResultInserter        5.4252e-05    8.00206e-05   0.000168327   7.5076e-05    3.05194e-05      10     
end_path:out1:RootOutput                           9.729e-06    1.35685e-05   1.8605e-05    1.30395e-05   2.78371e-06      10     
end_path:out1:RootOutput(write)                     2.38699       3.45036       4.30174       3.5388       0.529309        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4907.95 MB
  Peak resident set size usage (VmHWM): 4075.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.65 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9330844
-rw-r--r-- 1 dune01 dune01 2227323571 Oct 25 10:57 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2116916668 Oct 25 10:55 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1876307655 Oct 25 12:03 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1668803972 Oct 25 08:51 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1598812797 Oct 25 08:36 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   65989680 Oct 25 11:59 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242980 Oct 25 12:03 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 25 12:03 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 25 12:03 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 25 12:03 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 25 10:58 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 25 08:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 25 12:03 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 25 08:28 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 25 12:03 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 25 11:05 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 25 10:58 debugprod.log
justIN time: 2025-11-03 18:43:36 UTC       justIN version: 01.05.01