justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255202.1@dunegpschedd01.fnal.gov

Jobsub ID255202.1@dunegpschedd01.fnal.gov
Workflow ID10509
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-22 03:26:07
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-22 07:33:50
From worker nodeHostnamewn1905241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 03:27:02
Input filesmonte-carlo-010509-000001
JobscriptExit code0
Real time4h (14712s)
CPU time3h (14159s = 96%)
Max RSS bytes4113043456 (3922 MiB)
Outputting started2025-11-22 07:32:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10509/1/001/reco_stage1.root
Finished2025-11-22 07:33:50
Saved logsjustin-logs:255202.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

alFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113266 traces, 0 summary
Retagger: tagging trace set: gauss with 86716 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 86716 traces tagged "gauss"
FrameSaver: q=2.95382e+06 n=1542514 tag=gauss
wclsFrameSaver: saving 113266 traces tagged "wiener"
FrameSaver: q=3.11103e+06 n=1360181 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 16490 XUs and 12725 XVs -> 136 XUVs
C:0 T:1 33514 XUs and 27042 XVs -> 2442 XUVs
C:0 T:2 67274 XUs and 57199 XVs -> 2470 XUVs
C:0 T:3 37303 XUs and 27730 XVs -> 279 XUVs
C:0 T:4 11982 XUs and 13969 XVs -> 102 XUVs
C:0 T:5 50107 XUs and 51547 XVs -> 7567 XUVs
C:0 T:6 113328 XUs and 89411 XVs -> 4166 XUVs
C:0 T:7 50731 XUs and 36840 XVs -> 318 XUVs
C:0 T:8 8157 XUs and 11113 XVs -> 86 XUVs
C:0 T:9 58411 XUs and 93225 XVs -> 5482 XUVs
C:0 T:10 19918 XUs and 9321 XVs -> 1865 XUVs
C:0 T:11 72 XUs and 41 XVs -> 8 XUVs
24921 XUVs total
10529 collection wire objects
23047 potential space points
Neighbour search...
1116459 tests to find 623628 neighbours
Iterating with no regularization...
Begin: 5.18961e+07
0 4.5193e+07
1 4.46386e+07
2 4.4561e+07
3 4.4542e+07
Now with regularization...
Begin: 3.60554e+07
0 3.60573e+07
Warning: metric increased
run: 1 ; subrun: 0 ; event: 10
evttime: 410662411
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410662411
fLifetime: 30000 [us]
22-Nov-2025 07:32:12 GMT  Closed output file "reco_stage1.root"
22-Nov-2025 07:32:12 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          357.811       448.18        554.707       439.251       61.2494        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00174563    0.00533357     0.0121505     0.0050543    0.00279271       10     
reco:ophit:OpHitFinder                             0.349095      0.382955      0.411268      0.391114      0.0241057       10     
reco:opflash:OpFlashFinder                         0.160625      0.201338      0.238048      0.197897      0.020894        10     
reco:opslicer:OpSlicer                             0.738907      0.874653       1.03334      0.870689      0.0853529       10     
reco:caldata:DataPrepModule                         6.72547       7.61614       7.96974       7.7105       0.333327        10     
reco:wclsdatasp:WireCellToolkit                     44.2329       46.4441       54.2919       45.2528       3.15257        10     
reco:gaushit:GausHitFinder                          1.42505       1.61347       1.93013       1.53149      0.178417        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000608233   0.000737748   0.00108193    0.000697325    0.0001307       10     
reco:reco3d:SpacePointSolver                        11.2394       19.4232       28.7231       19.7181       4.99746        10     
reco:hitpdune:DisambigFromSpacePoints               1.60624       2.13144       2.7833        2.02944      0.371407        10     
reco:pandora:StandardPandora                        26.4235       41.1133       65.4769       39.0854       10.512         10     
reco:pandoraWriter:StandardPandora                 0.506054      0.628015      0.744147       0.63913      0.0739386       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.40218       6.02774       7.01581       6.18453      0.886549        10     
reco:pandoraShower:LArPandoraShowerCreation         4.29788       7.24675       11.9054       6.48122       2.23063        10     
reco:pandoracalo:Calorimetry                        2.51001       3.66819       4.87909       3.80547      0.678006        10     
reco:pandoracalonosce:Calorimetry                   1.86272       3.31539       4.16133       3.42116      0.676621        10     
reco:pandorapid:Chi2ParticleID                     0.007334     0.00821496     0.0107167    0.00786669    0.000916275      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0619332     0.0768399     0.0948932     0.0795086     0.0102795       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0069311     0.0077298    0.00839562    0.00787858    0.000489456      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.11108       7.45445       12.7788       6.97505       2.5125         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.79958       7.18977       12.4815       6.56652       2.54985        10     
reco:emtrkmichelid:EmTrackMichelId                  191.312       226.394       263.802       220.661       21.5144        10     
reco:crttag:SingleCRTMatchingProducer              0.0144181     0.0237858     0.0417508     0.0211213    0.00875857       10     
reco:crtreco:TwoCRTMatchingProducer                0.0102443     0.0147432     0.0193188     0.0140383    0.00280861       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.814012       1.02883       1.23536       1.01441       0.11492        10     
reco:pandora2Track:LArPandoraTrackCreation          12.2538       19.6364       28.8257       18.8557       4.65699        10     
reco:pandora2calo:Calorimetry                       5.92221       9.74035       14.3669       9.45827       2.54724        10     
reco:pandora2calonosce:Calorimetry                  5.74028       9.26996       14.3389       8.53627       2.5791         10     
reco:pandora2pid:Chi2ParticleID                    0.0163171     0.019471      0.022005      0.0197207    0.00174685       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0818555     0.096676      0.117932      0.0937278     0.0123137       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0159183     0.0188681     0.0215454     0.0189827    0.00178498       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.32261       6.92791       11.0489       6.50952       1.93471        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.79078       8.36144       13.3963       7.78795       2.57811        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.90583       7.4718        12.4248       6.80398       2.47174        10     
[art]:TriggerResults:TriggerResultInserter        7.9289e-05    0.000110235   0.000231525   9.19175e-05   4.25171e-05      10     
end_path:out1:RootOutput                          1.2674e-05    1.57686e-05   2.2422e-05    1.5484e-05    2.55663e-06      10     
end_path:out1:RootOutput(write)                     3.06514       3.55995       4.35216       3.48399      0.370604        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4954.84 MB
  Peak resident set size usage (VmHWM): 4113.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.66 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9801240
-rw-r--r-- 1 dune01 dune01 2313053968 Nov 22 06:14 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2196180338 Nov 22 06:11 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1976508115 Nov 22 07:32 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1776562110 Nov 22 03:51 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1703950430 Nov 22 03:35 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   69590200 Nov 22 07:27 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242742 Nov 22 07:32 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 22 07:32 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 22 07:32 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 22 07:32 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 22 06:14 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 22 03:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 22 07:32 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 22 03:27 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 22 07:32 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 22 06:22 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 22 06:14 debugprod.log
justIN time: 2026-02-08 22:54:22 UTC       justIN version: 01.06.00