justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241903.1@dunegpschedd01.fnal.gov

Jobsub ID241903.1@dunegpschedd01.fnal.gov
Workflow ID9518
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 07:19:14
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-02 13:48:48
From worker nodeHostnamewn1205081.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 07:20:05
Input filesmonte-carlo-009518-000001
JobscriptExit code0
Real time6h (23207s)
CPU time6h (22391s = 96%)
Max RSS bytes4461023232 (4254 MiB)
Outputting started2025-11-02 13:46:53
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09518/1/001/reco_stage1.root
Finished2025-11-02 13:48:48
Saved logsjustin-logs:241903.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 117955 traces, 0 summary
Retagger: tagging trace set: gauss with 90420 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90420 traces tagged "gauss"
FrameSaver: q=3.00431e+06 n=1583222 tag=gauss
wclsFrameSaver: saving 117955 traces tagged "wiener"
FrameSaver: q=3.18285e+06 n=1399901 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 21680 XUs and 17137 XVs -> 142 XUVs
C:0 T:1 36572 XUs and 33659 XVs -> 3080 XUVs
C:0 T:2 93581 XUs and 117752 XVs -> 4478 XUVs
C:0 T:3 46856 XUs and 50296 XVs -> 261 XUVs
C:0 T:4 8235 XUs and 7434 XVs -> 24 XUVs
C:0 T:5 21339 XUs and 14145 XVs -> 1767 XUVs
C:0 T:6 108794 XUs and 145572 XVs -> 6478 XUVs
C:0 T:7 41624 XUs and 51549 XVs -> 438 XUVs
C:0 T:8 7196 XUs and 8716 XVs -> 338 XUVs
C:0 T:9 68240 XUs and 147840 XVs -> 20283 XUVs
C:0 T:10 24186 XUs and 15176 XVs -> 3024 XUVs
C:0 T:11 184 XUs and 94 XVs -> 19 XUVs
40332 XUVs total
10315 collection wire objects
37119 potential space points
Neighbour search...
7217337 tests to find 2969986 neighbours
Iterating with no regularization...
Begin: 5.8141e+07
0 4.25404e+07
1 4.16166e+07
2 4.14807e+07
3 4.14347e+07
4 4.14126e+07
Now with regularization...
Begin: 3.32751e+07
0 3.32266e+07
1 3.31973e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410263113
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410263113
fLifetime: 30000 [us]
02-Nov-2025 13:46:50 GMT  Closed output file "reco_stage1.root"
02-Nov-2025 13:46:50 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          508.688       834.886       1050.03       883.895       171.025        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00205103    0.00739076     0.019446     0.00667603     0.0047221       10     
reco:ophit:OpHitFinder                             0.422682      0.481657      0.552984      0.474396      0.0431173       10     
reco:opflash:OpFlashFinder                         0.209424      0.315045      0.408007      0.312381       0.06293        10     
reco:opslicer:OpSlicer                              1.04118       1.32839       1.53413       1.32124      0.129781        10     
reco:caldata:DataPrepModule                         8.10733       8.97429       9.40519       9.07607      0.375875        10     
reco:wclsdatasp:WireCellToolkit                     57.7934       64.3159       67.7235       66.9975       4.00763        10     
reco:gaushit:GausHitFinder                          1.63586       2.36037       2.86395       2.54529      0.409011        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000923201   0.00155395    0.00259496    0.00139017    0.000451357      10     
reco:reco3d:SpacePointSolver                        15.4117       27.3708       38.0843       27.2455       6.38792        10     
reco:hitpdune:DisambigFromSpacePoints               1.27442       3.40838       5.02572       3.58702       1.06242        10     
reco:pandora:StandardPandora                        31.9661       83.3878       122.538       83.6621       29.5597        10     
reco:pandoraWriter:StandardPandora                 0.546728      0.980571       1.22734      0.999926      0.188239        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.3835        14.1274       19.9735       15.1928       4.03069        10     
reco:pandoraShower:LArPandoraShowerCreation         7.44337       16.9874       24.0273       18.7505       5.46146        10     
reco:pandoracalo:Calorimetry                        3.07371       8.60487       14.1659       9.18792       3.24094        10     
reco:pandoracalonosce:Calorimetry                   3.58073       8.06804       11.5185       8.73938       2.54227        10     
reco:pandorapid:Chi2ParticleID                    0.00975547     0.013173      0.017975      0.0128258    0.00228433       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0739224     0.111442      0.139363       0.1138       0.0180534       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00932599     0.012585      0.0146553     0.0127764    0.00162931       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.28566       18.7507       27.0748       20.2931       6.59392        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.06985       18.6213       26.7978       20.2207       6.55731        10     
reco:emtrkmichelid:EmTrackMichelId                  274.814       399.058       483.446       418.703       63.6835        10     
reco:crttag:SingleCRTMatchingProducer              0.0281915     0.036559      0.0504273     0.0360986    0.00627295       10     
reco:crtreco:TwoCRTMatchingProducer                0.0205757     0.0312854     0.0408147     0.0322648    0.00687459       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.09899       1.54196       1.84793       1.62169      0.249391        10     
reco:pandora2Track:LArPandoraTrackCreation          21.1505       44.5923       56.0325       48.1384       12.0522        10     
reco:pandora2calo:Calorimetry                       10.2325       26.0125       37.112        27.5802       8.67821        10     
reco:pandora2calonosce:Calorimetry                  9.81175       24.3775       33.7488       26.332        7.66983        10     
reco:pandora2pid:Chi2ParticleID                    0.0217059     0.0309346     0.0386203     0.0304502    0.00479927       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.101617      0.148411      0.180869      0.154101      0.0242982       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0240618     0.0306027     0.0380092     0.0298691    0.00435363       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.1711        14.3424       17.0406       15.5094       3.0033         10     
reco:pandora2Showercalo:ShowerCalorimetry           8.34725       21.9226       31.6729       23.9069       7.14314        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.67294       18.9993       28.2785       19.944        5.87489        10     
[art]:TriggerResults:TriggerResultInserter        4.4023e-05    7.37793e-05   0.000213803    5.534e-05    4.77198e-05      10     
end_path:out1:RootOutput                           6.773e-06    1.01371e-05   2.3795e-05     8.912e-06    4.64046e-06      10     
end_path:out1:RootOutput(write)                     3.53373       5.22533       6.29498       5.55363      0.844685        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5309.97 MB
  Peak resident set size usage (VmHWM): 4461.02 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.75 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11062008
-rw-r--r-- 1 dune01 dune01 2567307595 Nov  2 11:23 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2439724252 Nov  2 11:19 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2261499870 Nov  2 13:46 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2028058185 Nov  2 07:55 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1952812951 Nov  2 07:31 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   77466640 Nov  2 13:37 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242661 Nov  2 13:46 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  2 13:46 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  2 13:46 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  2 13:46 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  2 11:23 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 07:55 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 13:46 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  2 07:20 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  2 13:46 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  2 11:37 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  2 11:23 debugprod.log
justIN time: 2025-11-04 01:35:48 UTC       justIN version: 01.05.01