justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 248112.1@dunegpschedd02.fnal.gov

Jobsub ID248112.1@dunegpschedd02.fnal.gov
Workflow ID10335
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-18 15:27:52
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-18 22:16:04
From worker nodeHostnamewn1208081.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-18 15:29:09
Input filesmonte-carlo-010335-000001
JobscriptExit code0
Real time6h (24293s)
CPU time6h (23428s = 96%)
Max RSS bytes4238843904 (4042 MiB)
Outputting started2025-11-18 22:14:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10335/1/001/reco_stage1.root
Finished2025-11-18 22:16:04
Saved logsjustin-logs:248112.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

are:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120309 traces, 0 summary
Retagger: tagging trace set: gauss with 93297 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93297 traces tagged "gauss"
FrameSaver: q=3.22249e+06 n=1634972 tag=gauss
wclsFrameSaver: saving 120309 traces tagged "wiener"
FrameSaver: q=3.39411e+06 n=1445811 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30340 XUs and 25460 XVs -> 203 XUVs
C:0 T:1 47376 XUs and 39462 XVs -> 2961 XUVs
C:0 T:2 104461 XUs and 122977 XVs -> 4070 XUVs
C:0 T:3 54021 XUs and 55648 XVs -> 351 XUVs
C:0 T:4 12291 XUs and 8394 XVs -> 109 XUVs
C:0 T:5 40948 XUs and 23531 XVs -> 3546 XUVs
C:0 T:6 84381 XUs and 79006 XVs -> 2816 XUVs
C:0 T:7 34560 XUs and 31906 XVs -> 296 XUVs
C:0 T:8 6129 XUs and 6803 XVs -> 45 XUVs
C:0 T:9 32755 XUs and 51365 XVs -> 3173 XUVs
C:0 T:10 31285 XUs and 18758 XVs -> 3274 XUVs
C:0 T:11 290 XUs and 239 XVs -> 21 XUVs
20865 XUVs total
10087 collection wire objects
19408 potential space points
Neighbour search...
648632 tests to find 345490 neighbours
Iterating with no regularization...
Begin: 8.25013e+07
0 7.43646e+07
1 7.38942e+07
2 7.38461e+07
Now with regularization...
Begin: 6.42106e+07
0 6.42067e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410591921
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410591921
fLifetime: 30000 [us]
18-Nov-2025 22:13:59 GMT  Closed output file "reco_stage1.root"
18-Nov-2025 22:13:59 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          689.576       966.068       1175.75       1021.51       165.075        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00229978     0.318405       3.1057       0.0093085     0.929105        10     
reco:ophit:OpHitFinder                             0.376728      0.456674      0.508921       0.46409      0.0363917       10     
reco:opflash:OpFlashFinder                         0.206615      0.266864      0.308789      0.274749      0.0338881       10     
reco:opslicer:OpSlicer                             0.888144       1.28891       1.70232       1.25864      0.226759        10     
reco:caldata:DataPrepModule                         7.91851       9.51868       10.5726       9.66433      0.831824        10     
reco:wclsdatasp:WireCellToolkit                     63.6461       78.848        92.9431       77.4128       8.74095        10     
reco:gaushit:GausHitFinder                          1.86868       2.61741       3.18026       2.68497      0.416239        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00126363    0.00224638    0.00358993    0.00211334    0.000708427      10     
reco:reco3d:SpacePointSolver                        14.6458       29.561        42.3459       29.5693       7.59848        10     
reco:hitpdune:DisambigFromSpacePoints               2.12812       3.49594       5.41266       3.33129      0.908132        10     
reco:pandora:StandardPandora                        37.6142       81.7312       142.891       80.3073       27.9495        10     
reco:pandoraWriter:StandardPandora                 0.777332       1.15791       1.64017       1.16556      0.236874        10     
reco:pandoraTrack:LArPandoraTrackCreation           10.7408       17.7766       24.7496       18.7116       5.47588        10     
reco:pandoraShower:LArPandoraShowerCreation         12.218        22.5396       34.1876       25.0082       7.62126        10     
reco:pandoracalo:Calorimetry                        6.56399       10.953        15.2614       10.8685       3.24725        10     
reco:pandoracalonosce:Calorimetry                   6.12554       10.6763       15.743        9.5448        3.41224        10     
reco:pandorapid:Chi2ParticleID                     0.0082319     0.0149853     0.0224087     0.0140336    0.00428834       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0699911     0.140401      0.192614       0.14561      0.0416124       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00687141     0.0169007     0.0273331     0.0163962    0.00689721       10     
reco:pandoraShowercalo:ShowerCalorimetry            11.7263       26.1854       38.6218       27.4059       8.68577        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.91391       25.0209       34.5142       27.2762       8.0661         10     
reco:emtrkmichelid:EmTrackMichelId                  361.957       440.895       509.339       451.303       54.3554        10     
reco:crttag:SingleCRTMatchingProducer              0.0257862     0.0488356     0.0714121     0.0499262     0.0144368       10     
reco:crtreco:TwoCRTMatchingProducer                0.0186616     0.0410082     0.0635463     0.0431779     0.0126853       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.0075        1.60664       2.15948       1.62696      0.410205        10     
reco:pandora2Track:LArPandoraTrackCreation          29.748        56.4603       78.0905       61.2865       14.5751        10     
reco:pandora2calo:Calorimetry                       10.8032       32.2849       47.1915       36.4006       10.8987        10     
reco:pandora2calonosce:Calorimetry                  11.5427       32.0108       44.414        35.4465       9.65232        10     
reco:pandora2pid:Chi2ParticleID                    0.0195028     0.0299554     0.0428408     0.0291741    0.00672951       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.115554      0.151027      0.201738      0.143943      0.0308062       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0233528     0.0299284     0.0396386     0.0294911    0.00465862       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.35467       20.0606       25.8403       22.0092       5.33797        10     
reco:pandora2Showercalo:ShowerCalorimetry           10.4796       28.703        36.3401       33.3473       8.35508        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      12.1178       25.3338       34.3802       26.994        7.17211        10     
[art]:TriggerResults:TriggerResultInserter        5.3932e-05    0.000106546   0.00021746    7.62435e-05   5.74095e-05      10     
end_path:out1:RootOutput                           9.719e-06    1.3621e-05    2.0529e-05    1.0896e-05    4.14079e-06      10     
end_path:out1:RootOutput(write)                     4.23866       5.37061       6.19263       5.45611      0.693743        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5095.83 MB
  Peak resident set size usage (VmHWM): 4238.84 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.44 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.83 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10274968
-rw-r--r-- 1 dune01 dune01 2407287293 Nov 18 19:29 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2287743633 Nov 18 19:25 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2094533674 Nov 18 22:13 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1860614200 Nov 18 16:04 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1795626354 Nov 18 15:40 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   75129140 Nov 18 22:02 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242828 Nov 18 22:14 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 18 22:13 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 18 22:13 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 18 22:13 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 18 19:29 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 18 16:04 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 18 22:14 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 18 15:29 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 18 22:14 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 18 19:39 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 18 19:29 debugprod.log
justIN time: 2026-02-09 06:14:12 UTC       justIN version: 01.06.00