justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 229903.1@dunegpschedd02.fnal.gov

Jobsub ID229903.1@dunegpschedd02.fnal.gov
Workflow ID9082
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-17 10:20:52
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-10-17 16:13:33
From worker nodeHostnamehd77.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-17 10:22:40
Input filesmonte-carlo-009082-000001
JobscriptExit code0
Real time5h (20963s)
CPU time5h (20396s = 97%)
Max RSS bytes4446863360 (4240 MiB)
Outputting started2025-10-17 16:12:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09082/1/001/reco_stage1.root
Finished2025-10-17 16:13:33
Saved logsjustin-logs:229903.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 173895 traces, 0 summary
Retagger: tagging trace set: gauss with 131696 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 131696 traces tagged "gauss"
FrameSaver: q=4.64678e+06 n=2295484 tag=gauss
wclsFrameSaver: saving 173895 traces tagged "wiener"
FrameSaver: q=4.90967e+06 n=2013838 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 43769 XUs and 56071 XVs -> 778 XUVs
C:0 T:1 95491 XUs and 105197 XVs -> 6698 XUVs
C:0 T:2 85505 XUs and 87519 XVs -> 4899 XUVs
C:0 T:3 42943 XUs and 38211 XVs -> 330 XUVs
C:0 T:4 19198 XUs and 16484 XVs -> 226 XUVs
C:0 T:5 152242 XUs and 81869 XVs -> 11649 XUVs
C:0 T:6 82615 XUs and 84466 XVs -> 3182 XUVs
C:0 T:7 34539 XUs and 35238 XVs -> 262 XUVs
C:0 T:8 11554 XUs and 11383 XVs -> 159 XUVs
C:0 T:9 101418 XUs and 110064 XVs -> 7630 XUVs
C:0 T:10 47097 XUs and 42456 XVs -> 4963 XUVs
C:0 T:11 111 XUs and 100 XVs -> 8 XUVs
40784 XUVs total
15217 collection wire objects
37451 potential space points
Neighbour search...
2266885 tests to find 1180838 neighbours
Iterating with no regularization...
Begin: 9.94911e+07
0 7.32791e+07
1 7.16296e+07
2 7.14476e+07
3 7.13883e+07
Now with regularization...
Begin: 5.86219e+07
0 5.85776e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 409944029
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409944029
fLifetime: 30000 [us]
17-Oct-2025 16:12:01 UTC  Closed output file "reco_stage1.root"
17-Oct-2025 16:12:01 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          405.814       569.137       711.806       578.163       93.907         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00137466    0.00404154    0.00626835    0.00405416    0.00155277       10     
reco:ophit:OpHitFinder                             0.310479      0.405285      0.494636      0.389641      0.0540116       10     
reco:opflash:OpFlashFinder                         0.198197      0.279739      0.395675      0.267288      0.0564887       10     
reco:opslicer:OpSlicer                             0.938623       1.20605       1.51405       1.18928      0.156019        10     
reco:caldata:DataPrepModule                         7.18358       7.66613       8.6098        7.56066       0.40811        10     
reco:wclsdatasp:WireCellToolkit                     53.1676       56.0435       60.4938       55.5696       2.30737        10     
reco:gaushit:GausHitFinder                          1.3898        1.95928       2.6209        2.00146      0.327753        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000506742   0.000664118   0.000804631   0.000658042   9.44485e-05      10     
reco:reco3d:SpacePointSolver                        9.82258       18.756        24.5312       20.2733       4.52507        10     
reco:hitpdune:DisambigFromSpacePoints               1.43517       2.58687       3.80588       2.65932      0.727583        10     
reco:pandora:StandardPandora                        22.4801       53.9077       81.9832       51.7323       15.1642        10     
reco:pandoraWriter:StandardPandora                 0.467767      0.637914      0.916909      0.619043      0.120025        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.59174       7.12669       11.3641       6.71193       1.80835        10     
reco:pandoraShower:LArPandoraShowerCreation         3.84488       8.2602        14.0399       8.19228       2.68854        10     
reco:pandoracalo:Calorimetry                        2.34945       3.85203       4.95669       3.92818      0.809734        10     
reco:pandoracalonosce:Calorimetry                   2.18458       3.57961       4.76501       3.38826      0.871695        10     
reco:pandorapid:Chi2ParticleID                    0.00736732    0.00903713     0.0110353    0.00884842    0.00109755       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0700359     0.091255      0.113851      0.090825      0.0149331       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00745783    0.00842974    0.00961691    0.00839469    0.00067972       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.03088       8.24228       11.7399       8.85567       2.44045        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.8609        7.93575       11.292        8.36802       2.4198         10     
reco:emtrkmichelid:EmTrackMichelId                  243.482       308.683       379.996       314.698       44.6799        10     
reco:crttag:SingleCRTMatchingProducer              0.0149263      0.02388      0.0289816     0.0250021     0.0045694       10     
reco:crtreco:TwoCRTMatchingProducer                0.0105255     0.015743      0.019845      0.0170591    0.00298909       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.03989       1.36949       1.80834       1.27315       0.2656         10     
reco:pandora2Track:LArPandoraTrackCreation          13.5542       23.1827       32.8627       23.2965       5.84007        10     
reco:pandora2calo:Calorimetry                       5.76485       11.9062       17.7328       12.0311       3.85137        10     
reco:pandora2calonosce:Calorimetry                  5.28127       10.758        17.355        11.2865       3.45541        10     
reco:pandora2pid:Chi2ParticleID                    0.0169898     0.0220769     0.026613      0.0218525    0.00253328       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0840946     0.113692      0.152647      0.111306      0.0195383       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0167941     0.0218851     0.0294676     0.0209401    0.00320379       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.13191       8.53862       12.6884       8.80992       2.1567         10     
reco:pandora2Showercalo:ShowerCalorimetry           5.15138       9.41003       13.3451       10.1398       2.50112        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.65633       8.30474       11.8015       8.99136       2.22614        10     
[art]:TriggerResults:TriggerResultInserter        6.5413e-05    8.48464e-05   0.000182643   7.5482e-05    3.27925e-05      10     
end_path:out1:RootOutput                           1.044e-05    1.27189e-05   2.0358e-05    1.2153e-05    2.74893e-06      10     
end_path:out1:RootOutput(write)                     3.03205       4.08428       5.13368       4.16102      0.650796        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5262.95 MB
  Peak resident set size usage (VmHWM): 4446.86 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.83 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11119600
-rw-r--r-- 1 dnpil001 users 2578538861 Oct 17 14:34 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2448091297 Oct 17 14:30 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2279671798 Oct 17 16:12 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 2043715160 Oct 17 10:52 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1957739863 Oct 17 10:32 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   78099140 Oct 17 16:03 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     242858 Oct 17 16:12 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Oct 17 16:12 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Oct 17 16:12 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Oct 17 16:12 time.db
-rw-r--r-- 1 dnpil001 users        519 Oct 17 14:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Oct 17 10:52 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Oct 17 16:12 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Oct 17 10:22 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Oct 17 16:12 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Oct 17 14:42 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Oct 17 14:34 debugprod.log
justIN time: 2025-11-03 18:30:45 UTC       justIN version: 01.05.01