justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 246858.1@dunegpschedd02.fnal.gov

Jobsub ID246858.1@dunegpschedd02.fnal.gov
Workflow ID10243
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 04:28:32
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2025-11-16 06:23:26
From worker nodeHostnamedunegli-47944-0-cmswn4057.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 04:30:53
Input filesmonte-carlo-010243-000001
JobscriptExit code0
Real time1h (6738s)
CPU time1h (6697s = 99%)
Max RSS bytes4059152384 (3871 MiB)
Outputting started2025-11-16 06:23:11
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10243/1/001/reco_stage1.root
Finished2025-11-16 06:23:26
Saved logsjustin-logs:246858.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 101489 traces, 0 summary
Retagger: tagging trace set: gauss with 79073 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 79073 traces tagged "gauss"
FrameSaver: q=2.75536e+06 n=1416072 tag=gauss
wclsFrameSaver: saving 101489 traces tagged "wiener"
FrameSaver: q=2.89616e+06 n=1257253 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26966 XUs and 23596 XVs -> 178 XUVs
C:0 T:1 55984 XUs and 52455 XVs -> 9431 XUVs
C:0 T:2 50881 XUs and 61024 XVs -> 2738 XUVs
C:0 T:3 29911 XUs and 28443 XVs -> 93 XUVs
C:0 T:4 11373 XUs and 12791 XVs -> 127 XUVs
C:0 T:5 45488 XUs and 53114 XVs -> 7906 XUVs
C:0 T:6 70039 XUs and 68254 XVs -> 2240 XUVs
C:0 T:7 31317 XUs and 28592 XVs -> 199 XUVs
C:0 T:8 6421 XUs and 8284 XVs -> 54 XUVs
C:0 T:9 42148 XUs and 73491 XVs -> 5199 XUVs
C:0 T:10 19066 XUs and 13030 XVs -> 3017 XUVs
C:0 T:11 20 XUs and 24 XVs -> 6 XUVs
31188 XUVs total
9159 collection wire objects
29530 potential space points
Neighbour search...
4218292 tests to find 2068034 neighbours
Iterating with no regularization...
Begin: 7.34478e+07
0 6.65225e+07
1 6.60409e+07
2 6.5983e+07
Now with regularization...
Begin: 5.76065e+07
0 5.7573e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410542583
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410542583
fLifetime: 30000 [us]
16-Nov-2025 06:23:10 UTC  Closed output file "reco_stage1.root"
16-Nov-2025 06:23:10 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          150.372       196.767       297.048       186.375       44.0669        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000961215   0.00241991    0.00357017    0.00254233    0.000918821      10     
reco:ophit:OpHitFinder                             0.177935      0.199755      0.236336      0.192773      0.0182286       10     
reco:opflash:OpFlashFinder                         0.0936913     0.118341       0.14612      0.117228      0.0165554       10     
reco:opslicer:OpSlicer                             0.318493      0.424486      0.504726      0.435164      0.0519645       10     
reco:caldata:DataPrepModule                         3.33289       3.40498       3.5671        3.3986       0.0650084       10     
reco:wclsdatasp:WireCellToolkit                     21.4612       22.2426       22.9276       22.2196      0.501362        10     
reco:gaushit:GausHitFinder                         0.520714       0.68277      0.839103      0.644814      0.117497        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000273046   0.000358109   0.00059355    0.000329327   9.5839e-05       10     
reco:reco3d:SpacePointSolver                        4.77223       9.26889       19.6001       8.51231       4.10544        10     
reco:hitpdune:DisambigFromSpacePoints              0.582701      0.920807       1.3665       0.842135      0.260517        10     
reco:pandora:StandardPandora                        8.9411        22.0078       69.8229       15.8418       17.1506        10     
reco:pandoraWriter:StandardPandora                 0.184263      0.231805      0.303328      0.224185      0.0411243       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.28912       1.85834       3.00984       1.85753      0.512015        10     
reco:pandoraShower:LArPandoraShowerCreation         1.22954       2.04036       3.40818       1.75066      0.770121        10     
reco:pandoracalo:Calorimetry                       0.756736       1.11205       1.77352       1.08777      0.313368        10     
reco:pandoracalonosce:Calorimetry                  0.679448       1.02418       1.65991       1.00331      0.301274        10     
reco:pandorapid:Chi2ParticleID                    0.00257111    0.00328644    0.00476122    0.00313069    0.000604252      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0234022     0.0302702     0.0362667     0.0300631    0.00410196       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00245833    0.00287629     0.0036652    0.00279997    0.000352043      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.2325        2.29735       3.88197       2.02711      0.943132        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.20118       2.24571       3.82878       1.97627      0.934635        10     
reco:emtrkmichelid:EmTrackMichelId                  85.7005       103.96        129.862       101.448       15.8471        10     
reco:crttag:SingleCRTMatchingProducer             0.00493206    0.00739045     0.011452     0.00710599    0.00180351       10     
reco:crtreco:TwoCRTMatchingProducer               0.00350061     0.0045895    0.00563424    0.00469117    0.000676587      10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.4122       0.539093      0.664958      0.556154      0.0729819       10     
reco:pandora2Track:LArPandoraTrackCreation          3.6706        6.02763       9.72691       5.33768       2.14467        10     
reco:pandora2calo:Calorimetry                       1.90198       3.25052       5.2998        2.9568        1.19721        10     
reco:pandora2calonosce:Calorimetry                  1.79107       3.11119       5.09299       2.83362       1.16463        10     
reco:pandora2pid:Chi2ParticleID                    0.0055294    0.00740406     0.0098811    0.00724427    0.00131999       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0306051     0.0370275     0.0460722     0.034568     0.00556897       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00543315    0.00722544     0.0097598    0.00697811    0.00133588       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.46679       2.28104       3.63933       2.04069       0.77404        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.66955       2.90349       4.73209       2.67493       1.08246        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.55791       2.69622       4.37538       2.49763       1.01325        10     
[art]:TriggerResults:TriggerResultInserter         3.168e-05    4.76722e-05   0.000168338   3.39245e-05   4.02742e-05      10     
end_path:out1:RootOutput                           9.228e-06    1.36379e-05   2.0979e-05     1.292e-05    2.81196e-06      10     
end_path:out1:RootOutput(write)                     1.43614       1.76737       2.24103       1.73474      0.285462        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4877.87 MB
  Peak resident set size usage (VmHWM): 4059.15 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9499836
-rw-r--r-- 1 dunegli fnalgrid 2258240459 Nov 16 05:49 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2143260787 Nov 16 05:48 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1934564409 Nov 16 06:23 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1693803771 Nov 16 04:40 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1625623791 Nov 16 04:33 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   71722660 Nov 16 06:21 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242666 Nov 16 06:23 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Nov 16 06:23 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Nov 16 06:23 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Nov 16 06:23 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 05:49 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 04:40 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Nov 16 06:23 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Nov 16 04:30 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Nov 16 06:23 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:52 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Nov 16 05:49 debugprod.log
justIN time: 2026-02-09 10:58:18 UTC       justIN version: 01.06.00