justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294538.1@dunegpschedd02.fnal.gov

Jobsub ID294538.1@dunegpschedd02.fnal.gov
Workflow ID12366
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-24 11:01:35
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2026-01-24 13:31:22
From worker nodeHostnameb9p35p0045.cern.ch
cpuinfoAMD EPYC 9655P 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-24 11:02:37
Input filesmonte-carlo-012366-000001
JobscriptExit code0
Real time2h (8846s)
CPU time2h (7833s = 88%)
Max RSS bytes4218724352 (4023 MiB)
Outputting started2026-01-24 13:30:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12366/1/001/reco_stage1.root
Finished2026-01-24 13:31:22
Saved logsjustin-logs:294538.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113067 traces, 0 summary
Retagger: tagging trace set: gauss with 90755 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90755 traces tagged "gauss"
FrameSaver: q=3.25517e+06 n=1642577 tag=gauss
wclsFrameSaver: saving 113067 traces tagged "wiener"
FrameSaver: q=3.42245e+06 n=1477671 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 19816 XUs and 28070 XVs -> 102 XUVs
C:0 T:1 32980 XUs and 37151 XVs -> 2873 XUVs
C:0 T:2 54780 XUs and 56642 XVs -> 3129 XUVs
C:0 T:3 33927 XUs and 28074 XVs -> 105 XUVs
C:0 T:4 6811 XUs and 6928 XVs -> 30 XUVs
C:0 T:5 20216 XUs and 16029 XVs -> 2106 XUVs
C:0 T:6 67163 XUs and 64844 XVs -> 2179 XUVs
C:0 T:7 31032 XUs and 28507 XVs -> 176 XUVs
C:0 T:8 10885 XUs and 11751 XVs -> 141 XUVs
C:0 T:9 81012 XUs and 100302 XVs -> 5302 XUVs
C:0 T:10 82525 XUs and 59657 XVs -> 10507 XUVs
C:0 T:11 102 XUs and 64 XVs -> 6 XUVs
26656 XUVs total
9506 collection wire objects
25054 potential space points
Neighbour search...
2435856 tests to find 1107186 neighbours
Iterating with no regularization...
Begin: 7.27561e+07
0 6.36558e+07
1 6.30451e+07
2 6.29603e+07
3 6.2936e+07
Now with regularization...
Begin: 5.2584e+07
0 5.25702e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411936118
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411936118
fLifetime: 30000 [us]
24-Jan-2026 14:30:02 CET  Closed output file "reco_stage1.root"
24-Jan-2026 14:30:02 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          221.125       271.107       345.243       257.055       39.716         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000931893   0.00320504    0.00516704    0.00320128    0.00135705       10     
reco:ophit:OpHitFinder                             0.207499      0.269844       0.32321      0.264585      0.035387        10     
reco:opflash:OpFlashFinder                         0.113818      0.158452      0.196153      0.156057      0.0250434       10     
reco:opslicer:OpSlicer                             0.440252      0.621862      0.764491      0.612593       0.09541        10     
reco:caldata:DataPrepModule                          3.369        4.23001       5.00927       4.41416      0.619241        10     
reco:wclsdatasp:WireCellToolkit                     23.3432       25.4733       27.7192       25.3779       1.27513        10     
reco:gaushit:GausHitFinder                         0.693727       0.99554       1.46974      0.931692      0.224615        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000238211   0.00033111    0.000501702   0.000327922   8.58405e-05      10     
reco:reco3d:SpacePointSolver                        7.13501       10.2169       13.9938       9.55254       2.18404        10     
reco:hitpdune:DisambigFromSpacePoints              0.870757       1.35784       1.9996        1.25471      0.348292        10     
reco:pandora:StandardPandora                        16.4373       27.6776       40.165        26.5844       6.38952        10     
reco:pandoraWriter:StandardPandora                 0.266886      0.385021      0.654415      0.327044      0.118613        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.31188       3.19078       4.51199       3.15522      0.702329        10     
reco:pandoraShower:LArPandoraShowerCreation         2.37527       3.49305       5.25227       3.24354       1.02003        10     
reco:pandoracalo:Calorimetry                        1.18746       1.78849       2.1774        1.84179      0.322134        10     
reco:pandoracalonosce:Calorimetry                   1.14048       1.66077       2.60473       1.61555      0.403373        10     
reco:pandorapid:Chi2ParticleID                    0.00328186    0.00489514    0.00667488     0.0049659    0.00107297       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0302686     0.0526957     0.0749707     0.0557622     0.0150406       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00320341    0.00473683    0.00606358    0.00503632    0.000875118      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.35632       3.89961       7.10639       3.14231       1.56234        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.26379       3.63032       5.88054       3.19005       1.15895        10     
reco:emtrkmichelid:EmTrackMichelId                  126.577       144.699       176.509       139.489       15.6961        10     
reco:crttag:SingleCRTMatchingProducer              0.0101518     0.0125077     0.0164531     0.0122612    0.00168814       10     
reco:crtreco:TwoCRTMatchingProducer               0.00660906    0.00810961     0.0121164    0.00786209    0.00144287       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.473734      0.554077      0.654867      0.556847      0.0565704       10     
reco:pandora2Track:LArPandoraTrackCreation          8.58731       11.4231       19.1233       9.5834        3.61998        10     
reco:pandora2calo:Calorimetry                       4.14973       5.22804       7.32014       4.59607       1.11449        10     
reco:pandora2calonosce:Calorimetry                  3.52471       5.01043       8.08665       4.20569       1.59916        10     
reco:pandora2pid:Chi2ParticleID                   0.00768281     0.0116637     0.0167218     0.0112694    0.00298655       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0388628     0.0640587     0.0960443     0.0654384     0.0196608       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00780579     0.0118033     0.0170223     0.0116197    0.00297533       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.60268       3.85655       6.60465       3.01968       1.43034        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.08781       4.54251       7.18823       4.05359       1.39122        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.78887       3.88671       6.48296       3.63488       1.17588        10     
[art]:TriggerResults:TriggerResultInserter         2.64e-05     3.71081e-05    8.91e-05     3.2155e-05    1.74929e-05      10     
end_path:out1:RootOutput                           5.42e-06      6.328e-06     1.01e-05       5.9e-06     1.30784e-06      10     
end_path:out1:RootOutput(write)                     2.06643       2.55782       3.62353       2.36786      0.458063        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5075.49 MB
  Peak resident set size usage (VmHWM): 4218.72 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.70 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10562460
-rw-r--r--. 1 duneprd np-comp 2466176589 Jan 24 13:42 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2339215216 Jan 24 13:40 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2158080302 Jan 24 14:30 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1919613560 Jan 24 12:31 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1855972210 Jan 24 12:09 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   76285900 Jan 24 14:27 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243164 Jan 24 14:30 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 24 14:30 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 24 14:30 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 24 14:30 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 13:42 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 12:31 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 24 14:30 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 24 12:02 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 24 14:30 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 13:47 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 24 13:42 debugprod.log
justIN time: 2026-02-05 01:20:13 UTC       justIN version: 01.06.00