justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301417.0@dunegpschedd01.fnal.gov

Jobsub ID301417.0@dunegpschedd01.fnal.gov
Workflow ID12426
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-27 08:23:09
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce511
Last heartbeat2026-01-27 12:18:25
From worker nodeHostnameb9p08p9966.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 08:23:59
Input filesmonte-carlo-012426-000001
JobscriptExit code0
Real time3h (13976s)
CPU time3h (12704s = 90%)
Max RSS bytes4301242368 (4101 MiB)
Outputting started2026-01-27 12:16:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12426/1/001/reco_stage1.root
Finished2026-01-27 12:18:25
Saved logsjustin-logs:301417.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Service:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131186 traces, 0 summary
Retagger: tagging trace set: gauss with 95046 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95046 traces tagged "gauss"
FrameSaver: q=3.45587e+06 n=1609144 tag=gauss
wclsFrameSaver: saving 131186 traces tagged "wiener"
FrameSaver: q=3.68113e+06 n=1399841 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24189 XUs and 30445 XVs -> 235 XUVs
C:0 T:1 48718 XUs and 68859 XVs -> 5618 XUVs
C:0 T:2 91009 XUs and 72999 XVs -> 5594 XUVs
C:0 T:3 46255 XUs and 29295 XVs -> 211 XUVs
C:0 T:4 9487 XUs and 14780 XVs -> 92 XUVs
C:0 T:5 38864 XUs and 64964 XVs -> 6435 XUVs
C:0 T:6 100103 XUs and 68608 XVs -> 4311 XUVs
C:0 T:7 42173 XUs and 26999 XVs -> 243 XUVs
C:0 T:8 8301 XUs and 7805 XVs -> 153 XUVs
C:0 T:9 44328 XUs and 51420 XVs -> 3426 XUVs
C:0 T:10 64273 XUs and 74974 XVs -> 11850 XUVs
C:0 T:11 72 XUs and 44 XVs -> 10 XUVs
38178 XUVs total
10544 collection wire objects
35237 potential space points
Neighbour search...
4662589 tests to find 2109308 neighbours
Iterating with no regularization...
Begin: 6.41076e+07
0 5.0354e+07
1 4.93433e+07
2 4.91821e+07
3 4.91339e+07
Now with regularization...
Begin: 3.79679e+07
0 3.79443e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411994256
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411994256
fLifetime: 30000 [us]
27-Jan-2026 13:16:54 CET  Closed output file "reco_stage1.root"
27-Jan-2026 13:16:54 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          278.125       361.724       448.283       347.604       49.2182        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00109413    0.00424004    0.00693533    0.00443853    0.00204175       10     
reco:ophit:OpHitFinder                             0.252158      0.329032      0.387813      0.326334      0.0448064       10     
reco:opflash:OpFlashFinder                         0.121599      0.178168      0.215642      0.183668      0.0337742       10     
reco:opslicer:OpSlicer                             0.631288      0.794315      0.964022      0.779765      0.109887        10     
reco:caldata:DataPrepModule                         4.11765       6.08158       6.94505       6.61325      0.950108        10     
reco:wclsdatasp:WireCellToolkit                     27.5203       34.638        40.5815       35.0091       4.16727        10     
reco:gaushit:GausHitFinder                         0.756567       1.21012       1.92505       1.14782      0.300821        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000316714   0.000563042   0.000988524   0.000559968   0.000173397      10     
reco:reco3d:SpacePointSolver                        8.49205       16.9383       27.9555       16.0118       5.58491        10     
reco:hitpdune:DisambigFromSpacePoints              0.975667       1.80182       2.82909       1.80195      0.550559        10     
reco:pandora:StandardPandora                        19.3265       32.7543       53.0513       33.2221       10.4342        10     
reco:pandoraWriter:StandardPandora                 0.298035      0.487395      0.625666      0.486926      0.110318        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.58093       3.56459       5.01572       3.35421       0.73643        10     
reco:pandoraShower:LArPandoraShowerCreation         2.52289       3.90674       5.6497        3.90994      0.902538        10     
reco:pandoracalo:Calorimetry                        1.60251       2.10826       2.79601       2.09836      0.339529        10     
reco:pandoracalonosce:Calorimetry                   1.32096       1.88564       2.31019       1.7948        0.31509        10     
reco:pandorapid:Chi2ParticleID                    0.00349785    0.00595167    0.00952686    0.00563352    0.00154982       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0406033     0.0579245     0.0731874     0.0604655     0.0109698       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00360258    0.00526955    0.00710641    0.00496655    0.00117394       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.96842       4.34542       6.23878       4.09314      0.974145        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.63384       4.26657       5.91908       4.33193      0.928192        10     
reco:emtrkmichelid:EmTrackMichelId                  161.946       201.649       235.364       198.54        20.3001        10     
reco:crttag:SingleCRTMatchingProducer             0.00902011     0.0160298     0.0305782     0.0132785     0.0067633       10     
reco:crtreco:TwoCRTMatchingProducer               0.00603907    0.00932214     0.0133586    0.00935397    0.00208473       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.694431      0.931733       1.19419      0.938429      0.152161        10     
reco:pandora2Track:LArPandoraTrackCreation          8.24316       12.0769       18.5047       10.7829       3.23914        10     
reco:pandora2calo:Calorimetry                       4.52038       6.46634       9.84992       5.83963       1.74966        10     
reco:pandora2calonosce:Calorimetry                  4.35564       6.1331        8.99416       5.77939       1.51394        10     
reco:pandora2pid:Chi2ParticleID                    0.0149035     0.0180762     0.0230887     0.0173808    0.00249789       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0616309     0.0884925     0.120802      0.087087      0.0173484       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00891373     0.0157332     0.021415      0.0158205    0.00366743       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.29802       4.55667       5.70084       4.72626      0.793757        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.04313       5.83757       8.23224       5.63173       1.38256        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.64275       5.36358       7.8536        5.21533       1.34325        10     
[art]:TriggerResults:TriggerResultInserter         3.619e-05    7.0161e-05    0.000167863   6.13205e-05   3.43464e-05      10     
end_path:out1:RootOutput                           1.051e-05    1.27521e-05    1.852e-05    1.2185e-05    2.09146e-06      10     
end_path:out1:RootOutput(write)                     2.22797       3.09561       4.05637       3.05158      0.603143        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5159.41 MB
  Peak resident set size usage (VmHWM): 4301.24 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.83 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10397392
-rw-r--r--. 1 duneprd np-comp 2424715116 Jan 27 12:12 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2300650178 Jan 27 12:10 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2129346190 Jan 27 13:16 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1896520389 Jan 27 09:57 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1816693506 Jan 27 09:33 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   78389430 Jan 27 13:12 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242684 Jan 27 13:16 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 27 13:16 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 27 13:16 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 27 13:16 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 27 12:12 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 27 09:57 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 27 13:16 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 27 09:24 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 27 13:16 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 27 12:21 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 27 12:12 debugprod.log
justIN time: 2026-02-04 19:29:15 UTC       justIN version: 01.06.00