justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243542.0@dunegpschedd01.fnal.gov

Jobsub ID243542.0@dunegpschedd01.fnal.gov
Workflow ID9625
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-05 14:03:28
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-05 18:31:14
From worker nodeHostnamewn-la-04.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-05 14:03:55
Input filesmonte-carlo-009625-000001
JobscriptExit code0
Real time4h (15944s)
CPU time4h (15801s = 99%)
Max RSS bytes4445691904 (4239 MiB)
Outputting started2025-11-05 18:29:40
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09625/1/001/reco_stage1.root
Finished2025-11-05 18:31:14
Saved logsjustin-logs:243542.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 127931 traces, 0 summary
Retagger: tagging trace set: gauss with 97170 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 97170 traces tagged "gauss"
FrameSaver: q=3.5804e+06 n=1758185 tag=gauss
wclsFrameSaver: saving 127931 traces tagged "wiener"
FrameSaver: q=3.79205e+06 n=1572866 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27054 XUs and 20907 XVs -> 249 XUVs
C:0 T:1 38965 XUs and 31427 XVs -> 2946 XUVs
C:0 T:2 59916 XUs and 69321 XVs -> 5562 XUVs
C:0 T:3 31598 XUs and 31709 XVs -> 111 XUVs
C:0 T:4 13164 XUs and 12461 XVs -> 200 XUVs
C:0 T:5 60055 XUs and 41520 XVs -> 6270 XUVs
C:0 T:6 93656 XUs and 111544 XVs -> 5224 XUVs
C:0 T:7 38995 XUs and 39082 XVs -> 397 XUVs
C:0 T:8 11092 XUs and 10828 XVs -> 125 XUVs
C:0 T:9 70820 XUs and 85777 XVs -> 5249 XUVs
C:0 T:10 58420 XUs and 57883 XVs -> 9947 XUVs
C:0 T:11 926 XUs and 960 XVs -> 173 XUVs
36453 XUVs total
10781 collection wire objects
33572 potential space points
Neighbour search...
2798356 tests to find 1480050 neighbours
Iterating with no regularization...
Begin: 6.19117e+07
0 5.11142e+07
1 5.01852e+07
2 5.00378e+07
3 4.99948e+07
Now with regularization...
Begin: 3.96189e+07
0 3.95755e+07
1 3.95475e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410329310
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410329310
fLifetime: 30000 [us]
05-Nov-2025 19:29:38 CET  Closed output file "reco_stage1.root"
05-Nov-2025 19:29:38 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          279.414       504.953       697.219       502.503       107.31         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00170587     0.0068363     0.0108527    0.00736188    0.00308348       10     
reco:ophit:OpHitFinder                             0.344868      0.421451      0.457145      0.420362      0.0308179       10     
reco:opflash:OpFlashFinder                         0.210306       0.29568      0.345364      0.298051      0.0347658       10     
reco:opslicer:OpSlicer                             0.630422      0.858581       1.03422      0.856109      0.105901        10     
reco:caldata:DataPrepModule                         6.12707       6.54607       7.11395       6.49662      0.256061        10     
reco:wclsdatasp:WireCellToolkit                     44.6357       56.7211       63.9129       58.1463       6.2694         10     
reco:gaushit:GausHitFinder                         0.798096       1.47234       1.82753       1.52131      0.295542        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000459526   0.00100974    0.00148326    0.00101689    0.000343288      10     
reco:reco3d:SpacePointSolver                        8.95363       20.6298       36.3255       21.6927       8.10016        10     
reco:hitpdune:DisambigFromSpacePoints              0.790698       2.00248       3.11607       1.92462      0.618188        10     
reco:pandora:StandardPandora                        14.2056       47.4209       73.6085       47.9484       16.6914        10     
reco:pandoraWriter:StandardPandora                 0.393299      0.588406      0.751455      0.591349      0.0963159       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.71352       9.48508       15.1741       9.19066       2.71078        10     
reco:pandoraShower:LArPandoraShowerCreation         4.35603       11.8664       18.4912       11.9405       4.18411        10     
reco:pandoracalo:Calorimetry                        2.45871       4.7098        7.52097       4.80628       1.33449        10     
reco:pandoracalonosce:Calorimetry                   2.67182       4.5851        7.26716       4.48326       1.26716        10     
reco:pandorapid:Chi2ParticleID                    0.00501586    0.00684901    0.00893601    0.00697196    0.00110273       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0375979     0.0584898     0.0807963     0.0578406     0.0102284       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00581364     0.0067698    0.00785224    0.00685226    0.000698141      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.5185        10.3535       18.5721       11.0475       3.95702        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.16063       9.80912       16.7852       9.7174        3.63342        10     
reco:emtrkmichelid:EmTrackMichelId                  129.564       225.79        280.557       230.005       39.501         10     
reco:crttag:SingleCRTMatchingProducer              0.0154737     0.0268113     0.0478049     0.0232341    0.00954892       10     
reco:crtreco:TwoCRTMatchingProducer                0.0114579     0.0201051     0.0305869     0.0191979    0.00515866       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.658781       1.09374       1.4535        1.08791      0.184502        10     
reco:pandora2Track:LArPandoraTrackCreation          11.9053       29.7251       49.4658       29.1323       10.3905        10     
reco:pandora2calo:Calorimetry                       3.85909       13.2462       21.8415       12.7004       4.74807        10     
reco:pandora2calonosce:Calorimetry                  3.32132       12.4324       19.1034       13.0348       4.1326         10     
reco:pandora2pid:Chi2ParticleID                   0.00935759     0.0155039     0.019686      0.0152728    0.00282996       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0437954     0.0753661     0.0918681     0.0780136     0.0125275       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00937018     0.0157086     0.0202459     0.0157622    0.00288241       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.6936        9.83679       16.2906       9.6917        3.41949        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.90994       11.4481       19.0642       12.1278       4.12412        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.76803       9.52876       15.2497       9.65597       3.23047        10     
[art]:TriggerResults:TriggerResultInserter        4.8152e-05    6.79101e-05   0.000162372   5.6154e-05    3.24281e-05      10     
end_path:out1:RootOutput                           7.922e-06    9.5524e-06    1.5704e-05    8.8585e-06    2.11864e-06      10     
end_path:out1:RootOutput(write)                     2.08984       3.55616       4.5928        3.58554      0.622376        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5246.9 MB
  Peak resident set size usage (VmHWM): 4445.69 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.06 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10439736
-rw-r--r--. 1 dune009 dune 2456681581 Nov  5 18:03 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2331313317 Nov  5 18:00 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2152043972 Nov  5 19:29 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1873764300 Nov  5 15:22 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1797958837 Nov  5 15:11 g4_stage1.root
-rw-r--r--. 1 dune009 dune   77915550 Nov  5 19:23 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243024 Nov  5 19:29 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  5 19:29 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  5 19:29 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  5 19:29 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  5 18:03 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  5 15:22 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  5 19:29 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  5 15:03 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  5 19:29 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  5 18:11 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  5 18:03 debugprod.log
justIN time: 2026-02-11 06:13:32 UTC       justIN version: 01.06.00