justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302559.1@dunegpschedd01.fnal.gov

Jobsub ID302559.1@dunegpschedd01.fnal.gov
Workflow ID12606
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-31 08:46:03
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce503
Last heartbeat2026-01-31 12:12:53
From worker nodeHostnameb9p11p0303.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-31 08:47:00
Input filesmonte-carlo-012606-000001
JobscriptExit code0
Real time3h (12272s)
CPU time3h (11011s = 89%)
Max RSS bytes3994394624 (3809 MiB)
Outputting started2026-01-31 12:11:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12606/1/001/reco_stage1.root
Finished2026-01-31 12:12:53
Saved logsjustin-logs:302559.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

edRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 141218 traces, 0 summary
Retagger: tagging trace set: gauss with 104628 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104628 traces tagged "gauss"
FrameSaver: q=3.81085e+06 n=1819193 tag=gauss
wclsFrameSaver: saving 141218 traces tagged "wiener"
FrameSaver: q=4.06893e+06 n=1602893 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 49624 XUs and 51402 XVs -> 2084 XUVs
C:0 T:1 148870 XUs and 133714 XVs -> 28522 XUVs
C:0 T:2 71929 XUs and 51030 XVs -> 4386 XUVs
C:0 T:3 35405 XUs and 19014 XVs -> 117 XUVs
C:0 T:4 8599 XUs and 10456 XVs -> 99 XUVs
C:0 T:5 33421 XUs and 32398 XVs -> 4073 XUVs
C:0 T:6 134224 XUs and 100853 XVs -> 10126 XUVs
C:0 T:7 44046 XUs and 37934 XVs -> 719 XUVs
C:0 T:8 13656 XUs and 13194 XVs -> 154 XUVs
C:0 T:9 92754 XUs and 117785 XVs -> 6390 XUVs
C:0 T:10 43435 XUs and 27289 XVs -> 10499 XUVs
C:0 T:11 258 XUs and 182 XVs -> 25 XUVs
67194 XUVs total
12882 collection wire objects
60858 potential space points
Neighbour search...
11094526 tests to find 4960696 neighbours
Iterating with no regularization...
Begin: 8.71681e+07
0 6.54119e+07
1 6.34425e+07
2 6.30651e+07
3 6.29535e+07
4 6.29045e+07
Now with regularization...
Begin: 4.89973e+07
0 4.88867e+07
1 4.88166e+07
2 4.87684e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412075028
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412075028
fLifetime: 30000 [us]
31-Jan-2026 13:11:31 CET  Closed output file "reco_stage1.root"
31-Jan-2026 13:11:31 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          205.383       300.88        375.38        319.696       54.179         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00103058    0.00343454     0.0058348    0.00348104     0.0015554       10     
reco:ophit:OpHitFinder                             0.227546      0.276943      0.358235      0.277034      0.0399054       10     
reco:opflash:OpFlashFinder                         0.138586      0.172579       0.19655      0.172962      0.0196941       10     
reco:opslicer:OpSlicer                             0.476866      0.649443      0.876787      0.648243      0.140444        10     
reco:caldata:DataPrepModule                         3.88921       4.78461       5.62577       4.69565      0.571991        10     
reco:wclsdatasp:WireCellToolkit                     27.0073       30.5864       35.4801       30.3317       2.80749        10     
reco:gaushit:GausHitFinder                         0.725033       1.03696       1.71301      0.967462      0.269866        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000310714   0.000433525   0.000692209   0.000379965   0.000123223      10     
reco:reco3d:SpacePointSolver                        7.23519       15.2653       38.225        13.1454       8.72366        10     
reco:hitpdune:DisambigFromSpacePoints              0.901659       1.68478       3.06386       1.6044       0.578562        10     
reco:pandora:StandardPandora                        14.0116       30.486        42.9445       36.5233       10.2187        10     
reco:pandoraWriter:StandardPandora                 0.237031       0.39979      0.561264      0.402193      0.0946918       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.74393       3.1237        4.26423       3.23402      0.735133        10     
reco:pandoraShower:LArPandoraShowerCreation         2.01815       3.81412       5.20549       4.01893       1.00831        10     
reco:pandoracalo:Calorimetry                        1.09121       1.90751       2.39469       2.10518      0.432635        10     
reco:pandoracalonosce:Calorimetry                  0.984534       1.65453       2.35732       1.66394      0.413771        10     
reco:pandorapid:Chi2ParticleID                     0.0030509    0.00479778    0.00641455    0.00464501    0.00104666       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0285923     0.0452344     0.0689507     0.0452617      0.01178        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00300564    0.00450116    0.00614024    0.00433251    0.000925161      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.06523       3.83557       5.9755        3.76123       1.17847        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.96026       3.89529       5.68896       4.18219       1.18022        10     
reco:emtrkmichelid:EmTrackMichelId                  115.861       161.397       192.512       167.405       24.5423        10     
reco:crttag:SingleCRTMatchingProducer             0.00708009     0.0119576     0.0195588     0.0116926    0.00385893       10     
reco:crtreco:TwoCRTMatchingProducer               0.00451714    0.00764519     0.0105928    0.00745815    0.00203989       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.709048      0.863985       1.09051      0.853369      0.134058        10     
reco:pandora2Track:LArPandoraTrackCreation          6.6168        9.61564       14.3837       9.45793       2.45104        10     
reco:pandora2calo:Calorimetry                       3.43267       5.36378       7.38052       5.33782       1.30039        10     
reco:pandora2calonosce:Calorimetry                  2.93572       5.18437       8.24563       4.90256       1.50067        10     
reco:pandora2pid:Chi2ParticleID                   0.00920141     0.0121328     0.0185332     0.0112873    0.00305196       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0394745     0.0598106     0.0975797     0.0591899     0.0168486       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00911443     0.0121868     0.019399      0.0112667    0.00325549       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.76093       3.62767       4.55158       3.67608       0.64203        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.85517       4.65579       6.42962       4.7216       0.975783        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.57471       4.0166        5.37939       4.30628      0.938106        10     
[art]:TriggerResults:TriggerResultInserter         3.401e-05    6.26827e-05   0.000140112    5.291e-05    3.07423e-05      10     
end_path:out1:RootOutput                           6.98e-06     9.5241e-06    1.6681e-05     8.74e-06     2.60902e-06      10     
end_path:out1:RootOutput(write)                     1.86654       2.32822       2.86685       2.33992      0.282865        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4819.8 MB
  Peak resident set size usage (VmHWM): 3994.39 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9881320
-rw-r--r--. 1 duneprd np-comp 2321926047 Jan 31 12:18 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2199434776 Jan 31 12:16 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2011964992 Jan 31 13:11 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1793047343 Jan 31 10:14 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1715659897 Jan 31 09:54 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   75823790 Jan 31 13:07 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242768 Jan 31 13:11 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 13:11 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 13:11 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 13:11 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 12:18 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 10:14 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 13:11 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 31 09:47 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 13:11 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:25 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 31 12:18 debugprod.log
justIN time: 2026-02-04 09:20:53 UTC       justIN version: 01.06.00