justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296471.0@dunegpschedd02.fnal.gov

Jobsub ID296471.0@dunegpschedd02.fnal.gov
Workflow ID12507
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-29 09:31:46
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce507
Last heartbeat2026-01-29 12:54:16
From worker nodeHostnameb9p10p4185.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-29 09:32:19
Input filesmonte-carlo-012507-000001
JobscriptExit code0
Real time3h (12038s)
CPU time3h (11150s = 92%)
Max RSS bytes4080828416 (3891 MiB)
Outputting started2026-01-29 12:52:58
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12507/1/001/reco_stage1.root
Finished2026-01-29 12:54:16
Saved logsjustin-logs:296471.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 128053 traces, 0 summary
Retagger: tagging trace set: gauss with 95800 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95800 traces tagged "gauss"
FrameSaver: q=3.37296e+06 n=1671109 tag=gauss
wclsFrameSaver: saving 128053 traces tagged "wiener"
FrameSaver: q=3.58087e+06 n=1460517 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33773 XUs and 34266 XVs -> 373 XUVs
C:0 T:1 72983 XUs and 61867 XVs -> 5101 XUVs
C:0 T:2 166381 XUs and 270650 XVs -> 12258 XUVs
C:0 T:3 60764 XUs and 86934 XVs -> 744 XUVs
C:0 T:4 11492 XUs and 11420 XVs -> 152 XUVs
C:0 T:5 81335 XUs and 57622 XVs -> 9213 XUVs
C:0 T:6 101573 XUs and 137782 XVs -> 12329 XUVs
C:0 T:7 35971 XUs and 45978 XVs -> 381 XUVs
C:0 T:8 3955 XUs and 5039 XVs -> 108 XUVs
C:0 T:9 20757 XUs and 43473 XVs -> 2187 XUVs
C:0 T:10 23410 XUs and 52221 XVs -> 8697 XUVs
C:0 T:11 25 XUs and 18 XVs -> 5 XUVs
51548 XUVs total
12821 collection wire objects
47497 potential space points
Neighbour search...
7812981 tests to find 3562512 neighbours
Iterating with no regularization...
Begin: 7.92352e+07
0 5.53051e+07
1 5.37862e+07
2 5.36035e+07
3 5.35456e+07
4 5.35163e+07
Now with regularization...
Begin: 4.29485e+07
0 4.28764e+07
1 4.28354e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412035424
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412035424
fLifetime: 30000 [us]
29-Jan-2026 13:52:55 CET  Closed output file "reco_stage1.root"
29-Jan-2026 13:52:55 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          182.444       263.886       316.863       280.938       42.2138        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145673    0.00343557    0.00532728    0.00331378    0.00128349       10     
reco:ophit:OpHitFinder                             0.198029      0.250815      0.339152      0.227493      0.0502071       10     
reco:opflash:OpFlashFinder                         0.116705      0.142914      0.174382      0.138659      0.0169631       10     
reco:opslicer:OpSlicer                             0.414393      0.556345      0.753745      0.487235      0.127128        10     
reco:caldata:DataPrepModule                         3.98197       4.95416       6.7686        4.62097      0.943358        10     
reco:wclsdatasp:WireCellToolkit                     26.7305       30.3771       37.971        29.3753       3.43689        10     
reco:gaushit:GausHitFinder                         0.577789      0.813672       1.02958      0.823187      0.141083        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000299783   0.000378388   0.000541925   0.000361604   6.58193e-05      10     
reco:reco3d:SpacePointSolver                        6.08283       12.4577       20.2291       12.3844       4.40156        10     
reco:hitpdune:DisambigFromSpacePoints              0.619443       1.14297       1.9653        1.18643      0.381838        10     
reco:pandora:StandardPandora                        9.65058       25.2484       39.7691       26.3962       8.98393        10     
reco:pandoraWriter:StandardPandora                 0.218327      0.321893      0.477673      0.321603      0.0682126       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.93361       2.44108       3.50041       2.35392      0.482101        10     
reco:pandoraShower:LArPandoraShowerCreation         1.25056       2.65303       3.90165       2.5553       0.781543        10     
reco:pandoracalo:Calorimetry                        0.83388       1.48141       1.91886       1.5019       0.357089        10     
reco:pandoracalonosce:Calorimetry                   0.74734       1.37661       2.02934       1.41001      0.405621        10     
reco:pandorapid:Chi2ParticleID                    0.00307224    0.00429409    0.00728766    0.00386624    0.00128169       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0282996     0.0408276     0.0670833     0.0363431     0.0129398       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00290687    0.00400024    0.00555836    0.00383579    0.000866847      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.2793        2.95963       5.11048       2.92221       1.12061        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.3945        2.74838       4.49199       2.71531       1.00085        10     
reco:emtrkmichelid:EmTrackMichelId                  105.148       144.716       165.546       148.551       19.0054        10     
reco:crttag:SingleCRTMatchingProducer             0.00699898     0.0114255     0.0210684     0.010444     0.00379267       10     
reco:crtreco:TwoCRTMatchingProducer               0.00419993     0.0062702    0.00789297    0.00655835    0.00116145       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.509913      0.719203      0.842252      0.751959      0.111796        10     
reco:pandora2Track:LArPandoraTrackCreation          4.06026       7.81731       11.2877       7.77998       2.13036        10     
reco:pandora2calo:Calorimetry                       2.00891       4.12688       5.86968       4.29718       1.36802        10     
reco:pandora2calonosce:Calorimetry                  1.90273       3.78616       5.5237        3.58998       1.24388        10     
reco:pandora2pid:Chi2ParticleID                   0.00654074    0.00841168     0.0100837    0.00817878    0.00108373       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0340285     0.0423105     0.0501875     0.0428259    0.00471951       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00630621    0.00826415    0.00995094    0.00796991    0.00118059       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.5434        2.76884       4.20871       2.45801       0.88125        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.3053        3.81679       6.21303       3.54406       1.31233        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.07618       3.71634       6.13425       3.02618       1.4447         10     
[art]:TriggerResults:TriggerResultInserter         4.457e-05    6.38807e-05   0.000142941   5.1891e-05    2.82773e-05      10     
end_path:out1:RootOutput                            8.5e-06     1.3461e-05     2.369e-05     1.158e-05    4.12198e-06      10     
end_path:out1:RootOutput(write)                     1.64242       2.27911       3.09667       2.16888      0.471122        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4933.62 MB
  Peak resident set size usage (VmHWM): 4080.83 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9698200
-rw-r--r--. 1 duneprd np-comp 2297175930 Jan 29 13:06 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2185907008 Jan 29 13:04 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1958645886 Jan 29 13:52 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1743005494 Jan 29 10:57 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1676228618 Jan 29 10:39 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   69379550 Jan 29 13:49 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242931 Jan 29 13:52 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 29 13:52 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 29 13:52 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 29 13:52 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 29 13:06 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 29 10:57 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 29 13:52 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 29 10:32 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 29 13:52 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 29 13:12 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 29 13:06 debugprod.log
justIN time: 2026-02-04 15:16:31 UTC       justIN version: 01.06.00