justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296899.1@dunegpschedd02.fnal.gov

Jobsub ID296899.1@dunegpschedd02.fnal.gov
Workflow ID12599
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 21:51:33
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-30 23:29:41
From worker nodeHostnameb9p35p0005.cern.ch
cpuinfoAMD EPYC 9655P 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 21:52:04
Input filesmonte-carlo-012599-000001
JobscriptExit code0
Real time1h (5787s)
CPU time1h (5305s = 91%)
Max RSS bytes3905118208 (3724 MiB)
Outputting started2026-01-30 23:28:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12599/1/001/reco_stage1.root
Finished2026-01-30 23:29:41
Saved logsjustin-logs:296899.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

are:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 110867 traces, 0 summary
Retagger: tagging trace set: gauss with 82764 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 82764 traces tagged "gauss"
FrameSaver: q=2.86682e+06 n=1477857 tag=gauss
wclsFrameSaver: saving 110867 traces tagged "wiener"
FrameSaver: q=3.03745e+06 n=1305266 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33177 XUs and 32473 XVs -> 266 XUVs
C:0 T:1 64087 XUs and 59212 XVs -> 4492 XUVs
C:0 T:2 107756 XUs and 150329 XVs -> 13330 XUVs
C:0 T:3 44706 XUs and 48367 XVs -> 1687 XUVs
C:0 T:4 14226 XUs and 12989 XVs -> 115 XUVs
C:0 T:5 53004 XUs and 49500 XVs -> 5416 XUVs
C:0 T:6 80427 XUs and 70512 XVs -> 5204 XUVs
C:0 T:7 34726 XUs and 30244 XVs -> 259 XUVs
C:0 T:8 7032 XUs and 6773 XVs -> 60 XUVs
C:0 T:9 44770 XUs and 55053 XVs -> 6367 XUVs
C:0 T:10 10645 XUs and 6839 XVs -> 1347 XUVs
C:0 T:11 108 XUs and 56 XVs -> 21 XUVs
38564 XUVs total
9246 collection wire objects
34484 potential space points
Neighbour search...
3519322 tests to find 1696868 neighbours
Iterating with no regularization...
Begin: 5.87225e+07
0 4.88553e+07
1 4.80827e+07
2 4.79624e+07
3 4.79283e+07
Now with regularization...
Begin: 3.97751e+07
0 3.97648e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 412065942
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412065942
fLifetime: 30000 [us]
31-Jan-2026 00:28:31 CET  Closed output file "reco_stage1.root"
31-Jan-2026 00:28:31 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          148.103       196.119       246.534       209.449       35.1362        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00075585    0.00298681    0.00530871     0.0030204    0.00129236       10     
reco:ophit:OpHitFinder                             0.162631      0.195385      0.221226      0.192044      0.0162137       10     
reco:opflash:OpFlashFinder                         0.071157      0.0941547     0.114519      0.0966532     0.0142155       10     
reco:opslicer:OpSlicer                             0.369812      0.471944      0.559079      0.465205      0.0541669       10     
reco:caldata:DataPrepModule                         2.96466       3.20686       3.66914       3.13426      0.204801        10     
reco:wclsdatasp:WireCellToolkit                     19.5088       20.854        22.8732       20.404       0.968319        10     
reco:gaushit:GausHitFinder                         0.550512      0.756639       0.98291       0.78057       0.14111        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00019545    0.000273629   0.00046273    0.000249445   7.73912e-05      10     
reco:reco3d:SpacePointSolver                        3.87903       9.05378       18.5582       8.20965       3.63696        10     
reco:hitpdune:DisambigFromSpacePoints              0.646371      0.973191       1.26666       1.01015      0.198131        10     
reco:pandora:StandardPandora                        11.0513       21.1784       34.3376       21.4986       7.22914        10     
reco:pandoraWriter:StandardPandora                 0.245483       0.31646      0.382422      0.315075      0.0366766       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.34333       2.33597       4.59791       2.13285       0.90709        10     
reco:pandoraShower:LArPandoraShowerCreation         1.19848       2.26444       3.7577        2.3454       0.721142        10     
reco:pandoracalo:Calorimetry                       0.783953       1.21082       1.6894        1.20707      0.289844        10     
reco:pandoracalonosce:Calorimetry                  0.632236       1.04553       1.70268       1.05158      0.303168        10     
reco:pandorapid:Chi2ParticleID                    0.00246944    0.00308863    0.00420903    0.00296133    0.000516016      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0239791     0.0329404     0.0575231     0.0307333    0.00910991       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00238214    0.00307209    0.00435552    0.00296412    0.000585771      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.13621       2.49056       4.2805        2.52429      0.962651        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.03204       2.48197       3.8919        2.60764       0.95757        10     
reco:emtrkmichelid:EmTrackMichelId                  79.0026       100.475       122.835       102.833       15.893         10     
reco:crttag:SingleCRTMatchingProducer              0.0059111     0.0105207     0.0140668     0.0103392    0.00235826       10     
reco:crtreco:TwoCRTMatchingProducer               0.00405645    0.00676922     0.0117238    0.00702435    0.00205608       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.307822      0.415896      0.604419      0.407008      0.0850857       10     
reco:pandora2Track:LArPandoraTrackCreation          5.21861       8.50546       13.3487       7.4985        2.69713        10     
reco:pandora2calo:Calorimetry                       1.93087       3.64214       5.6678        3.71105       1.22161        10     
reco:pandora2calonosce:Calorimetry                  1.69238       3.3083        4.88278       3.65541       1.02457        10     
reco:pandora2pid:Chi2ParticleID                   0.00585427    0.00816791     0.0104714    0.00815418    0.00155218       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0311579     0.0463597     0.0662375     0.0448896     0.0118757       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00594931    0.00823804     0.0103793    0.00834185    0.00156789       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.55107       2.70694       3.95795       2.87728      0.810102        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.64124       3.09026       4.40249       3.18755       0.94323        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.44363       2.85367       4.28635       2.81862      0.927461        10     
[art]:TriggerResults:TriggerResultInserter         2.446e-05    3.5734e-05     8.264e-05    3.0905e-05    1.60807e-05      10     
end_path:out1:RootOutput                           5.41e-06      6.244e-06     9.14e-06      5.905e-06    1.03352e-06      10     
end_path:out1:RootOutput(write)                     1.52078       1.94807       2.41964       1.97481      0.300249        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4751.43 MB
  Peak resident set size usage (VmHWM): 3905.12 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   1.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.02 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9286684
-rw-r--r--. 1 duneprd np-comp 2204469595 Jan 30 23:52 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2091562481 Jan 30 23:50 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 1884256840 Jan 31 00:28 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 1658494971 Jan 30 23:03 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1596899400 Jan 30 22:57 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   73263100 Jan 31 00:26 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     242895 Jan 31 00:28 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 31 00:28 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 31 00:28 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 31 00:28 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 30 23:52 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 30 23:03 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 31 00:28 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 30 22:52 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 31 00:28 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 30 23:58 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 30 23:52 debugprod.log
justIN time: 2026-02-04 13:18:03 UTC       justIN version: 01.06.00