justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233055.0@dunegpschedd02.fnal.gov

Jobsub ID233055.0@dunegpschedd02.fnal.gov
Workflow ID9358
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-26 13:20:39
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-10-26 17:32:32
From worker nodeHostnamehnode65.pic.es
cpuinfoAMD EPYC 7402P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-26 13:27:50
Input filesmonte-carlo-009358-000001
JobscriptExit code0
Real time4h (14611s)
CPU time3h (14008s = 95%)
Max RSS bytes4239425536 (4043 MiB)
Outputting started2025-10-26 17:31:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09358/1/001/reco_stage1.root
Finished2025-10-26 17:32:32
Saved logsjustin-logs:233055.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 154047 traces, 0 summary
Retagger: tagging trace set: gauss with 118600 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 118600 traces tagged "gauss"
FrameSaver: q=4.07349e+06 n=2116058 tag=gauss
wclsFrameSaver: saving 154047 traces tagged "wiener"
FrameSaver: q=4.30267e+06 n=1875437 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32361 XUs and 35004 XVs -> 318 XUVs
C:0 T:1 71446 XUs and 68430 XVs -> 5689 XUVs
C:0 T:2 122482 XUs and 173790 XVs -> 9827 XUVs
C:0 T:3 54644 XUs and 64889 XVs -> 596 XUVs
C:0 T:4 17124 XUs and 11952 XVs -> 134 XUVs
C:0 T:5 58322 XUs and 32680 XVs -> 3194 XUVs
C:0 T:6 160521 XUs and 173431 XVs -> 6856 XUVs
C:0 T:7 59149 XUs and 64390 XVs -> 631 XUVs
C:0 T:8 10051 XUs and 8606 XVs -> 92 XUVs
C:0 T:9 66699 XUs and 74107 XVs -> 4397 XUVs
C:0 T:10 57142 XUs and 25589 XVs -> 4226 XUVs
C:0 T:11 546 XUs and 361 XVs -> 84 XUVs
36044 XUVs total
14705 collection wire objects
32732 potential space points
Neighbour search...
2261384 tests to find 1078872 neighbours
Iterating with no regularization...
Begin: 9.40406e+07
0 7.79213e+07
1 7.6657e+07
2 7.65394e+07
3 7.65118e+07
Now with regularization...
Begin: 6.40215e+07
0 6.40032e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410127504
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410127504
fLifetime: 30000 [us]
26-Oct-2025 18:31:20 CET  Closed output file "reco_stage1.root"
26-Oct-2025 18:31:20 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          341.174       519.73        634.02        527.413       96.0244        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00103912     0.0032547    0.00540375    0.00353031    0.00133342       10     
reco:ophit:OpHitFinder                             0.281335      0.338758      0.406833       0.34165      0.0409219       10     
reco:opflash:OpFlashFinder                          0.1383       0.207968      0.259983      0.204072      0.0325112       10     
reco:opslicer:OpSlicer                             0.730042      0.973426       1.24277       1.04326      0.173661        10     
reco:caldata:DataPrepModule                         4.72971       6.52642       6.94873       6.72078      0.623895        10     
reco:wclsdatasp:WireCellToolkit                     36.4689       41.8777       45.8637       42.0841       2.60504        10     
reco:gaushit:GausHitFinder                          1.20465       1.64253       2.12578       1.69156      0.253832        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000585226   0.000779133   0.000876635   0.000810857   8.96315e-05      10     
reco:reco3d:SpacePointSolver                        10.1959       18.5326       37.2859       17.1972       7.6466         10     
reco:hitpdune:DisambigFromSpacePoints               1.26083       2.09166       2.5287        2.25995      0.401445        10     
reco:pandora:StandardPandora                        20.5503       47.1539       64.434        50.5913       13.6585        10     
reco:pandoraWriter:StandardPandora                 0.442836      0.618155      0.787879      0.634936      0.100022        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.93988       6.08218       9.60678       5.89792       1.8324         10     
reco:pandoraShower:LArPandoraShowerCreation         2.48834       6.51182       10.5381       6.93051       2.36203        10     
reco:pandoracalo:Calorimetry                        1.55795       3.28491       5.13242       3.21469       1.06467        10     
reco:pandoracalonosce:Calorimetry                   1.33492        3.132        5.28541       3.03332       1.16622        10     
reco:pandorapid:Chi2ParticleID                    0.00545521    0.00766143     0.0102077    0.00774074    0.00159489       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0469343     0.0720968     0.104019      0.0685539      0.01749        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00521251    0.00731455    0.00988774    0.00738404    0.00159398       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.32927       6.13303       9.73693       6.26297       2.26309        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.21426       5.89345       8.8235        6.21995       2.17189        10     
reco:emtrkmichelid:EmTrackMichelId                  207.683       307.783       371.469       318.867       50.8537        10     
reco:crttag:SingleCRTMatchingProducer             0.00676016     0.0174244     0.0257021     0.0184923    0.00495627       10     
reco:crtreco:TwoCRTMatchingProducer               0.00595745     0.0138237     0.0181779     0.0154043    0.00385069       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.737978       1.16561       1.49715       1.2381       0.245978        10     
reco:pandora2Track:LArPandoraTrackCreation          8.57888       18.6583       27.2454       19.5112       5.7318         10     
reco:pandora2calo:Calorimetry                       3.61522       8.19751       12.2636       8.33629       2.64742        10     
reco:pandora2calonosce:Calorimetry                  2.41523       8.09195       12.4789       8.88158       2.91456        10     
reco:pandora2pid:Chi2ParticleID                    0.0092864     0.0182003     0.0261982     0.019025     0.00481796       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0438995     0.0923839     0.133126      0.102596      0.0287208       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00906191     0.0182102     0.0264049     0.0187265    0.00485355       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.16813       6.94735       11.0369       6.59006       2.7117         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.08577       7.18403       10.6457       7.21049       2.59257        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.93044       6.58025       10.0288       6.91577       2.17319        10     
[art]:TriggerResults:TriggerResultInserter        3.0381e-05    4.20792e-05   8.6812e-05    3.5541e-05    1.69486e-05      10     
end_path:out1:RootOutput                            5.2e-06     6.0673e-06     9.08e-06     5.6505e-06    1.08079e-06      10     
end_path:out1:RootOutput(write)                     2.5082        3.71154       4.81127       3.78363      0.741488        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5052.58 MB
  Peak resident set size usage (VmHWM): 4239.43 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.16 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10188436
-rw-r--r-- 1 dune001 dune 2397119382 Oct 26 17:01 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2271283309 Oct 26 16:58 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2099210074 Oct 26 18:31 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1830633137 Oct 26 14:51 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1755410322 Oct 26 14:36 g4_stage1.root
-rw-r--r-- 1 dune001 dune   78675540 Oct 26 18:23 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242746 Oct 26 18:31 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Oct 26 18:31 mem.db
-rw-r--r-- 1 dune001 dune      62376 Oct 26 18:31 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Oct 26 18:31 time.db
-rw-r--r-- 1 dune001 dune        519 Oct 26 17:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 26 14:51 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Oct 26 18:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Oct 26 14:27 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Oct 26 18:31 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Oct 26 17:08 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Oct 26 17:01 debugprod.log
justIN time: 2025-11-05 04:51:38 UTC       justIN version: 01.05.01