justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 13675.0@dunegpschedd02.fnal.gov

Jobsub ID13675.0@dunegpschedd02.fnal.gov
Workflow ID204
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-01 11:28:01
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-08-01 13:27:50
From worker nodeHostnamedunegli-6428384-0-fnpc23043.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-01 11:29:37
Input filesmonte-carlo-000204-000001
JobscriptExit code0
Real time1h (7081s)
CPU time1h (6999s = 98%)
Max RSS bytes4173602816 (3980 MiB)
Outputting started2025-08-01 13:27:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00204/1/001/reco_stage1.root
Finished2025-08-01 13:27:50
Saved logsjustin-logs:13675.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138820 traces, 0 summary
Retagger: tagging trace set: gauss with 106741 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106741 traces tagged "gauss"
FrameSaver: q=3.79126e+06 n=1930084 tag=gauss
wclsFrameSaver: saving 138820 traces tagged "wiener"
FrameSaver: q=4.0165e+06 n=1726019 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36801 XUs and 43907 XVs -> 600 XUVs
C:0 T:1 87789 XUs and 89156 XVs -> 7145 XUVs
C:0 T:2 47857 XUs and 59240 XVs -> 1578 XUVs
C:0 T:3 28805 XUs and 30437 XVs -> 135 XUVs
C:0 T:4 16218 XUs and 12926 XVs -> 102 XUVs
C:0 T:5 87029 XUs and 37975 XVs -> 5109 XUVs
C:0 T:6 85096 XUs and 115932 XVs -> 3518 XUVs
C:0 T:7 37036 XUs and 49973 XVs -> 508 XUVs
C:0 T:8 11428 XUs and 11478 XVs -> 373 XUVs
C:0 T:9 108070 XUs and 109235 XVs -> 17178 XUVs
C:0 T:10 35939 XUs and 27496 XVs -> 4226 XUVs
C:0 T:11 259 XUs and 192 XVs -> 19 XUVs
40491 XUVs total
12117 collection wire objects
36839 potential space points
Neighbour search...
4346421 tests to find 2017858 neighbours
Iterating with no regularization...
Begin: 7.14424e+07
0 5.62056e+07
1 5.5289e+07
2 5.5158e+07
3 5.51174e+07
Now with regularization...
Begin: 4.44163e+07
0 4.43878e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408381189
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408381189
fLifetime: 30000 [us]
01-Aug-2025 13:27:37 UTC  Closed output file "reco_stage1.root"
01-Aug-2025 13:27:37 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          133.869       203.896       331.533       187.306       58.7848        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000581412   0.00263601    0.00387029    0.00282711    0.00110546       10     
reco:ophit:OpHitFinder                             0.172219      0.199791      0.248212       0.19411      0.0236055       10     
reco:opflash:OpFlashFinder                         0.0977135     0.128185      0.176487      0.122738      0.024813        10     
reco:opslicer:OpSlicer                             0.362833      0.429735       0.61344      0.406047      0.0726272       10     
reco:caldata:DataPrepModule                         3.32945       3.43326       3.71274       3.39939      0.107568        10     
reco:wclsdatasp:WireCellToolkit                     22.252        23.296        24.9909       23.2193      0.768693        10     
reco:gaushit:GausHitFinder                         0.559487       0.78079       1.39012       0.69751      0.231907        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00020028    0.000334625   0.000524214   0.000311781   9.99368e-05      10     
reco:reco3d:SpacePointSolver                        3.79104       8.70717       16.1022       7.67856       3.79469        10     
reco:hitpdune:DisambigFromSpacePoints              0.569419       1.06111       2.2618       0.870605      0.480413        10     
reco:pandora:StandardPandora                        7.88515       20.7402       55.0071       16.4337       13.8871        10     
reco:pandoraWriter:StandardPandora                 0.175269      0.266693      0.390431      0.252966      0.0671869       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.03098       1.97329       3.06147       1.82379      0.646706        10     
reco:pandoraShower:LArPandoraShowerCreation         1.00136       2.29352       4.95524       1.92324       1.13921        10     
reco:pandoracalo:Calorimetry                       0.668099       1.21352       1.88727       1.1469       0.406003        10     
reco:pandoracalonosce:Calorimetry                  0.602148       1.09726       1.66917       1.06302      0.357756        10     
reco:pandorapid:Chi2ParticleID                    0.00246942    0.00334114    0.00498553    0.00305995    0.000737685      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0229898     0.0319354     0.0427955     0.0316579    0.00674839       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00227532    0.00308454    0.00405872    0.00301134    0.000566592      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.07661       2.59804       5.54648       2.25885       1.32023        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.03118       2.50872       5.40755       2.13148       1.29342        10     
reco:emtrkmichelid:EmTrackMichelId                  76.9408       108.572       161.712       103.075       25.3573        10     
reco:crttag:SingleCRTMatchingProducer             0.00557008    0.00963384     0.0172951    0.00849873    0.00360498       10     
reco:crtreco:TwoCRTMatchingProducer               0.00355217    0.00498374    0.00679583     0.0050441    0.00103992       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.316836      0.549009      0.806282      0.498913       0.1442         10     
reco:pandora2Track:LArPandoraTrackCreation          3.0701        6.61297       13.0735       5.69129       3.03367        10     
reco:pandora2calo:Calorimetry                       1.64105       3.58638       6.77408       3.2313        1.58636        10     
reco:pandora2calonosce:Calorimetry                  1.55367       3.41544       6.39846       3.11963       1.50265        10     
reco:pandora2pid:Chi2ParticleID                   0.00530414    0.00763568     0.0122628    0.00705808     0.0020452       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0284374     0.0397325     0.0567384     0.0367005    0.00928132       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00539871    0.00759763     0.012017     0.00693361    0.00193694       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.17534       2.3884        4.3458        2.15323      0.953901        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.4084        3.10989       5.67812       2.90617       1.32115        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.34084       2.86445       5.21023       2.72804       1.18475        10     
[art]:TriggerResults:TriggerResultInserter        2.9216e-05    4.58279e-05   0.000124386   3.72655e-05   2.68182e-05      10     
end_path:out1:RootOutput                           7.354e-06    1.03858e-05   1.6912e-05     9.253e-06    2.78283e-06      10     
end_path:out1:RootOutput(write)                     1.31721       1.8908        2.7763        1.77863      0.469995        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4978.31 MB
  Peak resident set size usage (VmHWM): 4173.6 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.15 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9908644
-rw-r--r-- 1 dunegli fnalgrid 2335241616 Aug  1 12:52 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2217454127 Aug  1 12:51 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2021521624 Aug  1 13:27 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 1777790891 Aug  1 11:39 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 1709272599 Aug  1 11:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   74636230 Aug  1 13:24 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242712 Aug  1 13:27 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Aug  1 13:27 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Aug  1 13:27 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Aug  1 13:27 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 12:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 11:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Aug  1 13:27 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Aug  1 11:29 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Aug  1 13:27 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Aug  1 12:56 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Aug  1 12:52 debugprod.log
justIN time: 2025-08-04 19:45:58 UTC       justIN version: 01.04.00