justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41851.0@dunegpschedd01.fnal.gov

Jobsub ID41851.0@dunegpschedd01.fnal.gov
Workflow ID2317
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-09-06 18:39:03
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2025-09-06 21:47:32
From worker nodeHostnamecomp15-01
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-06 18:40:31
Input filesmonte-carlo-002317-000001
JobscriptExit code0
Real time3h (11142s)
CPU time3h (10927s = 98%)
Max RSS bytes4121153536 (3930 MiB)
Outputting started2025-09-06 21:46:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/02317/1/001/reco_stage1.root
Finished2025-09-06 21:47:32
Saved logsjustin-logs:41851.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ervice:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123994 traces, 0 summary
Retagger: tagging trace set: gauss with 96426 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96426 traces tagged "gauss"
FrameSaver: q=3.37694e+06 n=1733215 tag=gauss
wclsFrameSaver: saving 123994 traces tagged "wiener"
FrameSaver: q=3.56546e+06 n=1541195 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 31160 XUs and 29703 XVs -> 1869 XUVs
C:0 T:2 141897 XUs and 144716 XVs -> 6028 XUVs
C:0 T:3 65699 XUs and 58644 XVs -> 605 XUVs
C:0 T:4 17277 XUs and 12820 XVs -> 206 XUVs
C:0 T:5 72250 XUs and 44654 XVs -> 6958 XUVs
C:0 T:6 98392 XUs and 117883 XVs -> 3708 XUVs
C:0 T:7 42340 XUs and 49232 XVs -> 306 XUVs
C:0 T:8 6953 XUs and 6093 XVs -> 62 XUVs
C:0 T:9 56711 XUs and 50801 XVs -> 4800 XUVs
C:0 T:10 45861 XUs and 20837 XVs -> 14287 XUVs
C:0 T:11 52 XUs and 28 XVs -> 9 XUVs
38838 XUVs total
10916 collection wire objects
36821 potential space points
Neighbour search...
5898275 tests to find 2521628 neighbours
Iterating with no regularization...
Begin: 6.02821e+07
0 4.93172e+07
1 4.85268e+07
2 4.8424e+07
3 4.83961e+07
Now with regularization...
Begin: 3.92238e+07
0 3.91951e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 409126221
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409126221
fLifetime: 30000 [us]
06-Sep-2025 22:46:11 BST  Closed output file "reco_stage1.root"
06-Sep-2025 22:46:11 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          225.843       296.034       346.489       299.659       41.3204        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000835027   0.00380146    0.00531457    0.00442556    0.00148315       10     
reco:ophit:OpHitFinder                             0.257435      0.300156      0.339206      0.298528      0.023385        10     
reco:opflash:OpFlashFinder                         0.150617      0.182159       0.2191       0.181362      0.0197094       10     
reco:opslicer:OpSlicer                             0.437662      0.591536      0.697657       0.59058      0.0709632       10     
reco:caldata:DataPrepModule                         4.80853       4.92913       5.24107       4.91586      0.123052        10     
reco:wclsdatasp:WireCellToolkit                     30.2431       32.4228       33.5173       32.5907      0.925526        10     
reco:gaushit:GausHitFinder                          0.7564       0.987052       1.15741       1.00509      0.114213        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000304922   0.000391958   0.000499194   0.000387874   4.97366e-05      10     
reco:reco3d:SpacePointSolver                        6.42318       12.3654       17.2537       13.131        3.58193        10     
reco:hitpdune:DisambigFromSpacePoints              0.818508       1.15543       1.43626       1.14725      0.205863        10     
reco:pandora:StandardPandora                        15.3185       25.2118       34.0584       25.8275       5.61278        10     
reco:pandoraWriter:StandardPandora                  0.31814      0.396202      0.482346      0.401121      0.0460189       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.72077       3.88071       5.4605        3.80805      0.862856        10     
reco:pandoraShower:LArPandoraShowerCreation         2.27594       4.16876       6.25913       4.13195       1.16049        10     
reco:pandoracalo:Calorimetry                        1.42412       2.34224       3.07656       2.41212      0.581282        10     
reco:pandoracalonosce:Calorimetry                   1.25614       2.2091        2.9386        2.25282      0.567066        10     
reco:pandorapid:Chi2ParticleID                    0.00398006    0.00475918    0.00554485     0.0047252    0.000467063      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0364693     0.0407912     0.0479332     0.0407701    0.00313866       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00393511    0.00467065    0.00539049    0.00473731    0.000425419      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.33063       4.80543       6.81413       4.8589        1.45233        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.13594       4.62701       6.62725       4.66816       1.45018        10     
reco:emtrkmichelid:EmTrackMichelId                  119.715       153.065       179.56        156.659       17.4017        10     
reco:crttag:SingleCRTMatchingProducer              0.0086667     0.011796      0.0154099     0.0117381    0.00213235       10     
reco:crtreco:TwoCRTMatchingProducer               0.00592951     0.0087431     0.0129695    0.00849926    0.00217868       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.61133       0.71306      0.870398      0.704353      0.0787028       10     
reco:pandora2Track:LArPandoraTrackCreation          7.32907       12.3722       16.5889       12.8617       2.98641        10     
reco:pandora2calo:Calorimetry                       3.32796       6.51867       8.85634       6.73053       1.79088        10     
reco:pandora2calonosce:Calorimetry                  3.09279       6.15937       8.39054       6.22892       1.74686        10     
reco:pandora2pid:Chi2ParticleID                   0.00840157     0.0107172     0.0123116     0.0110086    0.00118607       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0452253     0.0535228     0.0648259     0.0529251    0.00621478       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00888996     0.0108666     0.0124754     0.0111459    0.00116047       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.53108       4.2072        5.81877       4.16248      0.971195        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.90522       5.22156       7.0963        5.27176       1.35944        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.46127       4.36568       5.93756       4.44931       1.10803        10     
[art]:TriggerResults:TriggerResultInserter         5.568e-05    6.53117e-05   0.000125641   5.8326e-05    2.02422e-05      10     
end_path:out1:RootOutput                           1.142e-05    1.31782e-05    1.727e-05    1.2595e-05    1.75514e-06      10     
end_path:out1:RootOutput(write)                     2.0177        2.52329       2.95191       2.51014      0.279826        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4932.5 MB
  Peak resident set size usage (VmHWM): 4121.15 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.01 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10133564
-rw-r--r-- 1 pltdune002 pltdune 2377659412 Sep  6 21:55 detsim_stage2.root
-rw-r--r-- 1 pltdune002 pltdune 2261657067 Sep  6 21:53 detsim_stage1.root
-rw-r--r-- 1 pltdune002 pltdune 2044366378 Sep  6 22:46 reco_stage1.root
-rw-r--r-- 1 pltdune002 pltdune 1865196093 Sep  6 19:57 g4_stage2.root
-rw-r--r-- 1 pltdune002 pltdune 1757268081 Sep  6 19:46 g4_stage1.root
-rw-r--r-- 1 pltdune002 pltdune   69941320 Sep  6 22:42 Pandora_Events.pndr
-rw-r--r-- 1 pltdune002 pltdune     242730 Sep  6 22:46 jobscript.log
-rw-r--r-- 1 pltdune002 pltdune     217088 Sep  6 22:46 mem.db
-rw-r--r-- 1 pltdune002 pltdune      62376 Sep  6 22:46 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune002 pltdune      36864 Sep  6 22:46 time.db
-rw-r--r-- 1 pltdune002 pltdune        519 Sep  6 21:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune002 pltdune        519 Sep  6 19:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune002 pltdune        519 Sep  6 22:46 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune002 pltdune         52 Sep  6 19:40 all-input-dids.txt
-rw-r--r-- 1 pltdune002 pltdune          7 Sep  6 22:46 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune002 pltdune          0 Sep  6 21:59 ProtoDUNE.log
-rw-r--r-- 1 pltdune002 pltdune          0 Sep  6 21:55 debugprod.log
justIN time: 2025-09-18 17:52:30 UTC       justIN version: 01.05.00