justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241995.0@dunegpschedd01.fnal.gov

Jobsub ID241995.0@dunegpschedd01.fnal.gov
Workflow ID9527
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 14:19:37
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-11-02 18:30:31
From worker nodeHostnamewn-da-01.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 14:20:49
Input filesmonte-carlo-009527-000001
JobscriptExit code0
Real time4h (14893s)
CPU time4h (14807s = 99%)
Max RSS bytes4516159488 (4306 MiB)
Outputting started2025-11-02 18:29:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09527/1/001/reco_stage1.root
Finished2025-11-02 18:30:31
Saved logsjustin-logs:241995.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 177969 traces, 0 summary
Retagger: tagging trace set: gauss with 135075 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 135075 traces tagged "gauss"
FrameSaver: q=4.66916e+06 n=2391732 tag=gauss
wclsFrameSaver: saving 177969 traces tagged "wiener"
FrameSaver: q=4.94317e+06 n=2124321 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36187 XUs and 33115 XVs -> 239 XUVs
C:0 T:1 59377 XUs and 57684 XVs -> 3599 XUVs
C:0 T:2 153063 XUs and 222270 XVs -> 7802 XUVs
C:0 T:3 63234 XUs and 84220 XVs -> 646 XUVs
C:0 T:4 21003 XUs and 19905 XVs -> 200 XUVs
C:0 T:5 98697 XUs and 75425 XVs -> 7728 XUVs
C:0 T:6 119520 XUs and 172929 XVs -> 5400 XUVs
C:0 T:7 47980 XUs and 68516 XVs -> 762 XUVs
C:0 T:8 11808 XUs and 11447 XVs -> 122 XUVs
C:0 T:9 77235 XUs and 88559 XVs -> 5713 XUVs
C:0 T:10 31931 XUs and 23787 XVs -> 3269 XUVs
C:0 T:11 169 XUs and 119 XVs -> 17 XUVs
35497 XUVs total
15155 collection wire objects
32220 potential space points
Neighbour search...
916630 tests to find 488682 neighbours
Iterating with no regularization...
Begin: 1.05021e+08
0 9.00905e+07
1 8.90871e+07
2 8.89684e+07
3 8.894e+07
Now with regularization...
Begin: 7.53058e+07
0 7.52946e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410269043
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410269043
fLifetime: 30000 [us]
02-Nov-2025 19:29:01 CET  Closed output file "reco_stage1.root"
02-Nov-2025 19:29:01 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          358.549       485.088       674.986       454.845       99.171         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000849353   0.00578702     0.0101412    0.00562059    0.00293054       10     
reco:ophit:OpHitFinder                             0.285019       0.31924      0.344329      0.319874      0.0205671       10     
reco:opflash:OpFlashFinder                         0.178724      0.243369      0.285106      0.241107      0.0304966       10     
reco:opslicer:OpSlicer                             0.668263      0.783844      0.891295      0.775859      0.0689702       10     
reco:caldata:DataPrepModule                         4.57884       4.92075       5.17988       4.91556       0.16788        10     
reco:wclsdatasp:WireCellToolkit                     37.1713       55.8867       76.426        59.9471       11.9748        10     
reco:gaushit:GausHitFinder                          1.02493       1.42755       1.86871       1.39383      0.241813        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000611121   0.000910378    0.0011082    0.000965473   0.000136523      10     
reco:reco3d:SpacePointSolver                        11.8068       19.6026       33.0445       19.0492       6.89288        10     
reco:hitpdune:DisambigFromSpacePoints               1.21282       2.02888       3.02632       1.91791      0.540678        10     
reco:pandora:StandardPandora                        24.5882       52.1541       91.3792       49.0862       21.7068        10     
reco:pandoraWriter:StandardPandora                 0.424887      0.511244      0.670791      0.484519      0.0787082       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.18814       6.4371        8.91157       6.32992       1.57984        10     
reco:pandoraShower:LArPandoraShowerCreation         4.13936       7.7345        11.8604       6.51255       2.72061        10     
reco:pandoracalo:Calorimetry                        2.36435       3.39227       4.65562       3.41091      0.781494        10     
reco:pandoracalonosce:Calorimetry                   2.17147       3.16863       4.35048       3.17024      0.716383        10     
reco:pandorapid:Chi2ParticleID                    0.00587351    0.00672023     0.0078727    0.00672314    0.000545758      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0464977     0.0570237     0.0757343     0.0548429    0.00866187       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00506052    0.00638975    0.00754512    0.00632984    0.000702853      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.48405       7.51282        11.4         6.49692       2.35099        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.21666       7.29702       11.2454       6.30338       2.40614        10     
reco:emtrkmichelid:EmTrackMichelId                  183.737       242.665       312.732       234.904       36.2439        10     
reco:crttag:SingleCRTMatchingProducer              0.0093958     0.016422      0.0252567     0.0171498    0.00439627       10     
reco:crtreco:TwoCRTMatchingProducer               0.00776687     0.0121819     0.0171834     0.0118135    0.00278372       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.672494      0.938959       1.19278      0.949612      0.142672        10     
reco:pandora2Track:LArPandoraTrackCreation          12.1574       21.4334       31.9763       20.6955       6.70697        10     
reco:pandora2calo:Calorimetry                       6.02673       10.101        14.9247       10.0563       2.93743        10     
reco:pandora2calonosce:Calorimetry                  5.65093       9.63925       14.1196       9.4591        2.85124        10     
reco:pandora2pid:Chi2ParticleID                    0.0128146     0.0164887     0.0196692     0.0171458    0.00216697       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0555434     0.0743153     0.094619      0.0747041     0.0111795       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0126708     0.0165382     0.0198031     0.0170193    0.00212712       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.33475       7.6079        12.0058       7.37036       2.35066        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.06326       8.22011       12.1732       7.83154       2.24216        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.2274        7.17786       10.9759       6.42061       2.10519        10     
[art]:TriggerResults:TriggerResultInserter        3.1159e-05    4.40907e-05   0.00012595    3.35285e-05   2.75907e-05      10     
end_path:out1:RootOutput                           6.082e-06    7.3209e-06    1.3215e-05     6.613e-06    1.99425e-06      10     
end_path:out1:RootOutput(write)                     2.45099       3.49604       4.58576       3.42465      0.570876        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5315.87 MB
  Peak resident set size usage (VmHWM): 4516.16 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.05 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11477792
-rw-r--r--. 1 dune009 dune 2656964140 Nov  2 18:06 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2514598443 Nov  2 18:03 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2416272024 Nov  2 19:29 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2077938536 Nov  2 15:42 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1994792405 Nov  2 15:27 g4_stage1.root
-rw-r--r--. 1 dune009 dune   92078380 Nov  2 19:22 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242838 Nov  2 19:29 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov  2 19:29 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov  2 19:29 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov  2 19:29 time.db
-rw-r--r--. 1 dune009 dune        519 Nov  2 18:06 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  2 15:42 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov  2 19:29 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov  2 15:20 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov  2 19:29 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov  2 18:11 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov  2 18:06 debugprod.log
justIN time: 2025-11-03 18:30:47 UTC       justIN version: 01.05.01