justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238778.1@dunegpschedd01.fnal.gov

Jobsub ID238778.1@dunegpschedd01.fnal.gov
Workflow ID9366
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 06:08:32
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-10-27 12:17:14
From worker nodeHostnamehd84.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 06:09:55
Input filesmonte-carlo-009366-000001
JobscriptExit code0
Real time6h (21942s)
CPU time5h (21299s = 97%)
Max RSS bytes4419252224 (4214 MiB)
Outputting started2025-10-27 12:15:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09366/1/001/reco_stage1.root
Finished2025-10-27 12:17:14
Saved logsjustin-logs:238778.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123232 traces, 0 summary
Retagger: tagging trace set: gauss with 94506 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94506 traces tagged "gauss"
FrameSaver: q=3.19621e+06 n=1680130 tag=gauss
wclsFrameSaver: saving 123232 traces tagged "wiener"
FrameSaver: q=3.38913e+06 n=1494062 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32975 XUs and 26586 XVs -> 316 XUVs
C:0 T:1 50817 XUs and 43169 XVs -> 3270 XUVs
C:0 T:2 48957 XUs and 80257 XVs -> 3308 XUVs
C:0 T:3 24128 XUs and 31206 XVs -> 128 XUVs
C:0 T:4 13436 XUs and 12045 XVs -> 128 XUVs
C:0 T:5 59542 XUs and 41244 XVs -> 6055 XUVs
C:0 T:6 95321 XUs and 100669 XVs -> 4816 XUVs
C:0 T:7 38316 XUs and 38245 XVs -> 247 XUVs
C:0 T:8 8599 XUs and 10446 XVs -> 103 XUVs
C:0 T:9 54595 XUs and 92504 XVs -> 4648 XUVs
C:0 T:10 37728 XUs and 21564 XVs -> 3471 XUVs
C:0 T:11 172 XUs and 102 XVs -> 11 XUVs
26501 XUVs total
10986 collection wire objects
24746 potential space points
Neighbour search...
1329984 tests to find 726742 neighbours
Iterating with no regularization...
Begin: 6.37506e+07
0 5.40221e+07
1 5.33424e+07
2 5.32471e+07
3 5.32188e+07
Now with regularization...
Begin: 4.47716e+07
0 4.47523e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410141680
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410141680
fLifetime: 30000 [us]
27-Oct-2025 12:15:36 UTC  Closed output file "reco_stage1.root"
27-Oct-2025 12:15:36 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          486.962       626.042       890.347       608.374       118.425        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00120139     0.0045162    0.00671313     0.0055307    0.00179653       10     
reco:ophit:OpHitFinder                             0.328534      0.382214      0.429296      0.389139      0.033338        10     
reco:opflash:OpFlashFinder                         0.193615      0.256816      0.306285      0.264233      0.0355033       10     
reco:opslicer:OpSlicer                             0.984546       1.17284       1.36114       1.16064      0.121829        10     
reco:caldata:DataPrepModule                         7.23995       7.57892       8.30505       7.52992      0.304393        10     
reco:wclsdatasp:WireCellToolkit                     52.4101       57.9071       64.5018       57.5022       3.77698        10     
reco:gaushit:GausHitFinder                          1.8121        2.09417       2.43816       2.15231      0.214389        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000600724   0.000777588   0.00110553    0.000700433   0.000180598      10     
reco:reco3d:SpacePointSolver                        12.1272       25.9861       67.6795       19.9137       15.5968        10     
reco:hitpdune:DisambigFromSpacePoints               2.12614       2.80635       4.08981       2.68915      0.598642        10     
reco:pandora:StandardPandora                        37.9334       75.1847       194.936       67.2049       42.3628        10     
reco:pandoraWriter:StandardPandora                 0.596496      0.682723      0.875083      0.659203      0.0875405       10     
reco:pandoraTrack:LArPandoraTrackCreation           6.05109       8.42435       12.574        8.08074       2.12438        10     
reco:pandoraShower:LArPandoraShowerCreation         6.45751       10.4838       17.191        9.74512       3.4822         10     
reco:pandoracalo:Calorimetry                        3.45318       4.64988       6.68571       4.35364       1.07912        10     
reco:pandoracalonosce:Calorimetry                   3.14552       4.33245       6.34107       4.06119        1.034         10     
reco:pandorapid:Chi2ParticleID                    0.00814259    0.00945949     0.0120307    0.00914251    0.00118074       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0748235     0.0935253     0.112366      0.090685      0.0110871       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00772388    0.00890495     0.0103854    0.00890732    0.000790674      10     
reco:pandoraShowercalo:ShowerCalorimetry            6.78099       10.4232       16.3016       9.79985       3.04781        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.48489       9.95741       14.9069       9.53001       2.91989        10     
reco:emtrkmichelid:EmTrackMichelId                  267.275       314.434       375.861       311.268       34.063         10     
reco:crttag:SingleCRTMatchingProducer              0.0191584     0.0254275     0.0301499     0.0267439    0.00395219       10     
reco:crtreco:TwoCRTMatchingProducer                0.0145428     0.0166487     0.0201292     0.0167497    0.00170051       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.1307        1.37193       1.69903       1.37278      0.200951        10     
reco:pandora2Track:LArPandoraTrackCreation          19.5703       26.9395       39.7897       25.6914       6.18996        10     
reco:pandora2calo:Calorimetry                       9.22409       13.4212       20.2417       13.082        3.30962        10     
reco:pandora2calonosce:Calorimetry                  8.71562       12.6395       18.3258       12.333        2.97411        10     
reco:pandora2pid:Chi2ParticleID                    0.0198307     0.0245361     0.0308257     0.0250373    0.00315379       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0990927     0.118257       0.15394      0.117749      0.0162915       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.019411      0.0244252     0.0295334     0.0248907    0.00328755       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.12792       9.58783       14.4114       9.03669       2.26426        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.92581       11.0272       16.3278       10.4363       2.59601        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.01946       9.51872       13.6067       9.05787       2.11136        10     
[art]:TriggerResults:TriggerResultInserter        6.8659e-05    8.3129e-05    0.000181292   7.16305e-05   3.28698e-05      10     
end_path:out1:RootOutput                          1.2223e-05     1.531e-05    2.6921e-05    1.41565e-05   3.98462e-06      10     
end_path:out1:RootOutput(write)                     3.57386       4.28301       5.43808       4.2383       0.571411        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5242.4 MB
  Peak resident set size usage (VmHWM): 4419.25 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10982684
-rw-r--r-- 1 dnpil001 users 2547280818 Oct 27 10:27 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2418321243 Oct 27 10:24 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2261184931 Oct 27 12:15 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 2003635767 Oct 27 06:39 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1933476406 Oct 27 06:18 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   81754110 Oct 27 12:09 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243215 Oct 27 12:15 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Oct 27 12:15 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Oct 27 12:15 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Oct 27 12:15 time.db
-rw-r--r-- 1 dnpil001 users        519 Oct 27 10:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Oct 27 06:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Oct 27 12:15 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Oct 27 06:09 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Oct 27 12:15 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Oct 27 10:37 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Oct 27 10:27 debugprod.log
justIN time: 2025-11-05 04:51:49 UTC       justIN version: 01.05.01