justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 240826.0@dunegpschedd01.fnal.gov

Jobsub ID240826.0@dunegpschedd01.fnal.gov
Workflow ID9457
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-30 07:25:23
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-10-30 12:49:29
From worker nodeHostnamecn002.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 07:26:25
Input filesmonte-carlo-009457-000001
JobscriptExit code0
Real time5h (19167s)
CPU time5h (18811s = 98%)
Max RSS bytes4318150656 (4118 MiB)
Outputting started2025-10-30 12:45:53
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09457/1/001/reco_stage1.root
Finished2025-10-30 12:49:29
Saved logsjustin-logs:240826.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Retagger: tagging trace set: wiener with 146886 traces, 0 summary
Retagger: tagging trace set: gauss with 109276 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 109276 traces tagged "gauss"
FrameSaver: q=3.66338e+06 n=1864849 tag=gauss
wclsFrameSaver: saving 146886 traces tagged "wiener"
FrameSaver: q=3.89501e+06 n=1619834 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41013 XUs and 43059 XVs -> 2521 XUVs
C:0 T:1 145177 XUs and 161167 XVs -> 37946 XUVs
C:0 T:2 109686 XUs and 120069 XVs -> 5837 XUVs
C:0 T:3 54814 XUs and 47815 XVs -> 408 XUVs
C:0 T:4 11306 XUs and 8414 XVs -> 94 XUVs
C:0 T:5 70085 XUs and 44123 XVs -> 9209 XUVs
C:0 T:6 116883 XUs and 120343 XVs -> 4564 XUVs
C:0 T:7 44668 XUs and 47174 XVs -> 447 XUVs
C:0 T:8 7240 XUs and 8051 XVs -> 69 XUVs
C:0 T:9 73473 XUs and 72014 XVs -> 10513 XUVs
C:0 T:10 38432 XUs and 26352 XVs -> 6249 XUVs
C:0 T:11 42 XUs and 25 XVs -> 3 XUVs
77860 XUVs total
14084 collection wire objects
70192 potential space points
Neighbour search...
16130332 tests to find 6851212 neighbours
Iterating with no regularization...
Begin: 6.8341e+07
0 5.87973e+07
1 5.81193e+07
2 5.80161e+07
3 5.79878e+07
Now with regularization...
Begin: 4.66052e+07
0 4.65548e+07
1 4.6501e+07
2 4.6456e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410203032
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410203032
fLifetime: 30000 [us]
30-Oct-2025 12:45:50 GMT  Closed output file "reco_stage1.root"
30-Oct-2025 12:45:50 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          543.974       778.326       1177.35       735.521       189.493        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00194395    0.00480943    0.00736226    0.00547218    0.00191745       10     
reco:ophit:OpHitFinder                             0.320353      0.400805      0.564089      0.370501      0.0748351       10     
reco:opflash:OpFlashFinder                         0.216561      0.274926      0.402662       0.24706      0.0577022       10     
reco:opslicer:OpSlicer                             0.825743       1.01572       1.34791      0.954017      0.179873        10     
reco:caldata:DataPrepModule                         7.28786       8.54619       10.407        8.1802        1.00156        10     
reco:wclsdatasp:WireCellToolkit                     56.0998       62.5011       74.3894       59.8188       5.59517        10     
reco:gaushit:GausHitFinder                          1.46832       2.05174       2.82651       1.88129      0.462382        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000891344   0.00109782    0.00140903    0.00102677    0.000182236      10     
reco:reco3d:SpacePointSolver                        11.762        44.0037        84.19        43.3181       23.3622        10     
reco:hitpdune:DisambigFromSpacePoints               2.3268        3.98155       7.30299       3.32805       1.67232        10     
reco:pandora:StandardPandora                        39.2726       72.8089       164.517       58.4302       37.0899        10     
reco:pandoraWriter:StandardPandora                 0.589185       0.76246       1.06451       0.74603      0.158137        10     
reco:pandoraTrack:LArPandoraTrackCreation           12.9094       18.3172       30.9474       16.2293       5.44882        10     
reco:pandoraShower:LArPandoraShowerCreation         11.8618       20.1611       31.4024       20.1856       5.95128        10     
reco:pandoracalo:Calorimetry                        5.94429       8.75963        13.93        7.83977       2.51525        10     
reco:pandoracalonosce:Calorimetry                   5.7639        8.36513       13.5382       7.54155       2.40502        10     
reco:pandorapid:Chi2ParticleID                    0.00748025     0.0100952     0.0143433    0.00915095    0.00265552       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0685033     0.0912591     0.123513      0.0902235     0.0149353       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00750648    0.00887753     0.0127671    0.00825372    0.00164544       10     
reco:pandoraShowercalo:ShowerCalorimetry            11.1941       20.1758       29.8262       20.2161       6.06295        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       11.0967       19.7559       28.979        19.7201       5.90753        10     
reco:emtrkmichelid:EmTrackMichelId                  246.48        310.874       433.312       303.319       60.1212        10     
reco:crttag:SingleCRTMatchingProducer              0.0320752     0.0465389     0.0680125     0.0444685     0.010354        10     
reco:crtreco:TwoCRTMatchingProducer                0.0254007     0.0377679     0.0483557     0.0370766    0.00764444       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.14589       1.44194       1.87293       1.44423      0.220863        10     
reco:pandora2Track:LArPandoraTrackCreation          36.7933        58.27        90.996        56.9119       17.5922        10     
reco:pandora2calo:Calorimetry                       16.2735       26.9211       40.6737       26.9007       8.12407        10     
reco:pandora2calonosce:Calorimetry                  15.533        25.4441       37.7891       25.3475       7.47537        10     
reco:pandora2pid:Chi2ParticleID                    0.0159662     0.0221333     0.0323282     0.0214084    0.00463687       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0912842     0.115551      0.165644      0.112902      0.0208225       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0154894     0.0209505     0.0276132     0.0203674    0.00356175       10     
reco:pandora2Shower:LArPandoraShowerCreation        12.6582       17.5186       27.1695       16.3738       4.46855        10     
reco:pandora2Showercalo:ShowerCalorimetry           14.2868       21.3779       32.7043       20.7731       5.52157        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      12.8565       19.9649       31.8393       17.5905       6.35217        10     
[art]:TriggerResults:TriggerResultInserter        7.1839e-05    0.000105465   0.000189743   9.5396e-05    3.45316e-05      10     
end_path:out1:RootOutput                          1.0628e-05    1.59477e-05   2.9665e-05    1.37175e-05   5.50602e-06      10     
end_path:out1:RootOutput(write)                     3.02286       4.03002       5.54614       3.83634      0.835308        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5121.84 MB
  Peak resident set size usage (VmHWM): 4318.15 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.00 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10972788
-rw-r--r-- 1 pildune32 pildune 2543369647 Oct 30 10:32 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2411987471 Oct 30 10:28 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2249335658 Oct 30 12:45 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2012348185 Oct 30 08:03 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1938770839 Oct 30 07:36 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   79709980 Oct 30 12:36 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243479 Oct 30 12:45 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Oct 30 12:45 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Oct 30 12:45 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Oct 30 12:45 time.db
-rw-r--r-- 1 pildune32 pildune        519 Oct 30 10:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 30 08:03 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 30 12:45 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Oct 30 07:26 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Oct 30 12:45 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Oct 30 10:45 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Oct 30 10:32 debugprod.log
justIN time: 2025-11-04 01:55:17 UTC       justIN version: 01.05.01