justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 280542.0@dunegpschedd01.fnal.gov

Jobsub ID280542.0@dunegpschedd01.fnal.gov
Workflow ID12044
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-18 06:05:29
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-18 11:12:37
From worker nodeHostnameheplnc162.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-18 06:05:55
Input filesmonte-carlo-012044-000001
JobscriptExit code0
Real time5h (18295s)
CPU time4h (17570s = 96%)
Max RSS bytes4449742848 (4243 MiB)
Outputting started2026-01-18 11:10:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12044/1/001/reco_stage1.root
Finished2026-01-18 11:12:37
Saved logsjustin-logs:280542.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

is
Retagger: tagging trace set: wiener with 187130 traces, 0 summary
Retagger: tagging trace set: gauss with 141819 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 141819 traces tagged "gauss"
FrameSaver: q=4.98205e+06 n=2535032 tag=gauss
wclsFrameSaver: saving 187130 traces tagged "wiener"
FrameSaver: q=5.28529e+06 n=2254647 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 54894 XUs and 53974 XVs -> 1421 XUVs
C:0 T:1 131388 XUs and 134779 XVs -> 18555 XUVs
C:0 T:2 122356 XUs and 121820 XVs -> 4928 XUVs
C:0 T:3 61072 XUs and 55136 XVs -> 566 XUVs
C:0 T:4 21781 XUs and 23110 XVs -> 251 XUVs
C:0 T:5 105354 XUs and 87608 XVs -> 7727 XUVs
C:0 T:6 211054 XUs and 207669 XVs -> 12559 XUVs
C:0 T:7 70364 XUs and 70641 XVs -> 1186 XUVs
C:0 T:8 9892 XUs and 9283 XVs -> 138 XUVs
C:0 T:9 67064 XUs and 78061 XVs -> 6322 XUVs
C:0 T:10 66223 XUs and 25293 XVs -> 8012 XUVs
C:0 T:11 136 XUs and 68 XVs -> 15 XUVs
61680 XUVs total
16043 collection wire objects
54647 potential space points
Neighbour search...
6756757 tests to find 2986558 neighbours
Iterating with no regularization...
Begin: 9.3474e+07
0 7.51273e+07
1 7.37062e+07
2 7.34962e+07
3 7.34292e+07
Now with regularization...
Begin: 5.99548e+07
0 5.9902e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411811277
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411811277
fLifetime: 30000 [us]
18-Jan-2026 11:10:47 GMT  Closed output file "reco_stage1.root"
18-Jan-2026 11:10:47 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          410.15        600.876       797.154       632.389       113.148        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00138242    0.00510185    0.00809434    0.00519076    0.00221032       10     
reco:ophit:OpHitFinder                             0.361093      0.480063      0.613908      0.471252      0.0657704       10     
reco:opflash:OpFlashFinder                          0.22391      0.294972      0.369441      0.298803      0.0478003       10     
reco:opslicer:OpSlicer                             0.895219       1.13093       1.56725       1.11259      0.184354        10     
reco:caldata:DataPrepModule                         7.10733       8.05693       8.70262       8.02874       0.40219        10     
reco:wclsdatasp:WireCellToolkit                     43.8333       45.8566       47.4748       45.689        1.27441        10     
reco:gaushit:GausHitFinder                          1.41191       2.09062       2.53112       2.13204      0.319685        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000645841   0.000895368   0.00129137    0.000873998   0.000179416      10     
reco:reco3d:SpacePointSolver                        21.3397       34.8312       50.6352       33.2595       9.54549        10     
reco:hitpdune:DisambigFromSpacePoints               2.12037       3.38719       4.92354       3.49987      0.817597        10     
reco:pandora:StandardPandora                        28.5946       69.2895       112.103       69.9013       23.3918        10     
reco:pandoraWriter:StandardPandora                 0.545495       0.77009      0.917165      0.796263      0.116799        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.10664       8.05853       10.9071       8.18447       2.0172         10     
reco:pandoraShower:LArPandoraShowerCreation         5.02498       9.43875       16.3349       9.57717       3.46032        10     
reco:pandoracalo:Calorimetry                        2.68474       4.10789       5.19035       4.27476      0.854772        10     
reco:pandoracalonosce:Calorimetry                   2.51054       3.95832       5.15284       4.12315      0.902485        10     
reco:pandorapid:Chi2ParticleID                    0.00758418    0.00970459     0.0120107    0.00959197    0.00119828       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0749464     0.0997836     0.115053      0.101121      0.0116744       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00769938    0.00931133     0.010456     0.00949304    0.000971692      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.4637        10.0672       15.5959       10.9593       3.17191        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.79829       9.59599       15.7853       10.1591       3.08511        10     
reco:emtrkmichelid:EmTrackMichelId                  222.866       299.847       367.067       312.383       42.2467        10     
reco:crttag:SingleCRTMatchingProducer              0.0144864     0.0276276     0.0461756     0.0262785    0.00990312       10     
reco:crtreco:TwoCRTMatchingProducer               0.00920612     0.0149149     0.0204989     0.0153201     0.0031356       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.965073       1.24443       1.56179       1.25918      0.189632        10     
reco:pandora2Track:LArPandoraTrackCreation          15.5244       27.3237       43.2431       27.7669       9.33848        10     
reco:pandora2calo:Calorimetry                       7.5593        13.102        20.703        14.1784       3.95674        10     
reco:pandora2calonosce:Calorimetry                  7.13355       12.5272       18.265        12.952        3.57364        10     
reco:pandora2pid:Chi2ParticleID                    0.0183956     0.0256826     0.0337315     0.0262784    0.00399931       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0915331      0.12623      0.158463       0.12792      0.0200754       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0176639     0.025155      0.0319994     0.025936     0.00403745       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.35946       9.42935       14.3665       9.71745       2.92071        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.55057       10.971        15.371        11.4364       2.80805        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.74128       9.76097       13.8867       10.3172       2.5187         10     
[art]:TriggerResults:TriggerResultInserter        6.7317e-05    8.9505e-05    0.000195316   7.85225e-05   3.61498e-05      10     
end_path:out1:RootOutput                          1.1752e-05    1.4877e-05    2.1971e-05    1.4322e-05    2.57585e-06      10     
end_path:out1:RootOutput(write)                     3.46995       4.73407        5.846        4.87493      0.728232        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5355.28 MB
  Peak resident set size usage (VmHWM): 4449.74 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.12 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11788320
-rw-r--r-- 1 pltdune04 pltdune 2701750026 Jan 18 09:27 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2560729201 Jan 18 09:24 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2456070776 Jan 18 11:10 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2172580900 Jan 18 06:31 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2087846268 Jan 18 06:13 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   91647070 Jan 18 11:01 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242905 Jan 18 11:10 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 18 11:10 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 18 11:10 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 18 11:10 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 18 09:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 18 06:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 18 11:10 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 18 06:05 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 18 11:10 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 18 09:38 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 18 09:27 debugprod.log
justIN time: 2026-02-05 19:47:57 UTC       justIN version: 01.06.00