justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279848.1@dunegpschedd01.fnal.gov

Jobsub ID279848.1@dunegpschedd01.fnal.gov
Workflow ID12002
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-17 15:54:35
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-17 20:51:50
From worker nodeHostnameheplnc170.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-17 15:56:08
Input filesmonte-carlo-012002-000001
JobscriptExit code0
Real time4h (17647s)
CPU time4h (16852s = 95%)
Max RSS bytes4475408384 (4268 MiB)
Outputting started2026-01-17 20:50:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12002/1/001/reco_stage1.root
Finished2026-01-17 20:51:50
Saved logsjustin-logs:279848.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

leKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123006 traces, 0 summary
Retagger: tagging trace set: gauss with 95513 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95513 traces tagged "gauss"
FrameSaver: q=3.24894e+06 n=1691351 tag=gauss
wclsFrameSaver: saving 123006 traces tagged "wiener"
FrameSaver: q=3.45008e+06 n=1510757 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 72193 XUs and 24399 XVs -> 6131 XUVs
C:0 T:1 31455 XUs and 24360 XVs -> 2492 XUVs
C:0 T:2 283167 XUs and 219207 XVs -> 43634 XUVs
C:0 T:3 78684 XUs and 64995 XVs -> 1589 XUVs
C:0 T:4 7404 XUs and 9308 XVs -> 274 XUVs
C:0 T:5 24146 XUs and 21168 XVs -> 4010 XUVs
C:0 T:6 101649 XUs and 97589 XVs -> 5913 XUVs
C:0 T:7 38592 XUs and 38985 XVs -> 299 XUVs
C:0 T:8 6918 XUs and 8984 XVs -> 171 XUVs
C:0 T:9 41249 XUs and 63564 XVs -> 5211 XUVs
C:0 T:10 28034 XUs and 19744 XVs -> 2377 XUVs
C:0 T:11 214 XUs and 136 XVs -> 19 XUVs
72120 XUVs total
10976 collection wire objects
63249 potential space points
Neighbour search...
14594547 tests to find 5994674 neighbours
Iterating with no regularization...
Begin: 6.47506e+07
0 5.53742e+07
1 5.46848e+07
2 5.4567e+07
3 5.45284e+07
Now with regularization...
Begin: 4.50424e+07
0 4.49853e+07
1 4.49464e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411799374
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411799374
fLifetime: 30000 [us]
17-Jan-2026 20:50:13 GMT  Closed output file "reco_stage1.root"
17-Jan-2026 20:50:13 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          450.194       579.738       854.666       529.563       124.768        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00119905    0.00514392    0.00771758     0.0052417    0.00220493       10     
reco:ophit:OpHitFinder                             0.428826      0.467442      0.502586      0.468068      0.0224079       10     
reco:opflash:OpFlashFinder                         0.241434      0.277263      0.345113      0.278479      0.0304527       10     
reco:opslicer:OpSlicer                              1.02289       1.11558       1.25368       1.10788      0.0670238       10     
reco:caldata:DataPrepModule                         7.31937       8.15433       8.46297       8.20701      0.302155        10     
reco:wclsdatasp:WireCellToolkit                     44.9851       46.4573       48.277        46.362        0.9087         10     
reco:gaushit:GausHitFinder                          1.66693       2.05795       2.41687       2.08617      0.248844        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000689047   0.000862733   0.00107705    0.000856359   0.000115198      10     
reco:reco3d:SpacePointSolver                        19.1749       31.0387       44.3384       31.5612       7.74932        10     
reco:hitpdune:DisambigFromSpacePoints               1.97134       3.12825       5.28928       2.85405       0.94659        10     
reco:pandora:StandardPandora                        35.7074       77.196        166.026       60.7954       43.0654        10     
reco:pandoraWriter:StandardPandora                 0.583357      0.766341       1.05927      0.721039      0.138301        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.9379        7.06727       10.7889       6.71348       1.86927        10     
reco:pandoraShower:LArPandoraShowerCreation         4.70819       8.34946       15.2927       7.48798       3.13675        10     
reco:pandoracalo:Calorimetry                        3.08718       4.11457       6.51164       3.57933       1.00333        10     
reco:pandoracalonosce:Calorimetry                   2.85411       3.8178        6.25511       3.34246       1.03068        10     
reco:pandorapid:Chi2ParticleID                    0.00789389     0.0099071     0.0118664    0.00946874    0.00138502       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0796071     0.0942035     0.118616      0.0910546     0.0116613       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00776397    0.00963477     0.0117605    0.00911908    0.00137197       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.29287       8.99788       18.8619       8.17867       3.68458        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.12238       8.64598       17.7217       8.00851       3.47513        10     
reco:emtrkmichelid:EmTrackMichelId                  243.85        287.175       387.295       273.89        42.9749        10     
reco:crttag:SingleCRTMatchingProducer              0.015559      0.0220999     0.0309352     0.0214718    0.00432942       10     
reco:crtreco:TwoCRTMatchingProducer               0.00813769     0.0149017     0.0219601     0.0143607    0.00393803       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.959923       1.22721       1.70664       1.20448      0.209008        10     
reco:pandora2Track:LArPandoraTrackCreation          15.8735       23.7482       44.0072       20.2286       8.59061        10     
reco:pandora2calo:Calorimetry                       8.19389       11.9505       18.9194       11.7049       3.4726         10     
reco:pandora2calonosce:Calorimetry                  7.86436       11.7885        19.44        10.9246       3.88143        10     
reco:pandora2pid:Chi2ParticleID                    0.0178208     0.0259288     0.0373948     0.0251547    0.00535842       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.100849      0.127117       0.19083      0.120559      0.0254626       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0170274     0.0249442     0.0345766     0.0248088    0.00482758       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.55265       8.39292       13.5516       7.89025       2.4259         10     
reco:pandora2Showercalo:ShowerCalorimetry           6.85984       9.86021       16.0395       9.53758       2.55685        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.5703        8.79676       14.6669       8.00141       2.41909        10     
[art]:TriggerResults:TriggerResultInserter        7.0411e-05    9.39966e-05   0.000202578   7.9784e-05    3.85175e-05      10     
end_path:out1:RootOutput                          1.2493e-05    1.45102e-05   2.2432e-05    1.35955e-05   2.7525e-06       10     
end_path:out1:RootOutput(write)                     4.15059       4.63837       6.29948       4.31624      0.660325        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5362.49 MB
  Peak resident set size usage (VmHWM): 4475.41 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11357444
-rw-r--r-- 1 pltdune04 pltdune 2630573146 Jan 17 19:10 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2496490830 Jan 17 19:07 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2364001614 Jan 17 20:50 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2069507307 Jan 17 16:23 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1982382382 Jan 17 16:05 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   86391930 Jan 17 20:44 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243150 Jan 17 20:50 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 17 20:50 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 17 20:50 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 17 20:50 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 19:10 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 16:23 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 20:50 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 17 15:56 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 17 20:50 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:19 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:10 debugprod.log
justIN time: 2026-02-05 22:02:09 UTC       justIN version: 01.06.00