justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270418.0@dunegpschedd01.fnal.gov

Jobsub ID270418.0@dunegpschedd01.fnal.gov
Workflow ID11358
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-27 16:54:54
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-27 22:01:44
From worker nodeHostnameheplnc159.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-27 16:55:13
Input filesmonte-carlo-011358-000001
JobscriptExit code0
Real time5h (18273s)
CPU time4h (17527s = 95%)
Max RSS bytes5199216640 (4958 MiB)
Outputting started2025-12-27 21:59:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11358/1/001/reco_stage1.root
Finished2025-12-27 22:01:44
Saved logsjustin-logs:270418.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 201970 traces, 0 summary
Retagger: tagging trace set: gauss with 145300 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 145300 traces tagged "gauss"
FrameSaver: q=5.15704e+06 n=2469946 tag=gauss
wclsFrameSaver: saving 201970 traces tagged "wiener"
FrameSaver: q=5.49711e+06 n=2140030 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44801 XUs and 42962 XVs -> 626 XUVs
C:0 T:1 112398 XUs and 92498 XVs -> 9256 XUVs
C:0 T:2 128702 XUs and 208025 XVs -> 8316 XUVs
C:0 T:3 58229 XUs and 80125 XVs -> 669 XUVs
C:0 T:4 12328 XUs and 9515 XVs -> 206 XUVs
C:0 T:5 70675 XUs and 41463 XVs -> 8683 XUVs
C:0 T:6 113309 XUs and 115285 XVs -> 7080 XUVs
C:0 T:7 44513 XUs and 44143 XVs -> 1126 XUVs
C:0 T:8 11156 XUs and 11115 XVs -> 183 XUVs
C:0 T:9 99712 XUs and 117132 XVs -> 14846 XUVs
C:0 T:10 114441 XUs and 131685 XVs -> 13240 XUVs
C:0 T:11 913 XUs and 1149 XVs -> 81 XUVs
64312 XUVs total
19166 collection wire objects
58223 potential space points
Neighbour search...
4860341 tests to find 2585006 neighbours
Iterating with no regularization...
Begin: 1.09023e+08
0 8.99119e+07
1 8.84266e+07
2 8.81942e+07
3 8.81272e+07
Now with regularization...
Begin: 7.16084e+07
0 7.15448e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411377798
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411377798
fLifetime: 30000 [us]
27-Dec-2025 21:59:42 GMT  Closed output file "reco_stage1.root"
27-Dec-2025 21:59:42 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          388.813       597.165       1105.95       552.648       203.498        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00151714    0.00525261    0.00836548    0.00575975     0.0021628       10     
reco:ophit:OpHitFinder                             0.356352      0.453497      0.576411      0.460671      0.065357        10     
reco:opflash:OpFlashFinder                         0.223722      0.299477      0.410727      0.311334      0.0601247       10     
reco:opslicer:OpSlicer                             0.815481       1.09934       1.63967       1.1159       0.242752        10     
reco:caldata:DataPrepModule                         4.77261       7.35444       8.25432       7.49777      0.931875        10     
reco:wclsdatasp:WireCellToolkit                     33.9655       42.5726       47.6392       43.1794       3.94784        10     
reco:gaushit:GausHitFinder                          1.37329       2.02375       3.17043       1.9663       0.533001        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00059499    0.000842002   0.00120823    0.000830648   0.000212404      10     
reco:reco3d:SpacePointSolver                        14.6936       42.935        81.479        36.2299       21.5044        10     
reco:hitpdune:DisambigFromSpacePoints               1.69334       3.31192       5.3622        2.89213       1.22902        10     
reco:pandora:StandardPandora                        34.8466       76.0534       207.362       57.9979       51.4212        10     
reco:pandoraWriter:StandardPandora                 0.566317      0.734792       1.19325      0.696217      0.177517        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.69605       8.06909       16.7542       6.83703       3.35419        10     
reco:pandoraShower:LArPandoraShowerCreation         4.3214        8.71888       19.1273       7.80484       4.27502        10     
reco:pandoracalo:Calorimetry                        2.7613        4.14929       7.31003       3.63982       1.35205        10     
reco:pandoracalonosce:Calorimetry                   2.42734       3.89532       6.6328        3.5482        1.19186        10     
reco:pandorapid:Chi2ParticleID                    0.00728821    0.00908154     0.0130384    0.00877334    0.00164594       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0632548     0.0888154      0.12195      0.0882602     0.0180984       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00708737     0.0086801     0.0106469    0.00854861    0.00120192       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.27871       9.53444       25.3479       7.74956       5.73316        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.34308       9.75426       28.3581       7.76231       6.55579        10     
reco:emtrkmichelid:EmTrackMichelId                  209.911       286.733       429.349       267.699       65.9744        10     
reco:crttag:SingleCRTMatchingProducer              0.015378      0.0222391     0.0312282     0.0205932    0.00511424       10     
reco:crtreco:TwoCRTMatchingProducer                0.0109769     0.0147416     0.0200807     0.0141956     0.0028858       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.795925       1.29271       1.86007       1.25797      0.277019        10     
reco:pandora2Track:LArPandoraTrackCreation          15.2343       26.9026       60.9156       23.0163       12.7953        10     
reco:pandora2calo:Calorimetry                       7.55669       13.2245       35.1738       10.2077       7.73684        10     
reco:pandora2calonosce:Calorimetry                  6.94333       13.0209       37.2527       10.5907       8.45772        10     
reco:pandora2pid:Chi2ParticleID                    0.0111786     0.0228407     0.0398978     0.0214096    0.00787991       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0517121      0.11651      0.184772      0.108794      0.0380409       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0111866     0.0219491     0.0374323     0.020494     0.00697466       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.63793       8.70895       18.9339       7.13066       3.97335        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.28892       11.2733       29.0179       9.37947       6.20778        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.06608       10.0396       26.2317       8.98056       5.67002        10     
[art]:TriggerResults:TriggerResultInserter        5.0254e-05    9.2926e-05    0.000247285   8.29215e-05   5.26711e-05      10     
end_path:out1:RootOutput                          1.2654e-05    1.73467e-05   2.8093e-05    1.7353e-05    4.05953e-06      10     
end_path:out1:RootOutput(write)                     2.92653       4.5387        7.45015       4.29758       1.32287        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6069.43 MB
  Peak resident set size usage (VmHWM): 5199.22 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 13056592
-rw-r--r-- 1 pltdune04 pltdune 2957709525 Dec 27 20:17 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2811967340 Dec 27 20:14 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2730594211 Dec 27 21:59 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2455624475 Dec 27 17:22 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2318211361 Dec 27 17:04 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   95230430 Dec 27 21:51 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243217 Dec 27 21:59 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 27 21:59 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 27 21:59 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 27 21:59 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 20:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 17:22 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 21:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 27 16:55 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 27 21:59 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 27 20:28 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 27 20:17 debugprod.log
justIN time: 2026-02-07 03:35:42 UTC       justIN version: 01.06.00