justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301804.0@dunegpschedd01.fnal.gov

Jobsub ID301804.0@dunegpschedd01.fnal.gov
Workflow ID12452
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-27 21:01:50
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-28 01:12:31
From worker nodeHostnameheplnc164.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 21:03:24
Input filesmonte-carlo-012452-000001
JobscriptExit code0
Real time4h (14653s)
CPU time3h (13845s = 94%)
Max RSS bytes3844263936 (3666 MiB)
Outputting started2026-01-28 01:07:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12452/1/001/reco_stage1.root
Finished2026-01-28 01:12:31
Saved logsjustin-logs:301804.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123671 traces, 0 summary
Retagger: tagging trace set: gauss with 95177 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95177 traces tagged "gauss"
FrameSaver: q=3.31346e+06 n=1679855 tag=gauss
wclsFrameSaver: saving 123671 traces tagged "wiener"
FrameSaver: q=3.50123e+06 n=1497800 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 47328 XUs and 55982 XVs -> 1028 XUVs
C:0 T:1 93525 XUs and 106963 XVs -> 5390 XUVs
C:0 T:2 58111 XUs and 82529 XVs -> 2442 XUVs
C:0 T:3 32077 XUs and 37238 XVs -> 144 XUVs
C:0 T:4 7581 XUs and 13574 XVs -> 432 XUVs
C:0 T:5 35339 XUs and 55828 XVs -> 5481 XUVs
C:0 T:6 79585 XUs and 100898 XVs -> 5999 XUVs
C:0 T:7 32962 XUs and 42699 XVs -> 867 XUVs
C:0 T:8 5433 XUs and 6781 XVs -> 36 XUVs
C:0 T:9 33454 XUs and 60646 XVs -> 5067 XUVs
C:0 T:10 21484 XUs and 15775 XVs -> 2556 XUVs
C:0 T:11 221 XUs and 169 XVs -> 27 XUVs
29469 XUVs total
9764 collection wire objects
25873 potential space points
Neighbour search...
1332733 tests to find 700878 neighbours
Iterating with no regularization...
Begin: 7.67154e+07
0 6.49023e+07
1 6.41761e+07
2 6.40766e+07
3 6.40486e+07
Now with regularization...
Begin: 5.49741e+07
0 5.49668e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412004835
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412004835
fLifetime: 30000 [us]
28-Jan-2026 01:07:36 GMT  Closed output file "reco_stage1.root"
28-Jan-2026 01:07:36 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          371.223       457.234       565.936       456.829       54.8685        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00184079    0.00503555    0.00788202    0.00562038     0.0020542       10     
reco:ophit:OpHitFinder                              0.34239      0.389502      0.434724      0.392723      0.0265987       10     
reco:opflash:OpFlashFinder                         0.150534      0.196465      0.226274      0.197099       0.02138        10     
reco:opslicer:OpSlicer                             0.780763      0.930017       1.13929      0.915754      0.101504        10     
reco:caldata:DataPrepModule                         7.23662       8.0357        8.55465       8.08874      0.356379        10     
reco:wclsdatasp:WireCellToolkit                     43.1427       45.1974       47.0421       45.3366       1.23529        10     
reco:gaushit:GausHitFinder                          1.45012       1.74144       2.16649       1.68768      0.205592        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000637549   0.000774846   0.000982196   0.00075758     0.0001048       10     
reco:reco3d:SpacePointSolver                        14.9973       22.6433       31.9727       23.8884       4.93273        10     
reco:hitpdune:DisambigFromSpacePoints               1.72429       2.20313       2.9649        2.07612      0.402343        10     
reco:pandora:StandardPandora                        24.8568       41.1898       53.6143       41.7348       8.63932        10     
reco:pandoraWriter:StandardPandora                  0.5683       0.670962      0.785474      0.668074      0.0679233       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.94324       5.46352       7.59828       5.38165      0.915676        10     
reco:pandoraShower:LArPandoraShowerCreation         3.70202       6.12934       10.792        5.51939       1.97858        10     
reco:pandoracalo:Calorimetry                        2.00915       3.30625       4.49596       3.40284      0.662376        10     
reco:pandoracalonosce:Calorimetry                   1.94602       2.9536        3.88674       3.0166       0.528256        10     
reco:pandorapid:Chi2ParticleID                    0.00697737    0.00858522     0.0105881    0.00881518    0.00112927       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0742617     0.0835063     0.098472      0.0826164    0.00649967       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00723941    0.00822146    0.00919809     0.0081998    0.000590868      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.65087       6.70601       13.1581       6.19286       2.58637        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.44651       6.38141       10.8316       5.60381       2.12401        10     
reco:emtrkmichelid:EmTrackMichelId                  209.341       241.198       280.423       241.176       21.3742        10     
reco:crttag:SingleCRTMatchingProducer              0.0172023     0.0326969     0.0677156     0.0248408     0.0176292       10     
reco:crtreco:TwoCRTMatchingProducer                0.0100918     0.0140895     0.0209498     0.0126644    0.00306996       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.934821       1.08158       1.35555       1.04247      0.122577        10     
reco:pandora2Track:LArPandoraTrackCreation          10.4761       17.7474       26.4725       17.2143       4.48314        10     
reco:pandora2calo:Calorimetry                       5.36086       9.41185       14.0224       9.08688       2.51316        10     
reco:pandora2calonosce:Calorimetry                  5.48002       8.53528       13.4286       7.80802       2.37697        10     
reco:pandora2pid:Chi2ParticleID                    0.0155859     0.0212129     0.028078      0.0209791     0.0032897       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0944074     0.107946      0.128027      0.106291      0.0106983       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0154557     0.0209571     0.0261118     0.0208831    0.00309748       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.62267       6.06649       8.74155       5.64758       1.26402        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.19177       7.68549       10.8654       7.5191        1.67395        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.84418       6.9801        10.0819       6.49009       1.56957        10     
[art]:TriggerResults:TriggerResultInserter        8.4709e-05    0.000114673   0.000290436   9.58455e-05   5.91831e-05      10     
end_path:out1:RootOutput                          1.4547e-05    2.19373e-05   6.4983e-05    1.62355e-05   1.45995e-05      10     
end_path:out1:RootOutput(write)                     3.36507       3.91048       4.56655       3.83072      0.356033        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4713.89 MB
  Peak resident set size usage (VmHWM): 3844.26 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9335484
-rw-r--r-- 1 pltdune04 pltdune 2224706062 Jan 27 23:48 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2112485485 Jan 27 23:46 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1896961426 Jan 28 01:07 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1661758280 Jan 27 21:27 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1591684733 Jan 27 21:13 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   71323030 Jan 28 01:02 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243004 Jan 28 01:07 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 28 01:07 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 28 01:07 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 28 01:07 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 27 23:48 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 27 21:27 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 01:07 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 27 21:03 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 28 01:07 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 27 23:56 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 27 23:48 debugprod.log
justIN time: 2026-02-04 18:09:23 UTC       justIN version: 01.06.00