justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 301884.1@dunegpschedd01.fnal.gov

Jobsub ID301884.1@dunegpschedd01.fnal.gov
Workflow ID12460
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-28 08:06:25
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-28 11:54:56
From worker nodeHostnameheplnc164.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-28 08:07:43
Input filesmonte-carlo-012460-000001
JobscriptExit code0
Real time3h (13079s)
CPU time3h (12322s = 94%)
Max RSS bytes3857616896 (3678 MiB)
Outputting started2026-01-28 11:45:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12460/1/001/reco_stage1.root
Finished2026-01-28 11:54:56
Saved logsjustin-logs:301884.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

e:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 124137 traces, 0 summary
Retagger: tagging trace set: gauss with 90445 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90445 traces tagged "gauss"
FrameSaver: q=3.11632e+06 n=1591288 tag=gauss
wclsFrameSaver: saving 124137 traces tagged "wiener"
FrameSaver: q=3.32706e+06 n=1393747 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 17283 XUs and 26910 XVs -> 208 XUVs
C:0 T:1 39994 XUs and 57734 XVs -> 4735 XUVs
C:0 T:2 32172 XUs and 67310 XVs -> 2667 XUVs
C:0 T:3 18685 XUs and 29937 XVs -> 200 XUVs
C:0 T:4 17282 XUs and 17205 XVs -> 173 XUVs
C:0 T:5 83368 XUs and 70852 XVs -> 14218 XUVs
C:0 T:6 131492 XUs and 117967 XVs -> 4098 XUVs
C:0 T:7 53818 XUs and 42454 XVs -> 388 XUVs
C:0 T:8 9445 XUs and 7611 XVs -> 161 XUVs
C:0 T:9 68705 XUs and 74533 XVs -> 7176 XUVs
C:0 T:10 35289 XUs and 14607 XVs -> 3083 XUVs
C:0 T:11 8 XUs and 4 XVs -> 2 XUVs
37109 XUVs total
12070 collection wire objects
34990 potential space points
Neighbour search...
4012888 tests to find 1918674 neighbours
Iterating with no regularization...
Begin: 7.62249e+07
0 6.45589e+07
1 6.37356e+07
2 6.36086e+07
3 6.357e+07
Now with regularization...
Begin: 5.42011e+07
0 5.4163e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412014144
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412014144
fLifetime: 30000 [us]
28-Jan-2026 11:45:40 GMT  Closed output file "reco_stage1.root"
28-Jan-2026 11:45:40 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          238.385       413.515       583.617       425.316       86.3095        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00164365    0.00480086    0.00766934    0.00453433    0.00209885       10     
reco:ophit:OpHitFinder                             0.317866      0.374923      0.418993      0.374784      0.0315295       10     
reco:opflash:OpFlashFinder                         0.119007      0.175271      0.221742       0.17801      0.0315696       10     
reco:opslicer:OpSlicer                             0.634773      0.856221      0.995176      0.860321      0.111113        10     
reco:caldata:DataPrepModule                          7.907        8.22852       8.80108       8.15041      0.276576        10     
reco:wclsdatasp:WireCellToolkit                     42.5957        45.16        46.7451       45.3067       1.27534        10     
reco:gaushit:GausHitFinder                         0.824463       1.54394       2.16849       1.57873      0.371999        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000492175   0.000755971   0.00108558    0.000743362   0.000180134      10     
reco:reco3d:SpacePointSolver                        7.43983       16.5412       21.1137       17.6045       3.78671        10     
reco:hitpdune:DisambigFromSpacePoints              0.663308       1.92619       3.08003       1.96256      0.586821        10     
reco:pandora:StandardPandora                        9.85478       39.9626       74.5326       36.395        18.2957        10     
reco:pandoraWriter:StandardPandora                 0.309413      0.597332      0.734143      0.628493      0.119676        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.65884       4.52482       6.96259       4.92364       1.36205        10     
reco:pandoraShower:LArPandoraShowerCreation         1.07904       4.79791       11.9126       4.76298       2.64368        10     
reco:pandoracalo:Calorimetry                       0.845973       2.90095       4.46889       2.90885       0.99824        10     
reco:pandoracalonosce:Calorimetry                  0.760188       2.67847       4.45796       2.63024       1.02962        10     
reco:pandorapid:Chi2ParticleID                    0.00384509    0.00782348     0.0100851    0.00824915    0.00167477       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0306373     0.0778509     0.0999625     0.0798263     0.019259        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00381669    0.00764359    0.00878319    0.00807788    0.00145851       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.31353       5.1208        10.8842       5.21043       2.43418        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.59444       5.27886       12.8346       5.14339       2.85673        10     
reco:emtrkmichelid:EmTrackMichelId                  138.964       220.609       287.947       231.239       37.6767        10     
reco:crttag:SingleCRTMatchingProducer              0.0126391     0.0222342     0.0351998     0.0193779    0.00834673       10     
reco:crtreco:TwoCRTMatchingProducer               0.00677902     0.0121728     0.0199265     0.0123297    0.00358492       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.755529      0.991705       1.21651      0.983547      0.145569        10     
reco:pandora2Track:LArPandoraTrackCreation          5.30939       14.5722       29.2915       14.6722       5.91935        10     
reco:pandora2calo:Calorimetry                       2.89324       7.87356       14.7857       8.32844       3.04465        10     
reco:pandora2calonosce:Calorimetry                  2.67293       7.25122       13.2408       7.85506       2.71991        10     
reco:pandora2pid:Chi2ParticleID                    0.012753      0.0182856     0.0253374     0.0181537    0.00318038       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.058385      0.0960325     0.127566      0.0992169     0.0178885       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0108912     0.0173073     0.0246001     0.0177441    0.00358756       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.92729       5.37969       10.8906       5.46736       2.17737        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.64195       6.38377       11.7864       6.89853       2.51959        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.98972       5.7604        10.3483       6.06961       2.1578         10     
[art]:TriggerResults:TriggerResultInserter        7.1875e-05    0.000109301   0.000229502   9.43325e-05   4.52782e-05      10     
end_path:out1:RootOutput                          1.2894e-05    1.81341e-05   2.9896e-05    1.6326e-05    5.32129e-06      10     
end_path:out1:RootOutput(write)                     2.4397        3.59329       4.72931       3.64304      0.578692        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4712.24 MB
  Peak resident set size usage (VmHWM): 3857.62 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 8907448
-rw-r--r-- 1 pltdune04 pltdune 2147416834 Jan 28 10:34 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2042781133 Jan 28 10:31 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1792505758 Jan 28 11:45 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1572813079 Jan 28 08:29 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1499765950 Jan 28 08:17 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   65323960 Jan 28 11:40 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242592 Jan 28 11:45 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 28 11:45 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 28 11:45 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 28 11:45 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 10:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 08:29 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 28 11:45 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 28 08:07 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 28 11:45 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 28 10:41 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 28 10:34 debugprod.log
justIN time: 2026-02-04 18:09:34 UTC       justIN version: 01.06.00