justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264533.1@dunegpschedd02.fnal.gov

Jobsub ID264533.1@dunegpschedd02.fnal.gov
Workflow ID11353
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-27 10:06:34
SiteUK_Lancaster
EntryUBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat2025-12-27 13:35:01
From worker nodeHostnamecomp02-22
cpuinfoIntel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-27 10:14:20
Input filesmonte-carlo-011353-000001
JobscriptExit code0
Real time3h (11952s)
CPU time3h (11796s = 98%)
Max RSS bytes4422889472 (4217 MiB)
Outputting started2025-12-27 13:33:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11353/1/001/reco_stage1.root
Finished2025-12-27 13:35:01
Saved logsjustin-logs:264533.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144141 traces, 0 summary
Retagger: tagging trace set: gauss with 110839 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110839 traces tagged "gauss"
FrameSaver: q=3.78737e+06 n=2004916 tag=gauss
wclsFrameSaver: saving 144141 traces tagged "wiener"
FrameSaver: q=4.01664e+06 n=1787016 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 66537 XUs and 51359 XVs -> 1046 XUVs
C:0 T:1 184225 XUs and 134547 XVs -> 10998 XUVs
C:0 T:2 51797 XUs and 65442 XVs -> 2098 XUVs
C:0 T:3 28275 XUs and 33138 XVs -> 151 XUVs
C:0 T:4 14801 XUs and 15241 XVs -> 163 XUVs
C:0 T:5 80104 XUs and 56689 XVs -> 7092 XUVs
C:0 T:6 93039 XUs and 86017 XVs -> 3706 XUVs
C:0 T:7 39370 XUs and 34506 XVs -> 271 XUVs
C:0 T:8 8872 XUs and 5300 XVs -> 62 XUVs
C:0 T:9 53780 XUs and 50543 XVs -> 3247 XUVs
C:0 T:10 58301 XUs and 63487 XVs -> 10450 XUVs
C:0 T:11 180 XUs and 147 XVs -> 20 XUVs
39304 XUVs total
13151 collection wire objects
36449 potential space points
Neighbour search...
2859085 tests to find 1489312 neighbours
Iterating with no regularization...
Begin: 6.8419e+07
0 5.48101e+07
1 5.39229e+07
2 5.38033e+07
3 5.37673e+07
Now with regularization...
Begin: 4.44625e+07
0 4.44206e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411372102
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411372102
fLifetime: 30000 [us]
27-Dec-2025 13:33:30 GMT  Closed output file "reco_stage1.root"
27-Dec-2025 13:33:30 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          302.845       415.927       546.233       418.517       61.7653        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000934975   0.00350329    0.00532378    0.00377169    0.00154612       10     
reco:ophit:OpHitFinder                             0.248282      0.293074      0.316793      0.294741      0.0194206       10     
reco:opflash:OpFlashFinder                         0.171817      0.212046      0.238047      0.215551      0.0181568       10     
reco:opslicer:OpSlicer                             0.559066      0.741157      0.905882      0.757736      0.105853        10     
reco:caldata:DataPrepModule                         5.32544       5.43958       5.58937       5.45126      0.0791246       10     
reco:wclsdatasp:WireCellToolkit                     43.0464       46.6491       48.6598       46.7603       1.55907        10     
reco:gaushit:GausHitFinder                          1.24391       1.54039       1.76938       1.58061      0.175519        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000520894   0.000678535   0.000824337   0.000674091   7.91611e-05      10     
reco:reco3d:SpacePointSolver                        11.994        18.004        33.096         14.5         7.42328        10     
reco:hitpdune:DisambigFromSpacePoints               1.81524       2.87247       4.11916       2.93653      0.590236        10     
reco:pandora:StandardPandora                        28.1466       53.7951       92.9575       52.8061       17.3904        10     
reco:pandoraWriter:StandardPandora                 0.418241      0.497967      0.563023      0.501008      0.0443355       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.86525       6.13186       7.34407       6.48963       1.05558        10     
reco:pandoraShower:LArPandoraShowerCreation         3.44797       6.30919       8.99022       6.25253       1.64684        10     
reco:pandoracalo:Calorimetry                        1.91916       2.93809       3.54963       3.07111      0.456414        10     
reco:pandoracalonosce:Calorimetry                   1.59087       2.62219       3.14801       2.67852      0.433302        10     
reco:pandorapid:Chi2ParticleID                    0.00567808    0.00669222     0.0114705    0.00622644    0.00162416       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0560493     0.0711055     0.0853207     0.0708764    0.00795528       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00526475    0.00589631    0.00646498    0.00588778    0.000331481      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.14687       6.36619       9.05647       6.39562       1.61975        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.0309        6.2253        8.7923        6.44223       1.55544        10     
reco:emtrkmichelid:EmTrackMichelId                  152.203       191.205       235.049       190.448       21.0323        10     
reco:crttag:SingleCRTMatchingProducer              0.012398      0.0186167     0.0232055     0.0192458    0.00285445       10     
reco:crtreco:TwoCRTMatchingProducer               0.00738601     0.0117648     0.0140838     0.0120438    0.00190417       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.3321        1.68995       2.11338       1.70487      0.223623        10     
reco:pandora2Track:LArPandoraTrackCreation          10.7485       20.6607       26.5898       21.5016       4.14124        10     
reco:pandora2calo:Calorimetry                       4.52105       8.84067       11.0957       8.83918       1.71564        10     
reco:pandora2calonosce:Calorimetry                  4.40681       8.39143       11.5665        8.325        1.79907        10     
reco:pandora2pid:Chi2ParticleID                    0.0133209     0.016205      0.0192022     0.0164322    0.00173071       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0733683     0.0955934     0.124643      0.0941761     0.0135321       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0130723     0.0158564     0.0186066     0.016116     0.00155607       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.93908       6.55591       9.18564       6.42344       1.32921        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.04314       7.67889       10.3241       7.58318       1.60677        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.79214       7.01355       9.58736       6.91591       1.43965        10     
[art]:TriggerResults:TriggerResultInserter        4.3278e-05    5.57731e-05   0.000118986   5.0189e-05    2.1237e-05       10     
end_path:out1:RootOutput                           7.441e-06    9.2656e-06    1.3816e-05    8.9235e-06    1.76155e-06      10     
end_path:out1:RootOutput(write)                     2.1906        2.90964       3.58766       2.88039      0.343009        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5234.11 MB
  Peak resident set size usage (VmHWM): 4422.89 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.42 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11293359
-rw-r--r-- 1 pltdune004 pltdune 2608051991 Dec 27 12:22 detsim_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 2472901184 Dec 27 12:19 detsim_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 2327291492 Dec 27 13:33 reco_stage1.root
-rw-r--r-- 1 pltdune004 pltdune 2080196354 Dec 27 10:37 g4_stage2.root
-rw-r--r-- 1 pltdune004 pltdune 1992204136 Dec 27 10:21 g4_stage1.root
-rw-r--r-- 1 pltdune004 pltdune   83188290 Dec 27 13:28 Pandora_Events.pndr
-rw-r--r-- 1 pltdune004 pltdune     242985 Dec 27 13:33 jobscript.log
-rw-r--r-- 1 pltdune004 pltdune     217088 Dec 27 13:33 mem.db
-rw-r--r-- 1 pltdune004 pltdune      62376 Dec 27 13:33 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune004 pltdune      36864 Dec 27 13:33 time.db
-rw-r--r-- 1 pltdune004 pltdune        519 Dec 27 12:22 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune        519 Dec 27 10:37 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune004 pltdune        519 Dec 27 13:33 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune004 pltdune         52 Dec 27 10:14 all-input-dids.txt
-rw-r--r-- 1 pltdune004 pltdune          7 Dec 27 13:33 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune004 pltdune          0 Dec 27 12:28 ProtoDUNE.log
-rw-r--r-- 1 pltdune004 pltdune          0 Dec 27 12:22 debugprod.log
justIN time: 2026-02-07 03:33:58 UTC       justIN version: 01.06.00