justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274016.0@dunegpschedd02.fnal.gov

Jobsub ID274016.0@dunegpschedd02.fnal.gov
Workflow ID12000
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-17 15:54:35
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-17 20:38:01
From worker nodeHostnameheplnc170.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-17 15:57:07
Input filesmonte-carlo-012000-000001
JobscriptExit code0
Real time4h (16764s)
CPU time4h (16029s = 95%)
Max RSS bytes4476391424 (4269 MiB)
Outputting started2026-01-17 20:36:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12000/1/001/reco_stage1.root
Finished2026-01-17 20:38:01
Saved logsjustin-logs:274016.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

rvice:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144056 traces, 0 summary
Retagger: tagging trace set: gauss with 111061 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111061 traces tagged "gauss"
FrameSaver: q=3.86962e+06 n=2047095 tag=gauss
wclsFrameSaver: saving 144056 traces tagged "wiener"
FrameSaver: q=4.08599e+06 n=1826743 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 40153 XUs and 31462 XVs -> 329 XUVs
C:0 T:1 76849 XUs and 55668 XVs -> 4781 XUVs
C:0 T:2 204273 XUs and 193050 XVs -> 17197 XUVs
C:0 T:3 70652 XUs and 66273 XVs -> 983 XUVs
C:0 T:4 15867 XUs and 10515 XVs -> 133 XUVs
C:0 T:5 87344 XUs and 42611 XVs -> 8237 XUVs
C:0 T:6 116082 XUs and 116872 XVs -> 6676 XUVs
C:0 T:7 47820 XUs and 43756 XVs -> 479 XUVs
C:0 T:8 9920 XUs and 7270 XVs -> 59 XUVs
C:0 T:9 79881 XUs and 65277 XVs -> 7807 XUVs
C:0 T:10 40778 XUs and 34176 XVs -> 6180 XUVs
C:0 T:11 826 XUs and 557 XVs -> 100 XUVs
52961 XUVs total
14287 collection wire objects
48158 potential space points
Neighbour search...
6458122 tests to find 3176844 neighbours
Iterating with no regularization...
Begin: 7.12729e+07
0 5.67664e+07
1 5.57471e+07
2 5.5577e+07
3 5.55164e+07
4 5.54872e+07
Now with regularization...
Begin: 4.46517e+07
0 4.45813e+07
1 4.45329e+07
2 4.44984e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411799397
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411799397
fLifetime: 30000 [us]
17-Jan-2026 20:36:30 GMT  Closed output file "reco_stage1.root"
17-Jan-2026 20:36:30 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          399.332       555.046       690.818       560.607       81.9905        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00130872     0.0055349     0.0095203    0.00578935    0.00264146       10     
reco:ophit:OpHitFinder                             0.372346      0.449338      0.491022      0.459173      0.0365131       10     
reco:opflash:OpFlashFinder                          0.17324      0.256404      0.296201      0.272469      0.0360949       10     
reco:opslicer:OpSlicer                             0.840739       1.10338       1.32251       1.13343      0.131122        10     
reco:caldata:DataPrepModule                         7.40338       8.20834       8.76735       8.27455      0.423464        10     
reco:wclsdatasp:WireCellToolkit                     44.3298       45.9563       46.9899       46.023       0.814959        10     
reco:gaushit:GausHitFinder                          1.64285       1.96461       2.29203       1.98455      0.162524        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00059398    0.000808443   0.00114429    0.000814672   0.000145549      10     
reco:reco3d:SpacePointSolver                        18.7913       28.738        44.562        28.4582       7.93708        10     
reco:hitpdune:DisambigFromSpacePoints               2.00461       2.96197       4.08296       2.94117       0.61105        10     
reco:pandora:StandardPandora                        42.2904       66.195        94.1326       60.9697       19.349         10     
reco:pandoraWriter:StandardPandora                 0.455771       0.73459      0.897854      0.759247      0.123316        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.59273       7.02524       9.7709        6.96112       1.55624        10     
reco:pandoraShower:LArPandoraShowerCreation         4.48425       8.01335        12.11        7.65093       2.0862         10     
reco:pandoracalo:Calorimetry                        2.38756       4.11099       4.84901       4.23751      0.725793        10     
reco:pandoracalonosce:Calorimetry                   2.08732       3.73869       4.60039       3.7567       0.711277        10     
reco:pandorapid:Chi2ParticleID                    0.00763979     0.0102179     0.0119107     0.0104636    0.00107438       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0782115     0.0953264      0.10861      0.0969772    0.00946768       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00713195    0.00954913     0.0103955    0.00990629    0.000886804      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.05025       8.75455       11.911        8.85608       2.02216        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.70944       8.22796       11.4685       8.14718       2.14771        10     
reco:emtrkmichelid:EmTrackMichelId                  210.478       277.083       335.548       280.467       33.1003        10     
reco:crttag:SingleCRTMatchingProducer              0.0172086     0.0219117     0.0286537     0.0215072    0.00365551       10     
reco:crtreco:TwoCRTMatchingProducer               0.00974698     0.0140094     0.0174427     0.0141554     0.0019068       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.0537        1.23665       1.43254       1.26772      0.136366        10     
reco:pandora2Track:LArPandoraTrackCreation          11.1583       24.3593       37.1826       23.8165       6.66568        10     
reco:pandora2calo:Calorimetry                       6.33609       12.1267       16.3934       12.2666       2.93551        10     
reco:pandora2calonosce:Calorimetry                  6.48426       11.4024       14.6735       11.0771       2.45092        10     
reco:pandora2pid:Chi2ParticleID                    0.0204584     0.0248473     0.0299336     0.024586     0.00294766       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0994009     0.120063      0.142525      0.122011      0.0134202       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0199255     0.0239813     0.0294014     0.0232019    0.00299793       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.04561       8.48327       10.9024       8.39773       1.82126        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.0126        10.0498       12.9462       9.81248       2.20402        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.76899       8.86323       11.6986       8.35155       1.77183        10     
[art]:TriggerResults:TriggerResultInserter        6.0713e-05    0.000102413   0.000232605   9.1501e-05    4.48438e-05      10     
end_path:out1:RootOutput                           9.607e-06    1.45892e-05   2.3313e-05    1.4197e-05    3.24876e-06      10     
end_path:out1:RootOutput(write)                     3.61866       4.50334       5.3007        4.54471      0.440782        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5368.26 MB
  Peak resident set size usage (VmHWM): 4476.39 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11153216
-rw-r--r-- 1 pltdune04 pltdune 2590377447 Jan 17 19:01 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2461358416 Jan 17 18:57 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2317150785 Jan 17 20:36 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2019525107 Jan 17 16:21 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1947046249 Jan 17 16:05 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   84752380 Jan 17 20:29 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243244 Jan 17 20:36 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 17 20:36 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 17 20:36 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 17 20:36 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 19:01 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 16:21 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 17 20:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 17 15:57 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 17 20:36 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:10 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 17 19:01 debugprod.log
justIN time: 2026-02-05 22:02:09 UTC       justIN version: 01.06.00