justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270449.0@dunegpschedd01.fnal.gov

Jobsub ID270449.0@dunegpschedd01.fnal.gov
Workflow ID11364
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-27 22:55:13
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-28 03:30:55
From worker nodeHostnameheplnc159.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-27 22:56:18
Input filesmonte-carlo-011364-000001
JobscriptExit code0
Real time4h (16383s)
CPU time4h (15736s = 96%)
Max RSS bytes4450287616 (4244 MiB)
Outputting started2025-12-28 03:29:21
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11364/1/001/reco_stage1.root
Finished2025-12-28 03:30:55
Saved logsjustin-logs:270449.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

eKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 146464 traces, 0 summary
Retagger: tagging trace set: gauss with 112355 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 112355 traces tagged "gauss"
FrameSaver: q=4.02759e+06 n=1991008 tag=gauss
wclsFrameSaver: saving 146464 traces tagged "wiener"
FrameSaver: q=4.27118e+06 n=1772896 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46553 XUs and 43975 XVs -> 543 XUVs
C:0 T:1 98229 XUs and 92031 XVs -> 7066 XUVs
C:0 T:2 135385 XUs and 182671 XVs -> 7947 XUVs
C:0 T:3 62091 XUs and 64553 XVs -> 533 XUVs
C:0 T:4 13709 XUs and 17011 XVs -> 199 XUVs
C:0 T:5 65514 XUs and 59797 XVs -> 7128 XUVs
C:0 T:6 72163 XUs and 67747 XVs -> 3362 XUVs
C:0 T:7 30645 XUs and 25522 XVs -> 178 XUVs
C:0 T:8 9716 XUs and 10984 XVs -> 237 XUVs
C:0 T:9 57311 XUs and 84519 XVs -> 5132 XUVs
C:0 T:10 24761 XUs and 13006 XVs -> 3252 XUVs
C:0 T:11 213 XUs and 102 XVs -> 14 XUVs
35591 XUVs total
13289 collection wire objects
32648 potential space points
Neighbour search...
2396312 tests to find 1099000 neighbours
Iterating with no regularization...
Begin: 1.18555e+08
0 9.48955e+07
1 9.33807e+07
2 9.31987e+07
3 9.31523e+07
Now with regularization...
Begin: 7.70787e+07
0 7.69008e+07
1 7.68704e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411382734
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411382734
fLifetime: 30000 [us]
28-Dec-2025 03:29:20 GMT  Closed output file "reco_stage1.root"
28-Dec-2025 03:29:20 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          438.926       501.282       629.955       482.911       61.1203        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0010736     0.0048887    0.00907511    0.00492972    0.00246527       10     
reco:ophit:OpHitFinder                             0.299173      0.418663      0.520808      0.427552      0.076608        10     
reco:opflash:OpFlashFinder                         0.210351      0.273502      0.370088      0.282264      0.0482012       10     
reco:opslicer:OpSlicer                             0.608437       1.00692       1.35286       1.00462       0.23817        10     
reco:caldata:DataPrepModule                         4.90079       6.74904       7.93675       7.22936       1.1257         10     
reco:wclsdatasp:WireCellToolkit                     35.477        41.8779       45.4486       42.0846       2.6446         10     
reco:gaushit:GausHitFinder                          1.28188       1.83134       2.24657       1.87221      0.306343        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000506324   0.00103443    0.00278403    0.000828805   0.000651453      10     
reco:reco3d:SpacePointSolver                        15.0201       24.737        42.1027       23.8871       7.62176        10     
reco:hitpdune:DisambigFromSpacePoints               1.80773       2.45553       3.10244       2.5576       0.388925        10     
reco:pandora:StandardPandora                        38.0831       50.617        76.6654       48.7238       9.91816        10     
reco:pandoraWriter:StandardPandora                 0.430021       0.62236      0.786622      0.653713      0.121653        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.35656       6.9559        11.1779       6.8134        2.10357        10     
reco:pandoraShower:LArPandoraShowerCreation         4.99231       8.62738       14.6168       7.19234       2.78293        10     
reco:pandoracalo:Calorimetry                        2.84799       3.99724       6.5378        3.63943       1.02558        10     
reco:pandoracalonosce:Calorimetry                   2.04213       3.56355       5.96053       3.3259        1.02105        10     
reco:pandorapid:Chi2ParticleID                    0.00505347    0.00770789     0.0111648    0.00715147    0.00222266       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0466514     0.0767827     0.105834      0.0829431     0.0226099       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00522475    0.00768736     0.0106278    0.00814536    0.00180448       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.06186       8.4847         11.28        8.31621       1.47357        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.5267        8.16685       12.177        7.50581       2.1428         10     
reco:emtrkmichelid:EmTrackMichelId                  211.672       255.349       296.757       256.396       27.016         10     
reco:crttag:SingleCRTMatchingProducer              0.0160475     0.0206229     0.030293      0.0192467    0.00381033       10     
reco:crtreco:TwoCRTMatchingProducer                0.0119046     0.0139368     0.017065      0.0132123    0.00191373       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.889465       1.20067       1.52496       1.2351       0.187235        10     
reco:pandora2Track:LArPandoraTrackCreation          17.2145       23.1753       34.8339       20.6809       5.69037        10     
reco:pandora2calo:Calorimetry                       7.79112       11.1841       16.3066       10.4788       2.44245        10     
reco:pandora2calonosce:Calorimetry                  7.02213       10.4228       15.2905        9.838        2.39382        10     
reco:pandora2pid:Chi2ParticleID                    0.0133499     0.0228219      0.0317       0.0226418    0.00495523       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0648047     0.114456      0.143433      0.115771      0.0219307       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0132432     0.0223573     0.0300228     0.0225983    0.00437663       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.03965       7.42203       11.0413       7.2101        1.68851        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.97103       9.45994       13.5987       8.9502        2.01753        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.89165       8.23736       12.3859       7.6567        1.9737         10     
[art]:TriggerResults:TriggerResultInserter         4.812e-05    0.000102208   0.000313661   8.24855e-05   7.16829e-05      10     
end_path:out1:RootOutput                           1.058e-05    1.81171e-05   5.0345e-05    1.4913e-05    1.09746e-05      10     
end_path:out1:RootOutput(write)                     2.72196       3.97342       5.37981       3.82907       0.77364        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5290.77 MB
  Peak resident set size usage (VmHWM): 4450.29 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.24 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11748188
-rw-r--r-- 1 pltdune04 pltdune 2697047756 Dec 28 02:03 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2554597744 Dec 28 02:00 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2446667820 Dec 28 03:29 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2170730899 Dec 27 23:23 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2070876899 Dec 27 23:04 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   89605140 Dec 28 03:23 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243704 Dec 28 03:29 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 28 03:29 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 28 03:29 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 28 03:29 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 02:03 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 23:23 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 03:29 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 27 22:56 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 28 03:29 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 02:11 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 02:03 debugprod.log
justIN time: 2026-02-07 03:34:52 UTC       justIN version: 01.06.00