justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295945.1@dunegpschedd02.fnal.gov

Jobsub ID295945.1@dunegpschedd02.fnal.gov
Workflow ID12443
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-27 14:23:30
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat2026-01-27 18:11:04
From worker nodeHostnameheplnc166.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-27 14:24:50
Input filesmonte-carlo-012443-000001
JobscriptExit code0
Real time3h (13274s)
CPU time3h (12596s = 94%)
Max RSS bytes4025217024 (3838 MiB)
Outputting started2026-01-27 18:06:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12443/1/001/reco_stage1.root
Finished2026-01-27 18:11:04
Saved logsjustin-logs:295945.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

caleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 132236 traces, 0 summary
Retagger: tagging trace set: gauss with 101729 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101729 traces tagged "gauss"
FrameSaver: q=3.65588e+06 n=1837653 tag=gauss
wclsFrameSaver: saving 132236 traces tagged "wiener"
FrameSaver: q=3.86892e+06 n=1634997 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38743 XUs and 30773 XVs -> 641 XUVs
C:0 T:1 73921 XUs and 63886 XVs -> 5020 XUVs
C:0 T:2 159462 XUs and 199526 XVs -> 10134 XUVs
C:0 T:3 64850 XUs and 75921 XVs -> 666 XUVs
C:0 T:4 10722 XUs and 10580 XVs -> 109 XUVs
C:0 T:5 54141 XUs and 38977 XVs -> 3208 XUVs
C:0 T:6 122728 XUs and 148723 XVs -> 6471 XUVs
C:0 T:7 43857 XUs and 47142 XVs -> 647 XUVs
C:0 T:8 7343 XUs and 6225 XVs -> 56 XUVs
C:0 T:9 36244 XUs and 41738 XVs -> 2568 XUVs
C:0 T:10 24770 XUs and 14040 XVs -> 2144 XUVs
C:0 T:11 516 XUs and 275 XVs -> 40 XUVs
31704 XUVs total
12718 collection wire objects
27985 potential space points
Neighbour search...
1930835 tests to find 745836 neighbours
Iterating with no regularization...
Begin: 8.92027e+07
0 7.2686e+07
1 7.12994e+07
2 7.1097e+07
3 7.10346e+07
Now with regularization...
Begin: 5.90969e+07
0 5.90162e+07
1 5.89797e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411999300
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411999300
fLifetime: 30000 [us]
27-Jan-2026 18:06:03 GMT  Closed output file "reco_stage1.root"
27-Jan-2026 18:06:03 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          330.15        425.922       545.764       396.541       65.8294        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00120945    0.00406933    0.00771045    0.00383835    0.00172156       10     
reco:ophit:OpHitFinder                             0.247276      0.344793      0.482475      0.359424      0.0725139       10     
reco:opflash:OpFlashFinder                         0.139032      0.185697      0.274828      0.180629       0.03637        10     
reco:opslicer:OpSlicer                             0.569389      0.847151       1.20234      0.852605       0.22832        10     
reco:caldata:DataPrepModule                         5.66468       7.38114       8.3841        7.86321      0.912741        10     
reco:wclsdatasp:WireCellToolkit                     33.1452       41.5365       45.7782       42.6681       3.57453        10     
reco:gaushit:GausHitFinder                         0.989048       1.60878       2.30717       1.60998      0.406996        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000509934   0.000652932   0.000824423   0.00062259    0.000104959      10     
reco:reco3d:SpacePointSolver                        11.0722       19.2384       27.2351       17.4428       5.54352        10     
reco:hitpdune:DisambigFromSpacePoints               1.22217       2.09473       3.3236        1.98478      0.613556        10     
reco:pandora:StandardPandora                        29.1385       39.7317       58.7457       33.9507       10.5586        10     
reco:pandoraWriter:StandardPandora                 0.361576      0.613391      0.789701      0.625192      0.131502        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.5104        5.12799       6.87786       5.1204        1.06931        10     
reco:pandoraShower:LArPandoraShowerCreation         2.92113       5.44474       7.81449       5.41041       1.54472        10     
reco:pandoracalo:Calorimetry                        1.5316        3.06183       4.26675       3.06299      0.675914        10     
reco:pandoracalonosce:Calorimetry                   1.59148       2.79962       4.01827       2.66406      0.630561        10     
reco:pandorapid:Chi2ParticleID                    0.00542985    0.00831893     0.0108764    0.00822757    0.00133661       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0470402     0.0740754     0.101152      0.0732084     0.0148652       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00464532    0.00734642     0.010667     0.00753576    0.00185785       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.24018       5.64066       8.87408       4.92373       1.73206        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.02547       5.77072       8.58859       4.84146       1.66748        10     
reco:emtrkmichelid:EmTrackMichelId                   190.2        228.312         285         214.45        31.6058        10     
reco:crttag:SingleCRTMatchingProducer              0.0120406     0.0228278     0.0441939     0.020007     0.00868472       10     
reco:crtreco:TwoCRTMatchingProducer               0.00661506     0.0131027     0.0185976     0.0123688    0.00354123       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.721799       1.05868       1.53508       1.04297      0.235524        10     
reco:pandora2Track:LArPandoraTrackCreation          8.62593       16.1604       23.4288       16.0416       4.54225        10     
reco:pandora2calo:Calorimetry                       4.44573       8.0425        10.5319       7.85886       1.87341        10     
reco:pandora2calonosce:Calorimetry                  3.27523       7.84859       10.903        8.23237       2.17582        10     
reco:pandora2pid:Chi2ParticleID                   0.00933981     0.0178287     0.0263604     0.0181872    0.00555754       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0466114     0.0960086     0.126546      0.100777      0.0252182       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0094237     0.0181775     0.0249057     0.0181844    0.00481382       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.77005       5.69453       8.23948       5.67686       1.64289        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.02033       7.10676       9.43649       7.39151       1.96061        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.74303       6.23837       8.73716       6.05938       1.82691        10     
[art]:TriggerResults:TriggerResultInserter        4.7599e-05    8.73912e-05   0.000139892   8.8675e-05    2.71242e-05      10     
end_path:out1:RootOutput                           8.256e-06    1.61515e-05   5.7478e-05    1.19675e-05   1.39115e-05      10     
end_path:out1:RootOutput(write)                     2.25544       3.61691       4.64897       3.73912      0.723719        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4906.44 MB
  Peak resident set size usage (VmHWM): 4025.22 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.65 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9895208
-rw-r--r-- 1 pltdune04 pltdune 2334356321 Jan 27 16:52 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2219376710 Jan 27 16:50 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2012323124 Jan 27 18:06 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1785158969 Jan 27 14:46 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1707895599 Jan 27 14:34 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   72967310 Jan 27 18:00 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242823 Jan 27 18:06 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Jan 27 18:06 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Jan 27 18:06 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Jan 27 18:06 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 27 16:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 27 14:46 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Jan 27 18:06 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Jan 27 14:24 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Jan 27 18:06 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 27 16:59 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Jan 27 16:52 debugprod.log
justIN time: 2026-02-04 18:07:55 UTC       justIN version: 01.06.00