justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255663.0@dunegpschedd01.fnal.gov

Jobsub ID255663.0@dunegpschedd01.fnal.gov
Workflow ID10535
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-23 12:38:51
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2025-11-23 17:09:48
From worker nodeHostnameheplnc145.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-23 12:39:43
Input filesmonte-carlo-010535-000001
JobscriptExit code0
Real time4h (15584s)
CPU time4h (14873s = 95%)
Max RSS bytes4528345088 (4318 MiB)
Outputting started2025-11-23 16:59:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10535/1/001/reco_stage1.root
Finished2025-11-23 17:09:48
Saved logsjustin-logs:255663.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 145159 traces, 0 summary
Retagger: tagging trace set: gauss with 110726 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110726 traces tagged "gauss"
FrameSaver: q=4.02296e+06 n=1947375 tag=gauss
wclsFrameSaver: saving 145159 traces tagged "wiener"
FrameSaver: q=4.25787e+06 n=1720969 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42108 XUs and 45239 XVs -> 518 XUVs
C:0 T:1 88286 XUs and 99117 XVs -> 7328 XUVs
C:0 T:2 83994 XUs and 73197 XVs -> 5563 XUVs
C:0 T:3 34714 XUs and 30894 XVs -> 236 XUVs
C:0 T:4 12439 XUs and 12903 XVs -> 110 XUVs
C:0 T:5 43035 XUs and 35219 XVs -> 2955 XUVs
C:0 T:6 183401 XUs and 275322 XVs -> 11586 XUVs
C:0 T:7 59364 XUs and 83941 XVs -> 775 XUVs
C:0 T:8 2880 XUs and 3834 XVs -> 8 XUVs
C:0 T:9 22132 XUs and 50709 XVs -> 3832 XUVs
C:0 T:10 40969 XUs and 25376 XVs -> 5670 XUVs
C:0 T:11 104 XUs and 65 XVs -> 7 XUVs
38588 XUVs total
12800 collection wire objects
34584 potential space points
Neighbour search...
2503534 tests to find 1239436 neighbours
Iterating with no regularization...
Begin: 1.09049e+08
0 8.80175e+07
1 8.63637e+07
2 8.6117e+07
3 8.604e+07
Now with regularization...
Begin: 7.17864e+07
0 7.17025e+07
1 7.16669e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410690294
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410690294
fLifetime: 30000 [us]
23-Nov-2025 16:59:25 GMT  Closed output file "reco_stage1.root"
23-Nov-2025 16:59:25 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          373.68        498.829       666.026       462.12        98.6947        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00112843    0.00399326    0.00807303    0.00343627    0.00199892       10     
reco:ophit:OpHitFinder                             0.265768       0.35217      0.425411      0.351162      0.0483418       10     
reco:opflash:OpFlashFinder                         0.150147      0.187815      0.243656      0.180822      0.0317904       10     
reco:opslicer:OpSlicer                             0.625631      0.939629       1.24169      0.864171      0.206711        10     
reco:caldata:DataPrepModule                         4.72041       6.50975       7.77018       6.8115       0.946361        10     
reco:wclsdatasp:WireCellToolkit                     30.7217       43.3467       48.4465       44.768        5.01285        10     
reco:gaushit:GausHitFinder                          1.13042       1.49944       1.92837       1.48394      0.257396        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000504763   0.000745903   0.00104302    0.000724287   0.00017609       10     
reco:reco3d:SpacePointSolver                        10.6964       18.5056       34.4439       14.8271       7.89211        10     
reco:hitpdune:DisambigFromSpacePoints              0.978863       2.03573       3.46669       1.73614      0.771813        10     
reco:pandora:StandardPandora                        16.9707       45.3287       70.4753       45.3137       18.1226        10     
reco:pandoraWriter:StandardPandora                 0.358605      0.620329      0.890243       0.61862      0.160318        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.10361       6.3622        9.89715       5.87757       2.20196        10     
reco:pandoraShower:LArPandoraShowerCreation         3.27495       7.17546       12.7712       5.76889       3.39807        10     
reco:pandoracalo:Calorimetry                        1.62589       3.02277       4.72919       2.74283      0.979386        10     
reco:pandoracalonosce:Calorimetry                   1.36941       2.79736       4.38996       2.98376      0.923583        10     
reco:pandorapid:Chi2ParticleID                    0.00417899     0.0066759    0.00937442    0.00663956    0.00179526       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0405126     0.0662872     0.100959      0.0635475     0.0193636       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00399611    0.00622279    0.00925085    0.00589865    0.00158736       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.3769        6.88279       12.3763       5.69569       3.52441        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.29949       6.42395       11.718        5.27273       3.15927        10     
reco:emtrkmichelid:EmTrackMichelId                  237.916       281.257       350.978       275.433        38.8          10     
reco:crttag:SingleCRTMatchingProducer              0.0143881     0.0207499     0.0349572     0.0174136    0.00682845       10     
reco:crtreco:TwoCRTMatchingProducer                0.009885      0.0142788     0.0238727     0.0133589    0.00395652       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.867847       1.09323       1.65521       1.05287      0.211864        10     
reco:pandora2Track:LArPandoraTrackCreation          11.7739       19.7361       37.7541       17.6438       7.84984        10     
reco:pandora2calo:Calorimetry                       5.08475       9.37647       16.3914       8.52978       3.45702        10     
reco:pandora2calonosce:Calorimetry                  4.49955       8.92221       16.0184       7.89908       3.85773        10     
reco:pandora2pid:Chi2ParticleID                    0.0127878     0.0202283     0.029578      0.0179963    0.00533407       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0577196     0.103296      0.139014      0.0980021     0.0240262       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.012834      0.0204038     0.0286409     0.019621     0.00501258       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.82278       7.35786       13.4257       6.10231       3.36619        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.06008       8.04928       14.363        6.5151        3.56304        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.5562        6.69435       12.3502       5.62027       3.01356        10     
[art]:TriggerResults:TriggerResultInserter        4.8402e-05    8.13485e-05   0.000188736    7.135e-05    3.90493e-05      10     
end_path:out1:RootOutput                           9.799e-06    1.21518e-05   1.8715e-05    1.1451e-05    2.37904e-06      10     
end_path:out1:RootOutput(write)                     2.65152       3.92113       5.54539       3.64348      0.922598        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5347.91 MB
  Peak resident set size usage (VmHWM): 4528.35 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10710096
-rw-r--r-- 1 pltdune04 pltdune 2495684765 Nov 23 15:33 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2374522443 Nov 23 15:31 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2196666779 Nov 23 16:59 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 1946428728 Nov 23 13:05 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1875396909 Nov 23 12:47 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   77806980 Nov 23 16:52 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242747 Nov 23 16:59 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Nov 23 16:59 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Nov 23 16:59 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Nov 23 16:59 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Nov 23 15:33 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Nov 23 13:05 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Nov 23 16:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Nov 23 12:39 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Nov 23 16:59 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Nov 23 15:41 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Nov 23 15:33 debugprod.log
justIN time: 2026-02-08 21:30:08 UTC       justIN version: 01.06.00