justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270367.0@dunegpschedd01.fnal.gov

Jobsub ID270367.0@dunegpschedd01.fnal.gov
Workflow ID11355
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-27 10:06:34
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-27 14:43:59
From worker nodeHostnameheplnc159.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-27 10:09:27
Input filesmonte-carlo-011355-000001
JobscriptExit code0
Real time4h (16380s)
CPU time4h (15702s = 95%)
Max RSS bytes4533526528 (4323 MiB)
Outputting started2025-12-27 14:42:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11355/1/001/reco_stage1.root
Finished2025-12-27 14:43:59
Saved logsjustin-logs:270367.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 101113 traces, 0 summary
Retagger: tagging trace set: gauss with 82272 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 82272 traces tagged "gauss"
FrameSaver: q=2.87216e+06 n=1499188 tag=gauss
wclsFrameSaver: saving 101113 traces tagged "wiener"
FrameSaver: q=3.01103e+06 n=1346064 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 19084 XUs and 26669 XVs -> 177 XUVs
C:0 T:1 31510 XUs and 37602 XVs -> 2462 XUVs
C:0 T:2 127695 XUs and 136119 XVs -> 5208 XUVs
C:0 T:3 61988 XUs and 53671 XVs -> 1008 XUVs
C:0 T:4 11971 XUs and 12266 XVs -> 82 XUVs
C:0 T:5 45533 XUs and 41453 XVs -> 4431 XUVs
C:0 T:6 49569 XUs and 43075 XVs -> 1679 XUVs
C:0 T:7 23026 XUs and 19255 XVs -> 199 XUVs
C:0 T:8 2843 XUs and 3152 XVs -> 22 XUVs
C:0 T:9 15189 XUs and 20928 XVs -> 873 XUVs
C:0 T:10 40597 XUs and 22774 XVs -> 3148 XUVs
C:0 T:11 230 XUs and 130 XVs -> 23 XUVs
19312 XUVs total
8615 collection wire objects
17744 potential space points
Neighbour search...
695530 tests to find 323798 neighbours
Iterating with no regularization...
Begin: 7.10748e+07
0 6.32123e+07
1 6.26885e+07
2 6.26223e+07
3 6.26072e+07
Now with regularization...
Begin: 5.33169e+07
0 5.33111e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411372080
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411372080
fLifetime: 30000 [us]
27-Dec-2025 14:42:25 GMT  Closed output file "reco_stage1.root"
27-Dec-2025 14:42:25 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          356.412       503.957       700.037       490.274       96.7659        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00116069    0.00447989    0.00827142    0.00403574    0.00225593       10     
reco:ophit:OpHitFinder                             0.285673      0.391199      0.526036      0.378713      0.0779677       10     
reco:opflash:OpFlashFinder                         0.191147      0.269772      0.347317      0.266535      0.0544161       10     
reco:opslicer:OpSlicer                             0.609667      0.858928       1.13769      0.867705      0.173603        10     
reco:caldata:DataPrepModule                         4.77067       6.55437       7.99506       6.62409       1.09023        10     
reco:wclsdatasp:WireCellToolkit                     36.3493       41.0042       44.9104       41.5323       2.66974        10     
reco:gaushit:GausHitFinder                          1.26183       1.77932       2.5183        1.78209      0.356158        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000534085   0.000772101   0.00102986    0.000741942   0.000166741      10     
reco:reco3d:SpacePointSolver                        15.5237       25.4965       36.4299       23.5052       6.71956        10     
reco:hitpdune:DisambigFromSpacePoints               1.32732       2.53883       3.64181       2.58661      0.655144        10     
reco:pandora:StandardPandora                         29.49        60.923        130.139       50.869        26.387         10     
reco:pandoraWriter:StandardPandora                  0.48723      0.692222       1.02794      0.661638      0.152685        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.72524        7.084        11.408        6.57521       2.17034        10     
reco:pandoraShower:LArPandoraShowerCreation         3.90914       7.72607       11.4357       7.50409       2.50143        10     
reco:pandoracalo:Calorimetry                        2.44393       3.70909       5.04195       3.50728      0.828382        10     
reco:pandoracalonosce:Calorimetry                   2.13847       3.41396       4.86514       3.3701       0.930118        10     
reco:pandorapid:Chi2ParticleID                    0.00463797    0.00825736     0.0120172    0.00887438    0.00251853       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0453581     0.070345      0.103543      0.067366      0.0206262       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00466685    0.00771272     0.0121561    0.00766353    0.00227425       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.79466       8.07946       11.8624       7.95691       2.65238        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.1962        7.43503       11.3456       7.65327       2.59967        10     
reco:emtrkmichelid:EmTrackMichelId                  196.177       254.006       322.449       258.333       36.8877        10     
reco:crttag:SingleCRTMatchingProducer              0.0129459     0.0201608     0.0271724     0.0212995    0.00502255       10     
reco:crtreco:TwoCRTMatchingProducer                0.0080698     0.0125967     0.0181232     0.0123959    0.00271781       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.640802       1.11263       1.37668       1.14999      0.208262        10     
reco:pandora2Track:LArPandoraTrackCreation          11.9118       22.5727       39.9289       21.0131       8.54031        10     
reco:pandora2calo:Calorimetry                       5.38376       10.9786       16.5016       11.0543       3.41529        10     
reco:pandora2calonosce:Calorimetry                  5.09569       10.537        16.7449       10.4055       3.34714        10     
reco:pandora2pid:Chi2ParticleID                    0.0126598     0.0205439     0.0266828     0.0201998    0.00416089       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.062772      0.0970615     0.125863      0.0976124     0.0195122       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0126845     0.0201061     0.0256048     0.0200825    0.00399754       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.1943        7.08595       11.0486       6.92533       2.12493        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.41125       8.23128       11.5627       8.41819       2.12962        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.19406       7.27815       10.6231       7.20721       2.00644        10     
[art]:TriggerResults:TriggerResultInserter        4.7208e-05    8.23037e-05   0.000223681   7.25365e-05   4.85167e-05      10     
end_path:out1:RootOutput                           9.538e-06    1.33822e-05    2.125e-05    1.31895e-05   3.2132e-06       10     
end_path:out1:RootOutput(write)                     2.86846       3.77722       5.30953       3.52077      0.681846        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5363.95 MB
  Peak resident set size usage (VmHWM): 4533.53 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11815380
-rw-r--r-- 1 pltdune04 pltdune 2717126794 Dec 27 13:16 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2575759630 Dec 27 13:13 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2452423725 Dec 27 14:42 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2177432686 Dec 27 10:35 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2088464018 Dec 27 10:17 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   87127060 Dec 27 14:38 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242962 Dec 27 14:42 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 27 14:42 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 27 14:42 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 27 14:42 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 13:16 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 10:35 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 27 14:42 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 27 10:09 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 27 14:42 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 27 13:22 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 27 13:16 debugprod.log
justIN time: 2026-02-07 03:33:51 UTC       justIN version: 01.06.00