justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 266982.1@dunegpschedd01.fnal.gov

Jobsub ID266982.1@dunegpschedd01.fnal.gov
Workflow ID11162
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-11 20:10:40
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-12 15:49:21
From worker nodeHostnameheplnc152.pp.rl.ac.uk
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-12 11:33:24
Input filesmonte-carlo-011162-000001
JobscriptExit code0
Real time4h (15270s)
CPU time4h (14647s = 95%)
Max RSS bytes4380135424 (4177 MiB)
Outputting started2025-12-12 15:47:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11162/1/001/reco_stage1.root
Finished2025-12-12 15:49:21
Saved logsjustin-logs:266982.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

annels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148405 traces, 0 summary
Retagger: tagging trace set: gauss with 116808 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 116808 traces tagged "gauss"
FrameSaver: q=4.21711e+06 n=2146419 tag=gauss
wclsFrameSaver: saving 148405 traces tagged "wiener"
FrameSaver: q=4.4607e+06 n=1927094 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 53003 XUs and 39951 XVs -> 643 XUVs
C:0 T:1 103254 XUs and 84903 XVs -> 8525 XUVs
C:0 T:2 116506 XUs and 158411 XVs -> 10845 XUVs
C:0 T:3 52808 XUs and 62391 XVs -> 458 XUVs
C:0 T:4 18311 XUs and 19472 XVs -> 180 XUVs
C:0 T:5 77285 XUs and 60144 XVs -> 8135 XUVs
C:0 T:6 78233 XUs and 79170 XVs -> 2675 XUVs
C:0 T:7 34264 XUs and 35332 XVs -> 193 XUVs
C:0 T:8 8916 XUs and 5098 XVs -> 70 XUVs
C:0 T:9 70886 XUs and 53300 XVs -> 4949 XUVs
C:0 T:10 68917 XUs and 45036 XVs -> 10719 XUVs
C:0 T:11 354 XUs and 235 XVs -> 53 XUVs
47445 XUVs total
13307 collection wire objects
43515 potential space points
Neighbour search...
4600413 tests to find 2353412 neighbours
Iterating with no regularization...
Begin: 9.95935e+07
0 8.42567e+07
1 8.31754e+07
2 8.30135e+07
3 8.29618e+07
Now with regularization...
Begin: 6.89814e+07
0 6.89063e+07
1 6.8859e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411058630
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411058630
fLifetime: 30000 [us]
12-Dec-2025 15:47:52 GMT  Closed output file "reco_stage1.root"
12-Dec-2025 15:47:52 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          346.446       509.862       589.061       532.246       74.0001        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00130293    0.00381506    0.00651318    0.00388325    0.00156277       10     
reco:ophit:OpHitFinder                             0.282322      0.348247      0.426271      0.345862      0.0468984       10     
reco:opflash:OpFlashFinder                         0.196779      0.236779      0.285429      0.239073      0.0273286       10     
reco:opslicer:OpSlicer                             0.673654       0.88449       1.12061      0.844127      0.174534        10     
reco:caldata:DataPrepModule                         4.90872       6.02223       7.32281       6.05875      0.779769        10     
reco:wclsdatasp:WireCellToolkit                     35.0528       40.7327       45.6134       40.9236       3.23387        10     
reco:gaushit:GausHitFinder                         0.861834       1.45139       1.83698       1.53286      0.272838        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00053504    0.000778403   0.000929344   0.000785764   0.00012321       10     
reco:reco3d:SpacePointSolver                        9.48791       17.2028       26.3491       17.3337       4.51495        10     
reco:hitpdune:DisambigFromSpacePoints               1.48195       2.23889       3.3368        2.23861      0.551287        10     
reco:pandora:StandardPandora                        22.9372       53.8116       86.5901       58.5721       16.9395        10     
reco:pandoraWriter:StandardPandora                  0.47795      0.642528      0.795285      0.663466      0.0944893       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.2223        7.49912       10.0845       7.61222       1.58297        10     
reco:pandoraShower:LArPandoraShowerCreation         3.62005       8.38691       10.7892       8.81707       2.18407        10     
reco:pandoracalo:Calorimetry                        2.31156       3.6719        6.11215       3.39568       1.06009        10     
reco:pandoracalonosce:Calorimetry                   2.22321       3.72207       5.98245       3.64185       1.1386         10     
reco:pandorapid:Chi2ParticleID                    0.00541463    0.00771877     0.0109327    0.00740823    0.00187674       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0572803     0.0664724     0.084767      0.0626344     0.0091645       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00518645    0.00718807     0.0105503    0.00666207    0.00156095       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.48306       7.59857       10.4997       8.11275       2.4534         10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.35633       7.51116       10.3524       7.55444       2.30981        10     
reco:emtrkmichelid:EmTrackMichelId                  208.261       280.516       317.819       293.818       35.8296        10     
reco:crttag:SingleCRTMatchingProducer              0.0138667     0.0221262     0.0316262     0.021237     0.00568848       10     
reco:crtreco:TwoCRTMatchingProducer                0.011575      0.0157964     0.0211541     0.0153228    0.00285392       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.675217       1.07852       1.33218       1.13416      0.210924        10     
reco:pandora2Track:LArPandoraTrackCreation          11.0055       22.9501       30.2037       24.4792       5.83821        10     
reco:pandora2calo:Calorimetry                       4.28937       9.48058       12.4326       10.8157       2.52097        10     
reco:pandora2calonosce:Calorimetry                  4.14585       8.98573       11.8716       9.02761       2.2335         10     
reco:pandora2pid:Chi2ParticleID                    0.0115541     0.018338      0.0278681     0.0179913     0.0045807       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0538255     0.0855215     0.120525      0.0801231     0.0205987       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0110619     0.0186411     0.0258586     0.0189738    0.00413536       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.0652        7.14593       9.02363       7.2645        1.43712        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.85662       7.35582       10.1028       7.43079       1.32933        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.78073       6.45649       8.92159       6.81021       1.66384        10     
[art]:TriggerResults:TriggerResultInserter        5.3161e-05    7.5069e-05    0.000129355   6.12805e-05   2.73638e-05      10     
end_path:out1:RootOutput                           1.032e-05    7.15902e-05   0.00056651    1.1852e-05    0.000165462      10     
end_path:out1:RootOutput(write)                     2.44606       3.51588        4.248        3.46734      0.478387        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5214.46 MB
  Peak resident set size usage (VmHWM): 4380.14 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.76 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11347424
-rw-r--r-- 1 pltdune04 pltdune 2629317753 Dec 12 14:20 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2491780945 Dec 12 14:17 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2348821137 Dec 12 15:47 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2073515982 Dec 12 11:57 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1992225858 Dec 12 11:41 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   83481330 Dec 12 15:40 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243039 Dec 12 15:47 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 12 15:47 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 12 15:47 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 12 15:47 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 12 14:20 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 12 11:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 12 15:47 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 12 11:33 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 12 15:47 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 12 14:27 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 12 14:20 debugprod.log
justIN time: 2026-02-07 13:29:32 UTC       justIN version: 01.06.00