justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263477.0@dunegpschedd02.fnal.gov

Jobsub ID263477.0@dunegpschedd02.fnal.gov
Workflow ID11287
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-19 20:09:18
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-19 23:58:02
From worker nodeHostnameheplnc167.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-19 20:09:39
Input filesmonte-carlo-011287-000001
JobscriptExit code0
Real time3h (13613s)
CPU time3h (13141s = 96%)
Max RSS bytes4451438592 (4245 MiB)
Outputting started2025-12-19 23:56:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11287/1/001/reco_stage1.root
Finished2025-12-19 23:58:02
Saved logsjustin-logs:263477.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

emoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 154219 traces, 0 summary
Retagger: tagging trace set: gauss with 120316 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 120316 traces tagged "gauss"
FrameSaver: q=4.3127e+06 n=2168292 tag=gauss
wclsFrameSaver: saving 154219 traces tagged "wiener"
FrameSaver: q=4.55454e+06 n=1931033 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 46088 XUs and 29507 XVs -> 560 XUVs
C:0 T:1 92777 XUs and 68547 XVs -> 6336 XUVs
C:0 T:2 119574 XUs and 133561 XVs -> 10468 XUVs
C:0 T:3 58727 XUs and 59569 XVs -> 2156 XUVs
C:0 T:4 14503 XUs and 14892 XVs -> 131 XUVs
C:0 T:5 54150 XUs and 42815 XVs -> 3891 XUVs
C:0 T:6 146364 XUs and 107728 XVs -> 6848 XUVs
C:0 T:7 52586 XUs and 38546 XVs -> 494 XUVs
C:0 T:8 11983 XUs and 8846 XVs -> 122 XUVs
C:0 T:9 68636 XUs and 74052 XVs -> 4616 XUVs
C:0 T:10 53445 XUs and 36288 XVs -> 5547 XUVs
C:0 T:11 258 XUs and 166 XVs -> 43 XUVs
41212 XUVs total
13567 collection wire objects
35323 potential space points
Neighbour search...
2169401 tests to find 1154782 neighbours
Iterating with no regularization...
Begin: 9.05509e+07
0 7.46972e+07
1 7.32589e+07
2 7.30551e+07
3 7.30003e+07
Now with regularization...
Begin: 6.02989e+07
0 6.02519e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411219528
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411219528
fLifetime: 30000 [us]
19-Dec-2025 23:56:30 GMT  Closed output file "reco_stage1.root"
19-Dec-2025 23:56:30 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          337.905       429.662       539.909       433.528       56.585         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00086528     0.0042728    0.00824299    0.00442579    0.00207539       10     
reco:ophit:OpHitFinder                             0.282774       0.38309       0.46565      0.400383      0.0648696       10     
reco:opflash:OpFlashFinder                         0.210182      0.262355      0.321615      0.255135      0.0344188       10     
reco:opslicer:OpSlicer                             0.580796      0.937913       1.26303      0.965822       0.2011         10     
reco:caldata:DataPrepModule                         4.60501       6.65041       7.72437       6.74013      0.877928        10     
reco:wclsdatasp:WireCellToolkit                     34.3196       37.5706       41.7014       37.356        2.62585        10     
reco:gaushit:GausHitFinder                          1.22542       1.7049        2.49466       1.75808      0.379137        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000607327   0.000764126   0.000954046   0.000762523   0.000113249      10     
reco:reco3d:SpacePointSolver                        14.3336       22.6674       40.4201       20.8754       7.08118        10     
reco:hitpdune:DisambigFromSpacePoints               1.1065        2.3952        3.14623       2.61715      0.618459        10     
reco:pandora:StandardPandora                        20.9508       40.7448       61.4632       40.4455       11.7638        10     
reco:pandoraWriter:StandardPandora                 0.341807       0.45539      0.760238      0.414475      0.111965        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.34056       4.91855       6.88959       4.89043       1.19729        10     
reco:pandoraShower:LArPandoraShowerCreation         2.9939        5.47805       8.85892       5.17555       1.70522        10     
reco:pandoracalo:Calorimetry                        1.68741       2.84337       4.5338        2.74238      0.796993        10     
reco:pandoracalonosce:Calorimetry                   1.61112       2.65157       4.13046       2.4843       0.698432        10     
reco:pandorapid:Chi2ParticleID                    0.00457805     0.0064057     0.0106266    0.00518566    0.00209705       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0404442     0.059595      0.103748      0.0481463     0.0214742       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00453502    0.00610405    0.00868622     0.0052596    0.00158725       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.52594       5.86919       9.33944       5.6789        1.52873        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.57348       5.9257        8.04816       5.58625       1.46791        10     
reco:emtrkmichelid:EmTrackMichelId                  185.573       226.323       267.489       229.861       24.3464        10     
reco:crttag:SingleCRTMatchingProducer              0.0148773     0.0197166     0.0301845     0.0192847    0.00397503       10     
reco:crtreco:TwoCRTMatchingProducer               0.00914738     0.0119917     0.0142434     0.0120943    0.00152201       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.841998       1.12788       1.46233       1.12719      0.170446        10     
reco:pandora2Track:LArPandoraTrackCreation          10.4761       18.2105       24.019        18.8684       3.7774         10     
reco:pandora2calo:Calorimetry                       5.16884       8.33099       10.7039       8.45486       1.40968        10     
reco:pandora2calonosce:Calorimetry                  5.32907       8.81637       13.7399       9.06722       2.11125        10     
reco:pandora2pid:Chi2ParticleID                    0.0103011     0.0175076     0.0244728     0.0174566    0.00588656       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.052033      0.0871791     0.133974      0.0825667     0.0300418       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0105307     0.0171734     0.0242045     0.016981     0.00564793       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.66123       6.22219       10.6448       6.02102       1.92603        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.66283       8.10104       11.5759       7.6609        1.60639        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.7428        6.96205       11.4571       6.3541        1.77469        10     
[art]:TriggerResults:TriggerResultInserter        4.9352e-05    8.95285e-05   0.00020826    7.8302e-05    4.45884e-05      10     
end_path:out1:RootOutput                          1.0089e-05    1.8191e-05    5.2438e-05    1.38605e-05   1.2017e-05       10     
end_path:out1:RootOutput(write)                     2.6872        3.73735       5.04277       3.67787      0.761792        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5318.11 MB
  Peak resident set size usage (VmHWM): 4451.44 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.31 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11639920
-rw-r--r-- 1 pltdune04 pltdune 2670427688 Dec 19 22:42 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2531796354 Dec 19 22:40 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2401110357 Dec 19 23:56 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2154841376 Dec 19 20:31 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2075220926 Dec 19 20:17 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   85270150 Dec 19 23:50 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     243659 Dec 19 23:56 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 19 23:56 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 19 23:56 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 19 23:56 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 19 22:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 19 20:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 19 23:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 19 20:09 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 19 23:56 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 19 22:49 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 19 22:42 debugprod.log
justIN time: 2026-02-07 06:16:36 UTC       justIN version: 01.06.00