justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270480.1@dunegpschedd01.fnal.gov

Jobsub ID270480.1@dunegpschedd01.fnal.gov
Workflow ID11374
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-28 08:47:46
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-12-28 13:09:29
From worker nodeHostnameheplnc159.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-28 08:48:25
Input filesmonte-carlo-011374-000001
JobscriptExit code0
Real time4h (15577s)
CPU time4h (14977s = 96%)
Max RSS bytes4371165184 (4168 MiB)
Outputting started2025-12-28 13:08:03
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11374/1/001/reco_stage1.root
Finished2025-12-28 13:09:29
Saved logsjustin-logs:270480.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

rvice:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144637 traces, 0 summary
Retagger: tagging trace set: gauss with 112260 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 112260 traces tagged "gauss"
FrameSaver: q=3.9099e+06 n=1967565 tag=gauss
wclsFrameSaver: saving 144637 traces tagged "wiener"
FrameSaver: q=4.1242e+06 n=1748750 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37510 XUs and 31161 XVs -> 291 XUVs
C:0 T:1 68079 XUs and 56246 XVs -> 4482 XUVs
C:0 T:2 80189 XUs and 118039 XVs -> 3771 XUVs
C:0 T:3 39595 XUs and 56508 XVs -> 329 XUVs
C:0 T:4 19229 XUs and 16321 XVs -> 178 XUVs
C:0 T:5 70025 XUs and 49476 XVs -> 4389 XUVs
C:0 T:6 71143 XUs and 76820 XVs -> 2078 XUVs
C:0 T:7 33488 XUs and 33559 XVs -> 140 XUVs
C:0 T:8 5984 XUs and 6795 XVs -> 81 XUVs
C:0 T:9 30199 XUs and 45286 XVs -> 4113 XUVs
C:0 T:10 62205 XUs and 40683 XVs -> 6203 XUVs
C:0 T:11 105 XUs and 73 XVs -> 9 XUVs
26064 XUVs total
11903 collection wire objects
24441 potential space points
Neighbour search...
883053 tests to find 483540 neighbours
Iterating with no regularization...
Begin: 8.98675e+07
0 8.01047e+07
1 7.94464e+07
2 7.93712e+07
Now with regularization...
Begin: 6.79033e+07
0 6.78832e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411391115
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411391115
fLifetime: 30000 [us]
28-Dec-2025 13:08:01 GMT  Closed output file "reco_stage1.root"
28-Dec-2025 13:08:01 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          374.381       473.493       606.652       452.187       74.2905        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00117252    0.00445002    0.00620534    0.00549665    0.00176269       10     
reco:ophit:OpHitFinder                             0.296159      0.405716      0.532973      0.413276      0.0684783       10     
reco:opflash:OpFlashFinder                         0.226813      0.263024      0.305293      0.271042      0.0259485       10     
reco:opslicer:OpSlicer                             0.762732      0.998792       1.30898      0.989938      0.143663        10     
reco:caldata:DataPrepModule                         5.32743       7.04968       7.87967       7.47061      0.854628        10     
reco:wclsdatasp:WireCellToolkit                     39.258        41.6461       43.7525       41.8283       1.60311        10     
reco:gaushit:GausHitFinder                          1.16762       1.71164       2.39408       1.72632      0.343721        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000497557   0.000759583   0.00094532    0.000742458   0.000121879      10     
reco:reco3d:SpacePointSolver                        11.8697       26.3533       38.7286       25.0892       7.75905        10     
reco:hitpdune:DisambigFromSpacePoints               1.67492       2.60826       3.63263       2.55006      0.582407        10     
reco:pandora:StandardPandora                        34.2864       49.2996       84.9677       44.3658       15.1236        10     
reco:pandoraWriter:StandardPandora                 0.390435      0.633417      0.817388      0.643472      0.119975        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.42813       6.87765       10.7046       6.19688       1.58274        10     
reco:pandoraShower:LArPandoraShowerCreation         4.01975        7.081        12.2543       6.35705       2.44668        10     
reco:pandoracalo:Calorimetry                        2.80114       3.84047       5.9969        3.57866       0.86867        10     
reco:pandoracalonosce:Calorimetry                   2.73103       3.5968        5.53836       3.33788      0.787119        10     
reco:pandorapid:Chi2ParticleID                    0.00484336    0.00905621     0.0127543    0.00904855    0.00190278       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0468575     0.0850379     0.101032      0.0866479     0.0156583       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00486844    0.00882744     0.0109885    0.00911607    0.00154972       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.02481       7.35504       11.9391       7.32924       2.43578        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.1886        7.19568       12.0943       6.86619       2.5692         10     
reco:emtrkmichelid:EmTrackMichelId                  202.988       243.21        313.557       233.899       31.2225        10     
reco:crttag:SingleCRTMatchingProducer              0.0158577     0.0217351     0.0291097     0.0211909    0.00405252       10     
reco:crtreco:TwoCRTMatchingProducer                0.0100775     0.014542      0.0192428     0.0139957    0.00338082       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.873072       1.21102       1.44222       1.26071       0.15829        10     
reco:pandora2Track:LArPandoraTrackCreation          11.2558       19.2784       30.5607       17.7717       5.99003        10     
reco:pandora2calo:Calorimetry                       5.28929       9.50189       14.9219       8.40171       2.90313        10     
reco:pandora2calonosce:Calorimetry                   4.996        8.82353       13.7947       8.36642       2.62413        10     
reco:pandora2pid:Chi2ParticleID                    0.0104938     0.0175548     0.0261126     0.0171641    0.00526951       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0517903     0.0856932     0.128744      0.079098      0.0262825       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0103746     0.0174958     0.0250817     0.0177767     0.0055103       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.53863       6.08515       9.45809       5.63671       1.88571        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.37129       7.56323       11.4076       7.27273       2.11967        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.89693       6.97283       10.2587       6.7258        1.80498        10     
[art]:TriggerResults:TriggerResultInserter        6.8979e-05    8.72152e-05   0.000149782   8.1684e-05    2.31717e-05      10     
end_path:out1:RootOutput                          1.0239e-05    1.24173e-05   1.6391e-05    1.1687e-05    2.08995e-06      10     
end_path:out1:RootOutput(write)                     2.37802       3.49736       4.86165       3.44447      0.668098        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5212.75 MB
  Peak resident set size usage (VmHWM): 4371.17 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11179720
-rw-r--r-- 1 pltdune04 pltdune 2578569854 Dec 28 11:46 detsim_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 2438890700 Dec 28 11:44 detsim_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2301545925 Dec 28 13:08 reco_stage1.root
-rw-r--r-- 1 pltdune04 pltdune 2066742700 Dec 28 09:14 g4_stage2.root
-rw-r--r-- 1 pltdune04 pltdune 1978347050 Dec 28 08:56 g4_stage1.root
-rw-r--r-- 1 pltdune04 pltdune   83131420 Dec 28 13:02 Pandora_Events.pndr
-rw-r--r-- 1 pltdune04 pltdune     242885 Dec 28 13:08 jobscript.log
-rw-r--r-- 1 pltdune04 pltdune     217088 Dec 28 13:08 mem.db
-rw-r--r-- 1 pltdune04 pltdune      62376 Dec 28 13:08 Pandora_Geometry.xml
-rw-r--r-- 1 pltdune04 pltdune      36864 Dec 28 13:08 time.db
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 11:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 09:14 g4_protoDUNE_hist.root
-rw-r--r-- 1 pltdune04 pltdune        519 Dec 28 13:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pltdune04 pltdune         52 Dec 28 08:48 all-input-dids.txt
-rw-r--r-- 1 pltdune04 pltdune          7 Dec 28 13:08 justin-processed-pfns.txt
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 11:54 ProtoDUNE.log
-rw-r--r-- 1 pltdune04 pltdune          0 Dec 28 11:46 debugprod.log
justIN time: 2026-02-07 02:18:04 UTC       justIN version: 01.06.00