justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243311.0@dunegpschedd01.fnal.gov

Jobsub ID243311.0@dunegpschedd01.fnal.gov
Workflow ID9608
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-05 04:30:56
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_26
Last heartbeat2025-11-05 09:19:02
From worker nodeHostnamewn-a4-16
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-05 04:32:18
Input filesmonte-carlo-009608-000001
JobscriptExit code0
Real time4h (17104s)
CPU time4h (16399s = 95%)
Max RSS bytes5454942208 (5202 MiB)
Outputting started2025-11-05 09:17:24
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09608/1/001/reco_stage1.root
Finished2025-11-05 09:19:02
Saved logsjustin-logs:243311.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 130588 traces, 0 summary
Retagger: tagging trace set: gauss with 96674 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96674 traces tagged "gauss"
FrameSaver: q=3.327e+06 n=1676631 tag=gauss
wclsFrameSaver: saving 130588 traces tagged "wiener"
FrameSaver: q=3.5401e+06 n=1462504 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38949 XUs and 41661 XVs -> 637 XUVs
C:0 T:1 71784 XUs and 96111 XVs -> 8595 XUVs
C:0 T:2 161558 XUs and 187065 XVs -> 17858 XUVs
C:0 T:3 51390 XUs and 60781 XVs -> 1094 XUVs
C:0 T:4 8246 XUs and 9524 XVs -> 131 XUVs
C:0 T:5 36020 XUs and 28740 XVs -> 3629 XUVs
C:0 T:6 117448 XUs and 113548 XVs -> 8642 XUVs
C:0 T:7 42435 XUs and 37912 XVs -> 313 XUVs
C:0 T:8 4644 XUs and 2957 XVs -> 40 XUVs
C:0 T:9 53383 XUs and 27499 XVs -> 3472 XUVs
C:0 T:10 70606 XUs and 41494 XVs -> 8708 XUVs
C:0 T:11 215 XUs and 139 XVs -> 14 XUVs
53133 XUVs total
12136 collection wire objects
47114 potential space points
Neighbour search...
5366622 tests to find 2725206 neighbours
Iterating with no regularization...
Begin: 5.73726e+07
0 4.55603e+07
1 4.46107e+07
2 4.44538e+07
3 4.44026e+07
4 4.43768e+07
Now with regularization...
Begin: 3.49315e+07
0 3.48703e+07
1 3.48259e+07
2 3.47925e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410321212
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410321212
fLifetime: 30000 [us]
05-Nov-2025 09:17:21 GMT  Closed output file "reco_stage1.root"
05-Nov-2025 09:17:21 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          400.711       554.458       723.062       553.317       80.8791        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00155222    0.00498294    0.00966013    0.00495185    0.00234172       10     
reco:ophit:OpHitFinder                             0.287804      0.418627      0.524031      0.374866      0.0791902       10     
reco:opflash:OpFlashFinder                         0.183401      0.279891      0.357934      0.277339      0.0499615       10     
reco:opslicer:OpSlicer                              0.68731       1.07865       1.5584        1.04098       0.29167        10     
reco:caldata:DataPrepModule                         5.38137       7.34486       8.60248       7.78517      0.982553        10     
reco:wclsdatasp:WireCellToolkit                     36.3461       44.6995       53.2644       44.3278       5.31964        10     
reco:gaushit:GausHitFinder                          1.0646        1.79003       2.43665       1.7467       0.395795        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000536303   0.000800842   0.00108023    0.000806154   0.000155488      10     
reco:reco3d:SpacePointSolver                        15.7539       22.4076       28.7259       22.5673       4.22945        10     
reco:hitpdune:DisambigFromSpacePoints               1.5063        2.52102       3.37324       2.47487      0.626024        10     
reco:pandora:StandardPandora                        32.807        55.3195       81.1549       50.3648       17.0943        10     
reco:pandoraWriter:StandardPandora                 0.477952      0.667314      0.822953      0.656554      0.104028        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.07302       6.99001       11.0871       6.7804        1.56904        10     
reco:pandoraShower:LArPandoraShowerCreation         4.46047       8.02611       12.2874       7.79725       2.38407        10     
reco:pandoracalo:Calorimetry                        2.52456       3.91881       6.18007       3.41492       1.17586        10     
reco:pandoracalonosce:Calorimetry                   2.50013       3.87274       5.33195       3.73232      0.994083        10     
reco:pandorapid:Chi2ParticleID                    0.00658733     0.0101314     0.013355     0.00993076     0.0021369       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0651887     0.0993744     0.122955      0.105759      0.0207952       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00625344     0.0096251     0.0130627     0.0099018    0.00194968       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.60653       8.44272       14.0152       7.76347       2.61809        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.81604       7.68633       13.4482       7.65774       2.57945        10     
reco:emtrkmichelid:EmTrackMichelId                  214.919       303.964       388.477       302.893       40.863         10     
reco:crttag:SingleCRTMatchingProducer              0.014988      0.0196685     0.0251593     0.0190383    0.00339823       10     
reco:crtreco:TwoCRTMatchingProducer               0.00811083     0.0145503     0.0215017     0.0136177     0.0037099       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.974018       1.33498       1.70788       1.34139       0.19151        10     
reco:pandora2Track:LArPandoraTrackCreation          14.8477       24.0878       38.0537       20.9546       6.88069        10     
reco:pandora2calo:Calorimetry                       5.51774       10.0683       19.7533       9.64819       3.78808        10     
reco:pandora2calonosce:Calorimetry                  5.76079       9.7422        18.3051       9.13904       3.40445        10     
reco:pandora2pid:Chi2ParticleID                    0.0154296     0.0207201     0.0338973     0.0197702    0.00494035       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0720826     0.102152      0.174893      0.0896681     0.0286692       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0162924     0.0207784     0.0326768     0.0199309    0.00458019       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.3086        8.21316       13.9374       7.91162       2.39185        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.58079       8.66216       12.0449       8.49392       2.37476        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.43713       8.02006       10.9016       7.5542        2.11793        10     
[art]:TriggerResults:TriggerResultInserter        5.7141e-05    7.52257e-05   0.000117423   7.0806e-05    1.91981e-05      10     
end_path:out1:RootOutput                           1.147e-05    3.22122e-05   0.000197065   1.4126e-05    5.49647e-05      10     
end_path:out1:RootOutput(write)                     3.09205       4.42018       5.78567       4.44917       0.97237        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6285.12 MB
  Peak resident set size usage (VmHWM): 5454.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.57 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11761444
-rw-r--r--. 1 pildune003 dunepil 2702118934 Nov  5 07:41 detsim_stage2.root
-rw-r--r--. 1 pildune003 dunepil 2565400649 Nov  5 07:37 detsim_stage1.root
-rw-r--r--. 1 pildune003 dunepil 2430070340 Nov  5 09:17 reco_stage1.root
-rw-r--r--. 1 pildune003 dunepil 2175128961 Nov  5 04:59 g4_stage2.root
-rw-r--r--. 1 pildune003 dunepil 2085777643 Nov  5 04:40 g4_stage1.root
-rw-r--r--. 1 pildune003 dunepil   84600360 Nov  5 09:12 Pandora_Events.pndr
-rw-r--r--. 1 pildune003 dunepil     243014 Nov  5 09:17 jobscript.log
-rw-r--r--. 1 pildune003 dunepil     217088 Nov  5 09:17 mem.db
-rw-r--r--. 1 pildune003 dunepil      62376 Nov  5 09:17 Pandora_Geometry.xml
-rw-r--r--. 1 pildune003 dunepil      36864 Nov  5 09:17 time.db
-rw-r--r--. 1 pildune003 dunepil        519 Nov  5 07:41 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov  5 04:59 g4_protoDUNE_hist.root
-rw-r--r--. 1 pildune003 dunepil        519 Nov  5 09:17 protoDUNE_reco1_hist.root
-rw-r--r--. 1 pildune003 dunepil         52 Nov  5 04:32 all-input-dids.txt
-rw-r--r--. 1 pildune003 dunepil          7 Nov  5 09:17 justin-processed-pfns.txt
-rw-r--r--. 1 pildune003 dunepil          0 Nov  5 07:51 ProtoDUNE.log
-rw-r--r--. 1 pildune003 dunepil          0 Nov  5 07:41 debugprod.log
justIN time: 2026-02-11 07:48:01 UTC       justIN version: 01.06.00