justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269579.0@dunegpschedd01.fnal.gov

Jobsub ID269579.0@dunegpschedd01.fnal.gov
Workflow ID11302
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-21 21:31:47
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-12-22 03:28:13
From worker nodeHostnamehd77.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-21 21:32:43
Input filesmonte-carlo-011302-000001
JobscriptExit code0
Real time5h (21229s)
CPU time5h (20759s = 97%)
Max RSS bytes4301107200 (4101 MiB)
Outputting started2025-12-22 03:26:33
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11302/1/001/reco_stage1.root
Finished2025-12-22 03:28:13
Saved logsjustin-logs:269579.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123807 traces, 0 summary
Retagger: tagging trace set: gauss with 95139 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95139 traces tagged "gauss"
FrameSaver: q=3.13259e+06 n=1696596 tag=gauss
wclsFrameSaver: saving 123807 traces tagged "wiener"
FrameSaver: q=3.31501e+06 n=1505635 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28932 XUs and 31608 XVs -> 323 XUVs
C:0 T:1 59227 XUs and 54846 XVs -> 3109 XUVs
C:0 T:2 108796 XUs and 109400 XVs -> 11264 XUVs
C:0 T:3 43484 XUs and 41314 XVs -> 356 XUVs
C:0 T:4 10800 XUs and 10662 XVs -> 114 XUVs
C:0 T:5 34894 XUs and 25237 XVs -> 3176 XUVs
C:0 T:6 80297 XUs and 74605 XVs -> 5342 XUVs
C:0 T:7 32488 XUs and 29704 XVs -> 220 XUVs
C:0 T:8 6424 XUs and 5426 XVs -> 246 XUVs
C:0 T:9 57791 XUs and 60721 XVs -> 10767 XUVs
C:0 T:10 88041 XUs and 51586 XVs -> 8376 XUVs
C:0 T:11 458 XUs and 270 XVs -> 64 XUVs
43357 XUVs total
10974 collection wire objects
40085 potential space points
Neighbour search...
6066105 tests to find 2827368 neighbours
Iterating with no regularization...
Begin: 5.93428e+07
0 4.58659e+07
1 4.50879e+07
2 4.50017e+07
3 4.49752e+07
Now with regularization...
Begin: 3.76855e+07
0 3.76423e+07
1 3.76135e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411261001
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411261001
fLifetime: 30000 [us]
22-Dec-2025 03:26:30 UTC  Closed output file "reco_stage1.root"
22-Dec-2025 03:26:30 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          448.529       604.94        982.436       579.224       143.319        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00125309    0.00464263     0.010014     0.00414536    0.00241056       10     
reco:ophit:OpHitFinder                             0.353203      0.387049      0.442031      0.381051      0.0290097       10     
reco:opflash:OpFlashFinder                          0.2265       0.269668      0.324419      0.266889      0.0303133       10     
reco:opslicer:OpSlicer                             0.977806       1.1602        1.43904       1.12609      0.156549        10     
reco:caldata:DataPrepModule                         7.22572       7.37561       7.5866        7.34992      0.122653        10     
reco:wclsdatasp:WireCellToolkit                     52.9544       56.3593       58.6265       56.7197       1.86804        10     
reco:gaushit:GausHitFinder                          1.52394       2.04967       2.83399       1.90767      0.357289        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000540806   0.000709507   0.000874012   0.000689666   0.000100791      10     
reco:reco3d:SpacePointSolver                        12.4683       20.2746       40.1254       16.9351       7.51394        10     
reco:hitpdune:DisambigFromSpacePoints               1.7606        2.73476       4.80897       2.59254      0.796113        10     
reco:pandora:StandardPandora                        30.0724       68.3731       186.667       51.9622       43.2727        10     
reco:pandoraWriter:StandardPandora                 0.515814      0.654689      0.906759      0.649146      0.101985        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.88028       7.81556       14.4258       7.39728       2.32277        10     
reco:pandoraShower:LArPandoraShowerCreation         5.81007       9.09407       17.9362       8.82448       3.41761        10     
reco:pandoracalo:Calorimetry                        3.1401        4.25673       7.62071       4.13703       1.19313        10     
reco:pandoracalonosce:Calorimetry                   2.88736       3.94148       6.74353       3.90396       1.03191        10     
reco:pandorapid:Chi2ParticleID                     0.0077658    0.00931739     0.0118354     0.0090939    0.00129387       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0729359     0.0944099     0.119405      0.0899851     0.0155924       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00745124    0.00882787     0.0114577    0.00866865    0.00109231       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.78624       9.11447       16.2233       9.13061       2.94862        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.69426       8.8154        16.0476       8.66786       2.94455        10     
reco:emtrkmichelid:EmTrackMichelId                  244.78        319.403       450.758       312.038       53.5616        10     
reco:crttag:SingleCRTMatchingProducer              0.0200924     0.0243907     0.0343883     0.0217436    0.00464781       10     
reco:crtreco:TwoCRTMatchingProducer                0.0123793     0.0155324     0.0225808     0.0152489    0.00301049       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.13366       1.39264       1.7851        1.39248      0.196463        10     
reco:pandora2Track:LArPandoraTrackCreation          16.6334       25.1669       49.3122       23.2302       9.04942        10     
reco:pandora2calo:Calorimetry                       7.75555       12.0989       22.1814       11.6461       3.98861        10     
reco:pandora2calonosce:Calorimetry                  7.54974       11.2849       20.8193       10.5136       3.71563        10     
reco:pandora2pid:Chi2ParticleID                    0.0187253     0.0236315     0.0328803     0.0241289    0.00373679       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0903635     0.116758      0.159851       0.11726      0.0185764       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0182701     0.0229487     0.0319274     0.0224157    0.00363324       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.6596        9.07102       16.7234       8.71956       3.01717        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.16978       10.0964       17.8668       10.2365       3.24994        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.5452        8.95929       15.3464       9.15564       2.73087        10     
[art]:TriggerResults:TriggerResultInserter        5.9031e-05    9.19668e-05   0.000169238   7.92685e-05   3.30344e-05      10     
end_path:out1:RootOutput                          1.1232e-05    1.34021e-05   2.2262e-05    1.1842e-05    3.48868e-06      10     
end_path:out1:RootOutput(write)                     3.44853       4.30996       6.13855       4.26967      0.726097        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5125.46 MB
  Peak resident set size usage (VmHWM): 4301.11 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.72 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10799972
-rw-r--r-- 1 dnpil001 users 2505450606 Dec 22 01:42 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2374287460 Dec 22 01:39 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2223362964 Dec 22 03:26 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 1973349458 Dec 21 22:01 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1899852534 Dec 21 21:40 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   82249770 Dec 22 03:19 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243182 Dec 22 03:26 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Dec 22 03:26 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Dec 22 03:26 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Dec 22 03:26 time.db
-rw-r--r-- 1 dnpil001 users        519 Dec 22 01:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 21 22:01 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 22 03:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Dec 21 21:32 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Dec 22 03:26 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Dec 22 01:50 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Dec 22 01:42 debugprod.log
justIN time: 2026-02-07 06:23:27 UTC       justIN version: 01.06.00