justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 296362.0@dunegpschedd02.fnal.gov

Jobsub ID296362.0@dunegpschedd02.fnal.gov
Workflow ID12491
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-28 16:18:52
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-01-28 19:14:54
From worker nodeHostnamecn107.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-28 16:19:34
Input filesmonte-carlo-012491-000001
JobscriptExit code0
Real time2h (10306s)
CPU time2h (10137s = 98%)
Max RSS bytes4216053760 (4020 MiB)
Outputting started2026-01-28 19:11:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12491/1/001/reco_stage1.root
Finished2026-01-28 19:14:54
Saved logsjustin-logs:296362.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

asedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 119934 traces, 0 summary
Retagger: tagging trace set: gauss with 90814 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90814 traces tagged "gauss"
FrameSaver: q=3.28943e+06 n=1639057 tag=gauss
wclsFrameSaver: saving 119934 traces tagged "wiener"
FrameSaver: q=3.4721e+06 n=1456014 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24327 XUs and 24587 XVs -> 478 XUVs
C:0 T:1 46017 XUs and 42727 XVs -> 2503 XUVs
C:0 T:2 115825 XUs and 119351 XVs -> 8447 XUVs
C:0 T:3 49133 XUs and 46056 XVs -> 511 XUVs
C:0 T:4 12517 XUs and 15117 XVs -> 82 XUVs
C:0 T:5 54248 XUs and 55865 XVs -> 3931 XUVs
C:0 T:6 114438 XUs and 109827 XVs -> 5757 XUVs
C:0 T:8 6552 XUs and 7935 XVs -> 98 XUVs
C:0 T:9 37305 XUs and 55595 XVs -> 3419 XUVs
C:0 T:10 27862 XUs and 22831 XVs -> 3033 XUVs
C:0 T:11 84 XUs and 146 XVs -> 12 XUVs
28271 XUVs total
10678 collection wire objects
25466 potential space points
Neighbour search...
1633068 tests to find 824300 neighbours
Iterating with no regularization...
Begin: 8.45035e+07
0 7.0821e+07
1 6.94561e+07
2 6.93573e+07
3 6.93313e+07
Now with regularization...
Begin: 5.89853e+07
0 5.89332e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 412020996
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412020996
fLifetime: 30000 [us]
28-Jan-2026 19:11:19 GMT  Closed output file "reco_stage1.root"
28-Jan-2026 19:11:19 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          330.322       467.707       573.44        467.293       76.8027        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00123203    0.00321828    0.00511785    0.00341102    0.00131503       10     
reco:ophit:OpHitFinder                             0.216825      0.307493      0.393265      0.324043      0.0549061       10     
reco:opflash:OpFlashFinder                         0.150523      0.177474      0.219688      0.173961      0.0195279       10     
reco:opslicer:OpSlicer                              0.53452      0.769099       1.07073      0.756823      0.157935        10     
reco:caldata:DataPrepModule                         4.74194       6.79652       7.85712       7.46777       1.25341        10     
reco:wclsdatasp:WireCellToolkit                     38.2466       50.1263       55.8009       51.1094       5.36465        10     
reco:gaushit:GausHitFinder                          1.17536       1.4721        2.14117       1.36676      0.312687        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000630085   0.000805152   0.00120866    0.000738546   0.000154761      10     
reco:reco3d:SpacePointSolver                        6.95689       13.8674       19.1201       13.4248       3.55575        10     
reco:hitpdune:DisambigFromSpacePoints                1.507        1.93643       2.7241         1.843       0.371928        10     
reco:pandora:StandardPandora                        22.5103       36.9029       58.2383       34.5524       9.53959        10     
reco:pandoraWriter:StandardPandora                 0.370229      0.490814      0.641809      0.463721      0.0890764       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.45986       8.45152       12.6805       7.9769        2.19086        10     
reco:pandoraShower:LArPandoraShowerCreation         4.25013       8.89061       11.5883       9.46331       2.43778        10     
reco:pandoracalo:Calorimetry                        2.51138       3.94496       5.75949       3.84415      0.919575        10     
reco:pandoracalonosce:Calorimetry                   2.29326       3.65309       5.71068       3.65458      0.994263        10     
reco:pandorapid:Chi2ParticleID                     0.0041415     0.0056251    0.00838977    0.00497055    0.00133772       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0478787     0.0629824     0.0952057     0.0589825     0.013151        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00429516    0.00579774    0.00889463    0.00512922     0.0014358       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.40362       8.21461       11.4763       8.52905       2.14996        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.05992       8.03677       11.3475       8.38174       2.06899        10     
reco:emtrkmichelid:EmTrackMichelId                  158.94        229.843       304.675       245.212       50.7857        10     
reco:crttag:SingleCRTMatchingProducer              0.0188692     0.0288385     0.0388198     0.0277646    0.00638111       10     
reco:crtreco:TwoCRTMatchingProducer                0.0118661     0.0211473     0.0292121     0.020724     0.00550508       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.597756       1.10136       1.59605      0.995864      0.291416        10     
reco:pandora2Track:LArPandoraTrackCreation          12.089        28.6955       39.6492       29.2603       8.51004        10     
reco:pandora2calo:Calorimetry                       4.18683       11.8306       18.3476       11.8571       4.36333        10     
reco:pandora2calonosce:Calorimetry                  3.49876       11.2737       15.9668       11.6481       3.82974        10     
reco:pandora2pid:Chi2ParticleID                   0.00920402     0.0160307     0.021674      0.0173126    0.00460215       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0525869     0.0855223     0.107496      0.0924442     0.0204841       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00919272     0.0158978     0.0214833     0.0171553    0.00455233       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.27909       8.25694       12.5301       7.43951       3.24686        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.4405        9.64984       15.6353       9.19524       3.67525        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.13974       9.48908       15.4972       9.23011       3.56929        10     
[art]:TriggerResults:TriggerResultInserter        3.6398e-05    6.95427e-05   9.4975e-05     6.802e-05    1.81586e-05      10     
end_path:out1:RootOutput                           7.423e-06    1.01952e-05   1.5186e-05    9.6225e-06    2.23628e-06      10     
end_path:out1:RootOutput(write)                     1.87842       3.14005       4.20184       3.42714      0.792737        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5021.01 MB
  Peak resident set size usage (VmHWM): 4216.05 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.23 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10063076
-rw-r--r-- 1 pildune22 pildune 2363889994 Jan 28 17:51 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2248234068 Jan 28 17:49 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2045937863 Jan 28 19:11 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1815069044 Jan 28 16:36 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1756777392 Jan 28 16:24 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   74062580 Jan 28 19:04 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242836 Jan 28 19:11 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Jan 28 19:11 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Jan 28 19:11 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Jan 28 19:11 time.db
-rw-r--r-- 1 pildune22 pildune        519 Jan 28 17:51 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 28 16:36 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Jan 28 19:11 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Jan 28 16:19 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Jan 28 19:11 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Jan 28 17:56 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Jan 28 17:51 debugprod.log
justIN time: 2026-02-04 16:37:59 UTC       justIN version: 01.06.00