justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302385.0@dunegpschedd01.fnal.gov

Jobsub ID302385.0@dunegpschedd01.fnal.gov
Workflow ID12575
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 13:59:13
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-30 17:12:03
From worker nodeHostnamehnode80.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 14:00:50
Input filesmonte-carlo-012575-000001
JobscriptExit code0
Real time3h (11406s)
CPU time2h (7722s = 67%)
Max RSS bytes4108181504 (3917 MiB)
Outputting started2026-01-30 17:10:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12575/1/001/reco_stage1.root
Finished2026-01-30 17:12:03
Saved logsjustin-logs:302385.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

g tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 124020 traces, 0 summary
Retagger: tagging trace set: gauss with 91494 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 91494 traces tagged "gauss"
FrameSaver: q=3.08196e+06 n=1560408 tag=gauss
wclsFrameSaver: saving 124020 traces tagged "wiener"
FrameSaver: q=3.2762e+06 n=1359195 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32948 XUs and 39608 XVs -> 408 XUVs
C:0 T:1 62427 XUs and 86227 XVs -> 8596 XUVs
C:0 T:2 27069 XUs and 45666 XVs -> 1743 XUVs
C:0 T:3 15253 XUs and 22340 XVs -> 71 XUVs
C:0 T:4 10643 XUs and 13782 XVs -> 108 XUVs
C:0 T:5 57613 XUs and 86488 XVs -> 6471 XUVs
C:0 T:6 114109 XUs and 74762 XVs -> 7818 XUVs
C:0 T:7 45821 XUs and 26544 XVs -> 455 XUVs
C:0 T:8 11677 XUs and 11784 XVs -> 674 XUVs
C:0 T:9 66761 XUs and 90559 XVs -> 4939 XUVs
C:0 T:10 17210 XUs and 10926 XVs -> 1720 XUVs
C:0 T:11 124 XUs and 108 XVs -> 10 XUVs
33013 XUVs total
11117 collection wire objects
30018 potential space points
Neighbour search...
2187368 tests to find 1135576 neighbours
Iterating with no regularization...
Begin: 5.94153e+07
0 4.86688e+07
1 4.77943e+07
2 4.76563e+07
3 4.76151e+07
Now with regularization...
Begin: 3.83611e+07
0 3.83148e+07
1 3.82926e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412059303
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412059303
fLifetime: 30000 [us]
30-Jan-2026 18:10:55 CET  Closed output file "reco_stage1.root"
30-Jan-2026 18:10:55 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          224.117       355.488       605.576       291.628       136.928        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000652275   0.00319628    0.00648277    0.00296245    0.00180225       10     
reco:ophit:OpHitFinder                             0.178237      0.224742      0.309039      0.215801      0.0376787       10     
reco:opflash:OpFlashFinder                         0.113318      0.137082      0.188791       0.13071      0.0223957       10     
reco:opslicer:OpSlicer                             0.344859      0.479354      0.605341      0.490093      0.0968002       10     
reco:caldata:DataPrepModule                         3.4796        4.09191       4.7375        4.04911      0.443309        10     
reco:wclsdatasp:WireCellToolkit                     24.089        40.4787       150.54        30.1994       36.8211        10     
reco:gaushit:GausHitFinder                          0.65673      0.926488       1.44686      0.849802      0.219403        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000287337   0.000384415   0.000479042   0.00037953    6.30893e-05      10     
reco:reco3d:SpacePointSolver                        5.86462       22.9115       147.46        9.38452       41.5686        10     
reco:hitpdune:DisambigFromSpacePoints              0.898846       1.17996       1.92515       1.11561      0.281864        10     
reco:pandora:StandardPandora                        16.0062       62.5826       289.298       23.3023       84.912         10     
reco:pandoraWriter:StandardPandora                 0.238576      0.315171      0.523934      0.299674      0.0744924       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.0441        3.05349       5.9736        2.69353       1.08704        10     
reco:pandoraShower:LArPandoraShowerCreation         2.18306       3.12885       5.96682       2.83247       1.03634        10     
reco:pandoracalo:Calorimetry                        1.37301       1.72383       2.41558       1.51613      0.368227        10     
reco:pandoracalonosce:Calorimetry                   1.19326       1.65351       2.21122       1.64517      0.331873        10     
reco:pandorapid:Chi2ParticleID                    0.00248896     0.0039705     0.0072096    0.00370552    0.00130383       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0299623     0.0365635     0.053111      0.035832     0.00583355       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0027031    0.00358659    0.00505794    0.00351283    0.000585211      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.2204        3.3543        5.18068       3.31462      0.879161        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.9578        3.13621       4.99316       3.12851      0.831854        10     
reco:emtrkmichelid:EmTrackMichelId                  119.057       172.912       406.366       135.503       84.107         10     
reco:crttag:SingleCRTMatchingProducer             0.00704695     0.0103399     0.0178366    0.00929942    0.00327192       10     
reco:crtreco:TwoCRTMatchingProducer               0.00406629    0.00776361     0.0130629    0.00691931    0.00304503       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.61421      0.757875      0.937126      0.753343      0.112659        10     
reco:pandora2Track:LArPandoraTrackCreation          6.11085       9.21218       16.155        8.83043       2.72771        10     
reco:pandora2calo:Calorimetry                       2.79972       4.75544       8.77425       4.44002       1.68016        10     
reco:pandora2calonosce:Calorimetry                  2.90957       4.55262       8.01334       4.11154       1.41262        10     
reco:pandora2pid:Chi2ParticleID                   0.00677785    0.00964912     0.0143137    0.00927113    0.00225634       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0381684     0.0574166     0.0864844     0.0553518     0.0150887       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00776652     0.010856      0.0154162     0.0107757    0.00241631       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.28167       3.49024       5.88151       3.17563       1.04812        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.47236       4.19238       6.97297       4.30761       1.23068        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.30875       3.82881       6.43787       3.6312        1.31944        10     
[art]:TriggerResults:TriggerResultInserter         1.982e-05    3.41018e-05   8.6062e-05    2.49905e-05   1.89624e-05      10     
end_path:out1:RootOutput                           3.95e-06     5.1372e-06     9.061e-06    4.3655e-06    1.48956e-06      10     
end_path:out1:RootOutput(write)                     1.58662       2.16033       3.29427       2.05338      0.480239        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4925.67 MB
  Peak resident set size usage (VmHWM): 4108.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.60 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9652128
-rw-r--r-- 1 dune001 dune 2285604298 Jan 30 17:08 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2172506551 Jan 30 17:06 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1954636812 Jan 30 18:10 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1733858949 Jan 30 15:20 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1665033142 Jan 30 15:09 g4_stage1.root
-rw-r--r-- 1 dune001 dune   71505190 Jan 30 18:07 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242628 Jan 30 18:10 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 30 18:10 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 30 18:10 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 30 18:10 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 30 17:08 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 30 15:20 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 30 18:10 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 30 15:00 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 30 18:10 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 30 17:12 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 30 17:08 debugprod.log
justIN time: 2026-02-04 13:58:37 UTC       justIN version: 01.06.00