justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271353.0@dunegpschedd01.fnal.gov

Jobsub ID271353.0@dunegpschedd01.fnal.gov
Workflow ID11473
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-02 19:12:47
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-01-02 21:59:40
From worker nodeHostnametd808.pic.es
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-02 19:13:51
Input filesmonte-carlo-011473-000001
JobscriptExit code0
Real time2h (9911s)
CPU time2h (9431s = 95%)
Max RSS bytes4014813184 (3828 MiB)
Outputting started2026-01-02 21:59:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11473/1/001/reco_stage1.root
Finished2026-01-02 21:59:40
Saved logsjustin-logs:271353.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122772 traces, 0 summary
Retagger: tagging trace set: gauss with 96958 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96958 traces tagged "gauss"
FrameSaver: q=3.26064e+06 n=1711690 tag=gauss
wclsFrameSaver: saving 122772 traces tagged "wiener"
FrameSaver: q=3.43879e+06 n=1516910 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33760 XUs and 35822 XVs -> 731 XUVs
C:0 T:1 72374 XUs and 71842 XVs -> 5634 XUVs
C:0 T:2 42987 XUs and 51703 XVs -> 1909 XUVs
C:0 T:3 23814 XUs and 22755 XVs -> 82 XUVs
C:0 T:4 10612 XUs and 11401 XVs -> 70 XUVs
C:0 T:5 48258 XUs and 34439 XVs -> 4293 XUVs
C:0 T:6 87255 XUs and 80083 XVs -> 3312 XUVs
C:0 T:7 35345 XUs and 29891 XVs -> 255 XUVs
C:0 T:8 11238 XUs and 10232 XVs -> 172 XUVs
C:0 T:9 89543 XUs and 102021 XVs -> 7465 XUVs
C:0 T:10 30711 XUs and 24154 XVs -> 2641 XUVs
C:0 T:11 111 XUs and 80 XVs -> 15 XUVs
26579 XUVs total
11491 collection wire objects
24119 potential space points
Neighbour search...
1330413 tests to find 624892 neighbours
Iterating with no regularization...
Begin: 7.06281e+07
0 5.99658e+07
1 5.91288e+07
2 5.89941e+07
3 5.89559e+07
Now with regularization...
Begin: 4.90554e+07
0 4.90127e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411500471
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411500471
fLifetime: 30000 [us]
02-Jan-2026 22:59:01 CET  Closed output file "reco_stage1.root"
02-Jan-2026 22:59:01 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          224.54        309.013       465.145       301.433       69.0693        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0011174     0.0030808    0.00525857    0.00301353    0.00122875       10     
reco:ophit:OpHitFinder                             0.204379      0.246652      0.316278       0.23451      0.0368633       10     
reco:opflash:OpFlashFinder                         0.0909881     0.134794      0.187216      0.126816      0.030514        10     
reco:opslicer:OpSlicer                             0.416676      0.587028      0.818872      0.550077      0.121948        10     
reco:caldata:DataPrepModule                         4.32123       4.98631       5.53336       5.06861      0.373831        10     
reco:wclsdatasp:WireCellToolkit                     29.0185       33.5454       37.5919       34.0176       3.05813        10     
reco:gaushit:GausHitFinder                         0.739125       1.09263        1.515        1.07829      0.237495        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000381291   0.000516822   0.000643342   0.000475312   8.82289e-05      10     
reco:reco3d:SpacePointSolver                        4.79898        9.504        18.4158       8.29998       4.05301        10     
reco:hitpdune:DisambigFromSpacePoints              0.673284       1.21686       2.03557       1.11972      0.461415        10     
reco:pandora:StandardPandora                        13.8891       27.9866       56.9175        23.84        12.1842        10     
reco:pandoraWriter:StandardPandora                 0.275359      0.388873      0.602097       0.37204      0.0916973       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.67419       3.18518       5.79528       2.59365       1.43682        10     
reco:pandoraShower:LArPandoraShowerCreation         1.28298       3.38951       6.98904       3.12623       1.66381        10     
reco:pandoracalo:Calorimetry                       0.873851       1.52776        2.496        1.35576      0.533886        10     
reco:pandoracalonosce:Calorimetry                  0.777547       1.40565       2.49303       1.19665      0.578664        10     
reco:pandorapid:Chi2ParticleID                     0.0040105    0.00487963    0.00604143    0.00470474    0.000760244      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0336486     0.0434157     0.0569863     0.0421713    0.00773306       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00380313    0.00465845    0.00584816    0.00446066    0.000632247      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.19892       3.01899       5.82357       2.66102       1.25458        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.15719       2.91154       5.77983       2.56433       1.27387        10     
reco:emtrkmichelid:EmTrackMichelId                  136.522       179.007       245.151       177.805       32.7588        10     
reco:crttag:SingleCRTMatchingProducer             0.00707516     0.012342      0.0210333     0.0114495    0.00396632       10     
reco:crtreco:TwoCRTMatchingProducer                0.0047186    0.00808994     0.0165235    0.00706761    0.00317338       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.460737       0.68583      0.875723      0.671526      0.134579        10     
reco:pandora2Track:LArPandoraTrackCreation          4.62443       10.3382       18.0902       10.2219       4.55508        10     
reco:pandora2calo:Calorimetry                       2.06022       4.39835       8.0757        4.39343       1.96519        10     
reco:pandora2calonosce:Calorimetry                  1.9433        4.06556       8.06263       4.11046       1.78268        10     
reco:pandora2pid:Chi2ParticleID                   0.00918326     0.0123899     0.0174732     0.0115142    0.00267112       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0412499     0.0562399     0.0805137     0.0563558     0.0109651       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0089667     0.0121443     0.016745      0.0114656    0.00249502       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.7988        4.13156       9.85297       3.80808       2.2998         10     
reco:pandora2Showercalo:ShowerCalorimetry           1.85855       4.49364       12.6252       3.92301       3.03784        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.77598       3.88025       10.9971       3.52337       2.56066        10     
[art]:TriggerResults:TriggerResultInserter        2.2449e-05    4.09501e-05   0.000102251   3.5205e-05    2.21262e-05      10     
end_path:out1:RootOutput                           5.57e-06     9.5239e-06     1.458e-05     9.375e-06    2.1099e-06       10     
end_path:out1:RootOutput(write)                     2.01866       2.61288       3.81525       2.49824      0.513977        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4851.32 MB
  Peak resident set size usage (VmHWM): 4014.81 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.07 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9114252
-rw-r--r-- 1 dune001 dune 2189843418 Jan  2 22:05 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2084652861 Jan  2 22:03 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1839927703 Jan  2 22:59 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1606975703 Jan  2 20:31 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1544113563 Jan  2 20:21 g4_stage1.root
-rw-r--r-- 1 dune001 dune   66852850 Jan  2 22:55 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242873 Jan  2 22:59 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan  2 22:59 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan  2 22:59 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan  2 22:59 time.db
-rw-r--r-- 1 dune001 dune        519 Jan  2 22:05 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 20:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 22:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan  2 20:14 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan  2 22:59 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan  2 22:09 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan  2 22:05 debugprod.log
justIN time: 2026-02-06 22:18:49 UTC       justIN version: 01.06.00