justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271301.1@dunegpschedd01.fnal.gov

Jobsub ID271301.1@dunegpschedd01.fnal.gov
Workflow ID11462
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-02 05:36:05
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2026-01-02 08:57:41
From worker nodeHostnametds509.pic.es
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-02 05:38:10
Input filesmonte-carlo-011462-000001
JobscriptExit code0
Real time3h (11929s)
CPU time3h (11274s = 94%)
Max RSS bytes4053643264 (3865 MiB)
Outputting started2026-01-02 08:57:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11462/1/001/reco_stage1.root
Finished2026-01-02 08:57:41
Saved logsjustin-logs:271301.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lBasedRawDigitPrepService:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 97671 traces, 0 summary
Retagger: tagging trace set: gauss with 82057 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 82057 traces tagged "gauss"
FrameSaver: q=2.91269e+06 n=1570854 tag=gauss
wclsFrameSaver: saving 97671 traces tagged "wiener"
FrameSaver: q=3.0382e+06 n=1442399 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28497 XUs and 30592 XVs -> 390 XUVs
C:0 T:1 39803 XUs and 38319 XVs -> 1973 XUVs
C:0 T:2 105735 XUs and 108108 XVs -> 4216 XUVs
C:0 T:3 57957 XUs and 46715 XVs -> 368 XUVs
C:0 T:4 15074 XUs and 16427 XVs -> 105 XUVs
C:0 T:5 51245 XUs and 51196 XVs -> 3512 XUVs
C:0 T:6 57746 XUs and 49491 XVs -> 1722 XUVs
C:0 T:7 26384 XUs and 21427 XVs -> 106 XUVs
C:0 T:8 7546 XUs and 9085 XVs -> 96 XUVs
C:0 T:9 44867 XUs and 69943 XVs -> 3626 XUVs
C:0 T:10 8500 XUs and 3663 XVs -> 794 XUVs
C:0 T:11 28 XUs and 17 XVs -> 4 XUVs
16912 XUVs total
7781 collection wire objects
15594 potential space points
Neighbour search...
484688 tests to find 253532 neighbours
Iterating with no regularization...
Begin: 7.67274e+07
0 6.5953e+07
1 6.53755e+07
2 6.5313e+07
Now with regularization...
Begin: 5.74116e+07
0 5.7401e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411477793
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411477793
fLifetime: 30000 [us]
02-Jan-2026 09:56:57 CET  Closed output file "reco_stage1.root"
02-Jan-2026 09:56:57 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          269.746       352.316       511.049       331.917       64.8415        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00135035    0.00458904    0.00741229    0.00495793    0.00192143       10     
reco:ophit:OpHitFinder                             0.264509      0.334397      0.408829      0.340633      0.0454516       10     
reco:opflash:OpFlashFinder                         0.135556      0.183706      0.234251      0.173968      0.0287135       10     
reco:opslicer:OpSlicer                             0.545377      0.770795       1.06151      0.773514      0.178673        10     
reco:caldata:DataPrepModule                         4.87608       6.81255       7.99999       7.09545      0.837011        10     
reco:wclsdatasp:WireCellToolkit                     29.3729       39.2093       51.3016       40.3948       6.66893        10     
reco:gaushit:GausHitFinder                         0.865313       1.23024       1.79279       1.13847      0.322841        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000322436   0.000521425   0.000801627   0.000500045   0.000149716      10     
reco:reco3d:SpacePointSolver                        8.45345       14.5843       24.2626       12.7077       4.85902        10     
reco:hitpdune:DisambigFromSpacePoints                1.104        1.39673       2.40266       1.23458      0.373923        10     
reco:pandora:StandardPandora                        18.1059       29.6127       65.7093       25.5383       13.0212        10     
reco:pandoraWriter:StandardPandora                 0.285286      0.438613       0.70698      0.456197      0.118791        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.68758       4.32925       8.58305       3.62147       1.74598        10     
reco:pandoraShower:LArPandoraShowerCreation         2.09997       4.45073       9.06696       3.73524       2.09088        10     
reco:pandoracalo:Calorimetry                        1.22558       2.32211       3.83145       2.08707      0.780457        10     
reco:pandoracalonosce:Calorimetry                   1.27957       2.40384       3.78736       2.63879      0.803215        10     
reco:pandorapid:Chi2ParticleID                    0.00411882    0.00543586    0.00804601    0.00471472    0.00152524       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0358223      0.05195      0.0780215     0.0442887     0.0158457       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00387716    0.00550785    0.00814689    0.00499656    0.00144426       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.83141       4.53044       7.16606       3.82152       1.66787        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.99728       4.46307       8.05895       3.47577       1.84382        10     
reco:emtrkmichelid:EmTrackMichelId                  154.327       190.567       249.435       183.212       24.804         10     
reco:crttag:SingleCRTMatchingProducer              0.0126841     0.0177286     0.0241864     0.0171903    0.00347236       10     
reco:crtreco:TwoCRTMatchingProducer               0.00832431     0.0119334     0.0188652     0.0106996     0.0034177       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.775522      0.899901       1.07249      0.902364      0.0849715       10     
reco:pandora2Track:LArPandoraTrackCreation          7.14646       13.191        28.4701       11.1275       6.0743         10     
reco:pandora2calo:Calorimetry                       3.87001       6.62066       10.7973       6.35887       2.09126        10     
reco:pandora2calonosce:Calorimetry                  3.38002       6.36863       11.4928       5.84969       2.34664        10     
reco:pandora2pid:Chi2ParticleID                   0.00879441     0.0145846     0.023184      0.0138488    0.00474476       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.044037      0.0727431     0.105383      0.0799374     0.0210573       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00885378     0.0149765     0.0228407     0.0151586    0.00474698       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.60301       4.37249       6.32173       4.41131       1.30292        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.11732       5.49889       9.66043       5.36341        1.957         10     
reco:pandora2Showercalonosce:ShowerCalorimetry       2.769        4.68181       7.66408       4.50446       1.34818        10     
[art]:TriggerResults:TriggerResultInserter        4.4184e-05    8.23692e-05   0.000179267   7.36685e-05   3.78185e-05      10     
end_path:out1:RootOutput                           9.708e-06    1.42539e-05   2.5388e-05     1.324e-05    4.38358e-06      10     
end_path:out1:RootOutput(write)                     1.99075       2.68634       3.74426       2.47388      0.566435        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4880.53 MB
  Peak resident set size usage (VmHWM): 4053.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.34 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9822520
-rw-r--r-- 1 dune001 dune 2328413654 Jan  2 08:55 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2213093057 Jan  2 08:53 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1980410794 Jan  2 09:56 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1773274196 Jan  2 06:59 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1695558628 Jan  2 06:46 g4_stage1.root
-rw-r--r-- 1 dune001 dune   66883210 Jan  2 09:53 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242805 Jan  2 09:56 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan  2 09:56 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan  2 09:56 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan  2 09:56 time.db
-rw-r--r-- 1 dune001 dune        519 Jan  2 08:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 06:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan  2 09:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan  2 06:38 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan  2 09:56 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan  2 09:01 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan  2 08:55 debugprod.log
justIN time: 2026-02-06 23:32:34 UTC       justIN version: 01.06.00