justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241132.1@dunegpschedd02.fnal.gov

Jobsub ID241132.1@dunegpschedd02.fnal.gov
Workflow ID9921
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-11 01:45:42
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-11-11 06:03:11
From worker nodeHostnamewn-lb-16.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-11 01:47:09
Input filesmonte-carlo-009921-000001
JobscriptExit code0
Real time4h (15273s)
CPU time4h (15188s = 99%)
Max RSS bytes4132360192 (3940 MiB)
Outputting started2025-11-11 06:01:42
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09921/1/001/reco_stage1.root
Finished2025-11-11 06:03:11
Saved logsjustin-logs:241132.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 156680 traces, 0 summary
Retagger: tagging trace set: gauss with 115616 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115616 traces tagged "gauss"
FrameSaver: q=4.1072e+06 n=2045796 tag=gauss
wclsFrameSaver: saving 156680 traces tagged "wiener"
FrameSaver: q=4.35848e+06 n=1800894 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 48704 XUs and 40346 XVs -> 698 XUVs
C:0 T:1 101473 XUs and 90405 XVs -> 7374 XUVs
C:0 T:2 87293 XUs and 163861 XVs -> 14435 XUVs
C:0 T:3 41129 XUs and 49974 XVs -> 398 XUVs
C:0 T:4 9570 XUs and 12601 XVs -> 86 XUVs
C:0 T:5 45856 XUs and 50297 XVs -> 7788 XUVs
C:0 T:6 113860 XUs and 121353 XVs -> 3672 XUVs
C:0 T:7 46145 XUs and 48543 XVs -> 417 XUVs
C:0 T:8 8215 XUs and 8950 XVs -> 132 XUVs
C:0 T:9 57610 XUs and 95987 XVs -> 9980 XUVs
C:0 T:10 114263 XUs and 90650 XVs -> 32092 XUVs
C:0 T:11 499 XUs and 501 XVs -> 347 XUVs
77419 XUVs total
13618 collection wire objects
73715 potential space points
Neighbour search...
15675429 tests to find 6946806 neighbours
Iterating with no regularization...
Begin: 7.38756e+07
0 5.96683e+07
1 5.87427e+07
2 5.85911e+07
3 5.85423e+07
Now with regularization...
Begin: 4.77645e+07
0 4.76683e+07
1 4.75976e+07
2 4.754e+07
3 4.74919e+07
4 4.74521e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410444709
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410444709
fLifetime: 30000 [us]
11-Nov-2025 13:01:40 CET  Closed output file "reco_stage1.root"
11-Nov-2025 13:01:40 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          318.779       459.434       649.093       429.175       99.9844        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00145337    0.00554087    0.00827901    0.00556449    0.00234161       10     
reco:ophit:OpHitFinder                             0.344675      0.375794      0.461326      0.366216      0.0331387       10     
reco:opflash:OpFlashFinder                         0.199501      0.236032       0.35175      0.216726      0.0435421       10     
reco:opslicer:OpSlicer                             0.633635      0.749765       1.05436      0.704536      0.123732        10     
reco:caldata:DataPrepModule                         6.26525       6.45279       6.9086        6.42425      0.173705        10     
reco:wclsdatasp:WireCellToolkit                     44.9185       58.8976       67.2152       61.8454       8.1058         10     
reco:gaushit:GausHitFinder                         0.990754       1.31992       1.72258       1.2747       0.222129        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000397996   0.000751778   0.000992234   0.000786231   0.000155317      10     
reco:reco3d:SpacePointSolver                        11.2256       20.0246       33.8378       20.0496       6.72111        10     
reco:hitpdune:DisambigFromSpacePoints              0.906334       1.56814       2.2276        1.53204      0.349933        10     
reco:pandora:StandardPandora                        16.7168       42.1197       148.639       33.5328       36.3235        10     
reco:pandoraWriter:StandardPandora                 0.360801      0.499352      0.610027      0.488854      0.0721838       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.01703       8.45264       19.0112       7.65866       4.19305        10     
reco:pandoraShower:LArPandoraShowerCreation         3.80982       8.45868       13.2774       8.26276       2.95898        10     
reco:pandoracalo:Calorimetry                        1.98731       4.4768        10.011        3.80981       2.24341        10     
reco:pandoracalonosce:Calorimetry                   1.81214       4.19188       9.43457       3.58643       2.09237        10     
reco:pandorapid:Chi2ParticleID                    0.00524249    0.00644245    0.00911791    0.00598712     0.0011064       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0423624     0.054069      0.0679301     0.0533332     0.0062666       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00498839    0.00621335    0.00914803     0.0059621    0.00116621       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.35858       8.0097        12.3555       7.56884       2.96074        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.11707       7.74416       11.5664       6.87908       2.85711        10     
reco:emtrkmichelid:EmTrackMichelId                  166.772       209.014       265.664       201.966       26.0903        10     
reco:crttag:SingleCRTMatchingProducer             0.00925695     0.0208909     0.0354087     0.0185373     0.0080216       10     
reco:crtreco:TwoCRTMatchingProducer                0.0080818     0.0171898     0.0278911     0.0169115    0.00589087       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.774598       1.03538       1.33053       1.05415      0.169668        10     
reco:pandora2Track:LArPandoraTrackCreation          10.3132       23.4044       38.8569       23.5368       8.36595        10     
reco:pandora2calo:Calorimetry                       4.9668        11.5903        22.24        10.9012       4.97894        10     
reco:pandora2calonosce:Calorimetry                  4.31904       11.0753       21.8917       10.949        4.80437        10     
reco:pandora2pid:Chi2ParticleID                    0.0108276     0.0134712     0.0183021     0.0129251    0.00212819       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0519365     0.0669428     0.0847475     0.065534     0.00865355       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0107129     0.0136172     0.0186098     0.0132178    0.00221691       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.66225       8.74606       15.4768       8.95971       3.31987        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.80727       9.25489       16.4306       9.53001       3.38377        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.47455       7.99021       16.8886       7.2629        3.56573        10     
[art]:TriggerResults:TriggerResultInserter        3.9911e-05    6.38893e-05   0.000144885   5.2608e-05    2.91147e-05      10     
end_path:out1:RootOutput                           8.103e-06    9.9942e-06    1.6625e-05     9.224e-06    2.28419e-06      10     
end_path:out1:RootOutput(write)                     2.53145       3.26256       4.19572       3.24506      0.435519        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4937.92 MB
  Peak resident set size usage (VmHWM): 4132.36 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9594248
-rw-r--r--. 1 dune009 dune 2278757725 Nov 11 11:43 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2166319289 Nov 11 11:41 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1950120547 Nov 11 13:01 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1720890407 Nov 11 09:07 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1636517653 Nov 11 08:53 g4_stage1.root
-rw-r--r--. 1 dune009 dune   71290470 Nov 11 12:53 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242691 Nov 11 13:01 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 11 13:01 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 11 13:01 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 11 13:01 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 11 11:43 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 09:07 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 13:01 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 11 08:47 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 11 13:01 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 11 11:49 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 11 11:43 debugprod.log
justIN time: 2026-02-10 07:55:37 UTC       justIN version: 01.06.00