justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 246900.1@dunegpschedd01.fnal.gov

Jobsub ID246900.1@dunegpschedd01.fnal.gov
Workflow ID9917
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-11 01:43:40
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-11-11 05:15:02
From worker nodeHostnamewn-da-03.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-11 01:45:16
Input filesmonte-carlo-009917-000001
JobscriptExit code0
Real time3h (12500s)
CPU time3h (12390s = 99%)
Max RSS bytes4295036928 (4096 MiB)
Outputting started2025-11-11 05:13:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09917/1/001/reco_stage1.root
Finished2025-11-11 05:15:02
Saved logsjustin-logs:246900.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 77211 traces, 0 summary
Retagger: tagging trace set: gauss with 61088 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 61088 traces tagged "gauss"
FrameSaver: q=2.14156e+06 n=1078469 tag=gauss
wclsFrameSaver: saving 77211 traces tagged "wiener"
FrameSaver: q=2.26797e+06 n=968329 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 16661 XUs and 20216 XVs -> 181 XUVs
C:0 T:1 46844 XUs and 37681 XVs -> 8698 XUVs
C:0 T:2 28505 XUs and 46882 XVs -> 1692 XUVs
C:0 T:3 17416 XUs and 22290 XVs -> 35 XUVs
C:0 T:4 4853 XUs and 4346 XVs -> 25 XUVs
C:0 T:5 12754 XUs and 7346 XVs -> 1663 XUVs
C:0 T:6 85165 XUs and 85063 XVs -> 10258 XUVs
C:0 T:7 31730 XUs and 34521 XVs -> 467 XUVs
C:0 T:8 6519 XUs and 5582 XVs -> 66 XUVs
C:0 T:9 48718 XUs and 45284 XVs -> 5684 XUVs
C:0 T:10 17354 XUs and 8831 XVs -> 1532 XUVs
C:0 T:11 10 XUs and 7 XVs -> 1 XUVs
30302 XUVs total
6055 collection wire objects
28166 potential space points
Neighbour search...
5493730 tests to find 2403162 neighbours
Iterating with no regularization...
Begin: 4.39283e+07
0 3.84674e+07
1 3.80509e+07
2 3.79761e+07
3 3.795e+07
Now with regularization...
Begin: 3.15266e+07
0 3.14876e+07
1 3.14609e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410444763
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410444763
fLifetime: 30000 [us]
11-Nov-2025 12:13:35 CET  Closed output file "reco_stage1.root"
11-Nov-2025 12:13:35 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          218.542       375.894       571.666       376.69        87.1454        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000861974   0.00460514    0.00792541    0.00455033    0.00248369       10     
reco:ophit:OpHitFinder                             0.228457      0.276794      0.308789      0.284778      0.0248066       10     
reco:opflash:OpFlashFinder                          0.12446      0.171373      0.202521      0.177553      0.024457        10     
reco:opslicer:OpSlicer                              0.51924      0.697913      0.897554      0.705258      0.114519        10     
reco:caldata:DataPrepModule                         4.69465       5.02083       5.31025       5.02078       0.16816        10     
reco:wclsdatasp:WireCellToolkit                     34.5026       54.8595       72.4044       57.7266       12.7248        10     
reco:gaushit:GausHitFinder                         0.722275       1.23241       1.5318        1.32817      0.227008        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000344534   0.000563177   0.000737188   0.000563227   0.000113925      10     
reco:reco3d:SpacePointSolver                        6.78811       11.367        18.1489       10.0809       3.46898        10     
reco:hitpdune:DisambigFromSpacePoints              0.597122       1.45122       2.35306       1.40459      0.459463        10     
reco:pandora:StandardPandora                        9.92498       32.9875       77.4475       30.7356       17.3499        10     
reco:pandoraWriter:StandardPandora                 0.230914      0.405355      0.592814       0.39954      0.0898321       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.67678       3.97307       8.89322       3.59714       1.80873        10     
reco:pandoraShower:LArPandoraShowerCreation         1.3918        4.53959       9.07518       4.38692       1.97171        10     
reco:pandoracalo:Calorimetry                       0.902937       2.10514       4.50443       1.9526       0.888661        10     
reco:pandoracalonosce:Calorimetry                  0.811964       1.89635       4.11057       1.74387      0.806552        10     
reco:pandorapid:Chi2ParticleID                    0.00430972    0.00586665     0.0082889    0.00561437    0.00100565       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0379318     0.0477844     0.0576037     0.0468656    0.00516006       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00419219    0.00553284    0.00809855    0.00532347    0.000948569      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.32369       4.36415       8.7715        4.24041       1.89645        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.2652        4.28776       8.62325       4.19643       1.87744        10     
reco:emtrkmichelid:EmTrackMichelId                   124.6        203.601       281.68        203.737       38.5118        10     
reco:crttag:SingleCRTMatchingProducer             0.00581002     0.0100589     0.015595     0.00994443    0.00270385       10     
reco:crtreco:TwoCRTMatchingProducer               0.00490981    0.00873435     0.0160242    0.00856585     0.0027952       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.540689      0.784171       0.97079      0.778774      0.124817        10     
reco:pandora2Track:LArPandoraTrackCreation          4.81153       12.6143       26.0682       12.3027       5.29988        10     
reco:pandora2calo:Calorimetry                       2.34667       5.99133       11.7338       5.95913       2.37618        10     
reco:pandora2calonosce:Calorimetry                  1.92739       5.86346        12.5         5.46121       2.68459        10     
reco:pandora2pid:Chi2ParticleID                   0.00958816     0.0147794     0.0217543     0.014221      0.0030909       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0391089     0.0604982     0.0839457     0.0605074     0.0113367       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00941766     0.0146763     0.021438       0.01415      0.0030437       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.73189        4.779        9.85645       4.43096       2.01027        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.82217       5.07765       9.89152       4.85335       2.00265        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.66812       4.40104       7.78708       4.21214       1.51686        10     
[art]:TriggerResults:TriggerResultInserter        3.0899e-05    4.03826e-05   0.000104739   3.35285e-05   2.15017e-05      10     
end_path:out1:RootOutput                           6.303e-06    7.1889e-06    1.1442e-05     6.698e-06    1.4448e-06       10     
end_path:out1:RootOutput(write)                     1.74958       2.85542       4.00244       2.8602       0.552586        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5098.77 MB
  Peak resident set size usage (VmHWM): 4295.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.14 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10532612
-rw-r--r--. 1 dune009 dune 2468794752 Nov 11 11:08 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2351914215 Nov 11 11:06 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2145113806 Nov 11 12:13 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1908925206 Nov 11 09:03 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1837517878 Nov 11 08:51 g4_stage1.root
-rw-r--r--. 1 dune009 dune   72513890 Nov 11 12:11 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242957 Nov 11 12:13 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 11 12:13 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 11 12:13 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 11 12:13 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 11 11:08 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 09:03 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 11 12:13 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 11 08:45 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 11 12:13 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 11 11:14 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 11 11:08 debugprod.log
justIN time: 2026-02-10 07:57:36 UTC       justIN version: 01.06.00