justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 302384.1@dunegpschedd01.fnal.gov

Jobsub ID302384.1@dunegpschedd01.fnal.gov
Workflow ID12572
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-30 13:59:12
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-30 16:37:37
From worker nodeHostnametd808.pic.es
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-30 14:00:29
Input filesmonte-carlo-012572-000001
JobscriptExit code0
Real time2h (9356s)
CPU time2h (8827s = 94%)
Max RSS bytes3969044480 (3785 MiB)
Outputting started2026-01-30 16:36:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12572/1/001/reco_stage1.root
Finished2026-01-30 16:37:37
Saved logsjustin-logs:302384.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

re:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 132527 traces, 0 summary
Retagger: tagging trace set: gauss with 100171 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100171 traces tagged "gauss"
FrameSaver: q=3.58355e+06 n=1774453 tag=gauss
wclsFrameSaver: saving 132527 traces tagged "wiener"
FrameSaver: q=3.78202e+06 n=1557399 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23166 XUs and 30597 XVs -> 770 XUVs
C:0 T:1 70442 XUs and 80024 XVs -> 12555 XUVs
C:0 T:2 52410 XUs and 65778 XVs -> 4102 XUVs
C:0 T:3 21957 XUs and 26730 XVs -> 136 XUVs
C:0 T:4 11746 XUs and 10262 XVs -> 107 XUVs
C:0 T:5 39342 XUs and 32413 XVs -> 2616 XUVs
C:0 T:6 174414 XUs and 175866 XVs -> 8867 XUVs
C:0 T:7 61386 XUs and 62558 XVs -> 747 XUVs
C:0 T:8 4223 XUs and 6594 XVs -> 37 XUVs
C:0 T:9 28039 XUs and 54990 XVs -> 3381 XUVs
C:0 T:10 57743 XUs and 39411 XVs -> 5094 XUVs
C:0 T:11 290 XUs and 192 XVs -> 25 XUVs
38437 XUVs total
12507 collection wire objects
34960 potential space points
Neighbour search...
3491164 tests to find 1690070 neighbours
Iterating with no regularization...
Begin: 7.73771e+07
0 6.64034e+07
1 6.55338e+07
2 6.5412e+07
3 6.53763e+07
Now with regularization...
Begin: 5.48113e+07
0 5.47718e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 412059313
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 412059313
fLifetime: 30000 [us]
30-Jan-2026 17:36:25 CET  Closed output file "reco_stage1.root"
30-Jan-2026 17:36:25 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          204.794       282.556       396.589       289.177       52.0712        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00105836    0.00269794    0.00519574    0.00250051    0.00110042       10     
reco:ophit:OpHitFinder                             0.189332      0.234582      0.310442      0.234683      0.0322142       10     
reco:opflash:OpFlashFinder                         0.0900819     0.126354      0.142742      0.131903      0.0161976       10     
reco:opslicer:OpSlicer                             0.408201      0.531917      0.756342      0.505601      0.099721        10     
reco:caldata:DataPrepModule                         4.19497       4.57546       5.55834       4.42342      0.415102        10     
reco:wclsdatasp:WireCellToolkit                     26.4212       31.617        41.8142       29.8704       4.16855        10     
reco:gaushit:GausHitFinder                         0.734402      0.975311       1.26227       1.00414      0.146672        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000295883   0.000505054   0.000805414   0.00047323    0.000144526      10     
reco:reco3d:SpacePointSolver                        4.70681       9.95859       14.7824       10.2036       3.0297         10     
reco:hitpdune:DisambigFromSpacePoints              0.743883       1.06197       1.42887       1.09965      0.216811        10     
reco:pandora:StandardPandora                        11.9124       20.5655       32.4288       21.4887       6.69299        10     
reco:pandoraWriter:StandardPandora                 0.251156      0.345825      0.473453      0.357289      0.0641005       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.43846       2.68793       5.97288       2.35935       1.19226        10     
reco:pandoraShower:LArPandoraShowerCreation         1.2952        3.00695       8.30285       2.50328       1.88276        10     
reco:pandoracalo:Calorimetry                       0.781466       1.33375       2.30246       1.26409      0.400168        10     
reco:pandoracalonosce:Calorimetry                  0.708333       1.24537       2.4411        1.13244      0.464241        10     
reco:pandorapid:Chi2ParticleID                    0.00338354    0.00448629    0.00602791    0.00445589    0.000748563      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0326263     0.0437238     0.0634001      0.04104     0.00987648       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00342743    0.00409701    0.00488845    0.00389017    0.000441907      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.28486       2.97538       8.49283       2.56207       1.96222        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.26609       2.6192        5.42714       2.4433        1.13042        10     
reco:emtrkmichelid:EmTrackMichelId                  129.017       167.661       217.441       170.199       25.1316        10     
reco:crttag:SingleCRTMatchingProducer             0.00811275     0.0117211     0.0173358     0.0110009    0.00286723       10     
reco:crtreco:TwoCRTMatchingProducer               0.00452697    0.00580017    0.00702908    0.00606803    0.000876354      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.487385      0.662129      0.772878      0.710395       0.10529        10     
reco:pandora2Track:LArPandoraTrackCreation          4.23618       8.65754       17.0496       8.35586       3.61303        10     
reco:pandora2calo:Calorimetry                       1.97666       4.23609       8.6625        4.04746       1.97857        10     
reco:pandora2calonosce:Calorimetry                  1.82759       3.89493       7.89401       3.64657       1.81927        10     
reco:pandora2pid:Chi2ParticleID                    0.0079889     0.0117464     0.0156077     0.0126448    0.00232272       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0409184     0.0525136     0.0719668     0.0520114     0.0092627       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0078039     0.0117478     0.0171047     0.0112747    0.00284895       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.6734        3.69669       6.87364       3.65454       1.51024        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.73586       3.69358       6.78756       3.49989       1.60234        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.62271       3.46028       6.6444        3.34727       1.49303        10     
[art]:TriggerResults:TriggerResultInserter         2.361e-05    3.77434e-05   0.000111682    2.769e-05    2.53976e-05      10     
end_path:out1:RootOutput                           9.13e-06     1.07961e-05    1.467e-05     1.068e-05    1.49646e-06      10     
end_path:out1:RootOutput(write)                     1.8529        2.47537       3.1695        2.43929      0.396488        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4806.25 MB
  Peak resident set size usage (VmHWM): 3969.04 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.80 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9095204
-rw-r--r-- 1 dune001 dune 2178217475 Jan 30 16:47 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2071254048 Jan 30 16:45 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 1826442701 Jan 30 17:36 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1614709881 Jan 30 15:18 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1556547918 Jan 30 15:09 g4_stage1.root
-rw-r--r-- 1 dune001 dune   65682560 Jan 30 17:31 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     242885 Jan 30 17:36 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 30 17:36 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 30 17:36 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 30 17:36 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 30 16:47 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 30 15:18 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 30 17:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 30 15:00 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 30 17:36 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 30 16:52 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 30 16:47 debugprod.log
justIN time: 2026-02-04 13:58:52 UTC       justIN version: 01.06.00