justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265701.0@dunegpschedd02.fnal.gov

Jobsub ID265701.0@dunegpschedd02.fnal.gov
Workflow ID11491
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-03 20:26:11
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce02
Last heartbeat2026-01-03 23:32:31
From worker nodeHostnameccwcondor0376
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-03 20:26:52
Input filesmonte-carlo-011491-000001
JobscriptExit code0
Real time3h (11052s)
CPU time2h (10735s = 97%)
Max RSS bytes4037931008 (3850 MiB)
Outputting started2026-01-03 23:31:04
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11491/1/001/reco_stage1.root
Finished2026-01-03 23:32:31
Saved logsjustin-logs:265701.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ice:prepare:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112860 traces, 0 summary
Retagger: tagging trace set: gauss with 86021 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 86021 traces tagged "gauss"
FrameSaver: q=3.06287e+06 n=1516849 tag=gauss
wclsFrameSaver: saving 112860 traces tagged "wiener"
FrameSaver: q=3.22233e+06 n=1343090 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 23273 XUs and 21357 XVs -> 256 XUVs
C:0 T:1 39713 XUs and 34308 XVs -> 4256 XUVs
C:0 T:2 85262 XUs and 114697 XVs -> 8726 XUVs
C:0 T:3 34326 XUs and 41667 XVs -> 320 XUVs
C:0 T:4 16848 XUs and 11191 XVs -> 153 XUVs
C:0 T:5 60244 XUs and 33780 XVs -> 6629 XUVs
C:0 T:6 69713 XUs and 98952 XVs -> 7184 XUVs
C:0 T:7 26037 XUs and 37283 XVs -> 258 XUVs
C:0 T:8 8327 XUs and 5723 XVs -> 99 XUVs
C:0 T:9 46111 XUs and 39967 XVs -> 5500 XUVs
C:0 T:10 20507 XUs and 11101 XVs -> 2964 XUVs
C:0 T:11 82 XUs and 55 XVs -> 4 XUVs
36349 XUVs total
9168 collection wire objects
33556 potential space points
Neighbour search...
3563126 tests to find 1861864 neighbours
Iterating with no regularization...
Begin: 6.18502e+07
0 5.488e+07
1 5.43035e+07
2 5.42015e+07
3 5.41685e+07
Now with regularization...
Begin: 4.54014e+07
0 4.53664e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411520606
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411520606
fLifetime: 30000 [us]
04-Jan-2026 00:31:02 CET  Closed output file "reco_stage1.root"
04-Jan-2026 00:31:02 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          249.505       334.517       469.28        317.529       69.8566        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00141502    0.00449789    0.00865227    0.00468779    0.00191204       10     
reco:ophit:OpHitFinder                              0.23121      0.273607      0.329782      0.275039      0.0332498       10     
reco:opflash:OpFlashFinder                         0.105936      0.141286       0.17324      0.148193      0.0238758       10     
reco:opslicer:OpSlicer                              0.4664       0.608258       0.73167      0.580193      0.096774        10     
reco:caldata:DataPrepModule                         5.17149       5.94888       6.67327       5.88623      0.413828        10     
reco:wclsdatasp:WireCellToolkit                     33.4694       36.6065       40.5685       36.9372       1.88898        10     
reco:gaushit:GausHitFinder                         0.818256       1.12628       1.53164       1.09794      0.209098        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000390834   0.000483785   0.000809951   0.000445192   0.000118234      10     
reco:reco3d:SpacePointSolver                        5.81424       13.3319       24.1806       11.7785       6.12015        10     
reco:hitpdune:DisambigFromSpacePoints               0.78692       1.36242       2.0703        1.2592       0.442872        10     
reco:pandora:StandardPandora                        15.3342       24.549        39.4222       20.3211       8.73326        10     
reco:pandoraWriter:StandardPandora                 0.357236      0.471457      0.645751       0.44323      0.086607        10     
reco:pandoraTrack:LArPandoraTrackCreation           2.85471       4.51108       7.32569       4.30409       1.27742        10     
reco:pandoraShower:LArPandoraShowerCreation         1.5773        4.86181       8.35139       4.7921        1.98244        10     
reco:pandoracalo:Calorimetry                        1.41037       2.36973       4.09349       2.2094       0.742983        10     
reco:pandoracalonosce:Calorimetry                   1.56207       2.27882       4.22167       2.02659      0.740738        10     
reco:pandorapid:Chi2ParticleID                    0.00478931    0.00589562    0.00871932    0.00579611    0.00104514       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0448093     0.063777      0.0873256     0.0616211     0.011854        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0046603    0.00555893    0.00642858    0.00554596    0.000507724      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.96476       4.54761       9.0453        4.24043       2.12502        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.02921       4.68623       9.98207       3.80547       2.42151        10     
reco:emtrkmichelid:EmTrackMichelId                  139.257       181.838       240.234       176.325       30.8259        10     
reco:crttag:SingleCRTMatchingProducer              0.011483      0.0171852     0.0239381     0.0164401     0.0035987       10     
reco:crtreco:TwoCRTMatchingProducer               0.00680782    0.00995158     0.014941     0.00959182    0.00232373       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.654324      0.881873       1.14349      0.881123      0.153374        10     
reco:pandora2Track:LArPandoraTrackCreation          5.74891       13.2589       21.9016       12.0328       4.88267        10     
reco:pandora2calo:Calorimetry                       4.02438       6.42095       10.7813       5.98082       2.22143        10     
reco:pandora2calonosce:Calorimetry                  3.75874       6.24741       11.2555       6.1223        2.22613        10     
reco:pandora2pid:Chi2ParticleID                    0.010779      0.0128738     0.0169077     0.0123462    0.00172132       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0600865     0.0794421     0.107662      0.0720197     0.0161996       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0113393     0.0126649     0.0157399     0.0122146    0.00134922       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.5779        4.67688       8.34801       4.7136        1.69522        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.64749       5.42657       10.8002       5.31162       2.35033        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.50424       5.08005       9.67348       4.72504       2.1195         10     
[art]:TriggerResults:TriggerResultInserter         6.559e-05    9.62995e-05   0.000183359   8.0653e-05    3.60228e-05      10     
end_path:out1:RootOutput                          1.1047e-05    1.43948e-05   2.0211e-05    1.2899e-05    3.14811e-06      10     
end_path:out1:RootOutput(write)                     1.84776       2.60808       3.50011       2.45916      0.487219        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4860.81 MB
  Peak resident set size usage (VmHWM): 4037.93 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.64 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9535928
-rw-r--r-- 1 dune001 lbno 2261094759 Jan  3 23:33 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2155836380 Jan  3 23:31 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 1915192325 Jan  4 00:31 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 1712462732 Jan  3 21:44 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 1652588689 Jan  3 21:33 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   66996180 Jan  4 00:27 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242808 Jan  4 00:31 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan  4 00:31 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan  4 00:31 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan  4 00:31 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan  3 23:33 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  3 21:44 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan  4 00:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan  3 21:26 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan  4 00:31 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan  3 23:40 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan  3 23:33 debugprod.log
justIN time: 2026-02-06 22:18:13 UTC       justIN version: 01.06.00