justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270056.0@dunegpschedd02.fnal.gov

Jobsub ID270056.0@dunegpschedd02.fnal.gov
Workflow ID11741
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 08:41:54
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2026-01-12 12:40:59
From worker nodeHostnameccwcondor0367
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 08:42:47
Input filesmonte-carlo-011741-000001
JobscriptExit code0
Real time3h (14182s)
CPU time3h (13832s = 97%)
Max RSS bytes4773908480 (4552 MiB)
Outputting started2026-01-12 12:39:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11741/1/001/reco_stage1.root
Finished2026-01-12 12:40:59
Saved logsjustin-logs:270056.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

SignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 184042 traces, 0 summary
Retagger: tagging trace set: gauss with 135887 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 135887 traces tagged "gauss"
FrameSaver: q=4.72576e+06 n=2358909 tag=gauss
wclsFrameSaver: saving 184042 traces tagged "wiener"
FrameSaver: q=5.0423e+06 n=2077955 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 62365 XUs and 47859 XVs -> 1146 XUVs
C:0 T:1 190090 XUs and 125851 XVs -> 16163 XUVs
C:0 T:2 66406 XUs and 134178 XVs -> 7632 XUVs
C:0 T:3 28393 XUs and 47883 XVs -> 291 XUVs
C:0 T:4 20014 XUs and 20454 XVs -> 251 XUVs
C:0 T:5 138756 XUs and 101385 XVs -> 21876 XUVs
C:0 T:6 116153 XUs and 178707 XVs -> 12873 XUVs
C:0 T:7 40355 XUs and 62605 XVs -> 1226 XUVs
C:0 T:8 10554 XUs and 12581 XVs -> 169 XUVs
C:0 T:9 100995 XUs and 108459 XVs -> 10323 XUVs
C:0 T:10 39027 XUs and 41547 XVs -> 4689 XUVs
C:0 T:11 444 XUs and 356 XVs -> 99 XUVs
76738 XUVs total
17595 collection wire objects
68068 potential space points
Neighbour search...
12176028 tests to find 5280858 neighbours
Iterating with no regularization...
Begin: 9.47653e+07
0 7.62805e+07
1 7.512e+07
2 7.49415e+07
3 7.48855e+07
Now with regularization...
Begin: 6.08667e+07
0 6.08037e+07
1 6.07571e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411692785
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411692785
fLifetime: 30000 [us]
12-Jan-2026 13:39:07 CET  Closed output file "reco_stage1.root"
12-Jan-2026 13:39:07 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          381.957       471.723       560.613       452.287       59.3472        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00131058     0.0040294    0.00638102    0.00403402    0.00171576       10     
reco:ophit:OpHitFinder                             0.283857      0.348754      0.483267      0.339651      0.0526168       10     
reco:opflash:OpFlashFinder                         0.171795       0.25214       0.3995       0.247154      0.0555779       10     
reco:opslicer:OpSlicer                             0.556812      0.819904       1.21402      0.863756      0.182644        10     
reco:caldata:DataPrepModule                         4.79258       5.67282       5.97231       5.73163      0.308724        10     
reco:wclsdatasp:WireCellToolkit                     33.3013       35.9887       38.5235       36.3057       1.75791        10     
reco:gaushit:GausHitFinder                          1.37717       1.54634       1.79181       1.53749      0.146696        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000479598   0.000580457   0.000909712   0.000556964   0.000117403      10     
reco:reco3d:SpacePointSolver                        15.9118       21.9627       27.8561       20.662        4.18222        10     
reco:hitpdune:DisambigFromSpacePoints                1.649        2.26268       3.03653       2.25394      0.422524        10     
reco:pandora:StandardPandora                        30.9931       44.1524       62.6359       43.4106       9.57532        10     
reco:pandoraWriter:StandardPandora                 0.506473      0.586743      0.670435      0.594893      0.0499813       10     
reco:pandoraTrack:LArPandoraTrackCreation           5.52295       7.36433       8.97924       7.57511       1.15098        10     
reco:pandoraShower:LArPandoraShowerCreation         5.04235       8.59419       12.6249       8.56265       2.33223        10     
reco:pandoracalo:Calorimetry                        2.77384       4.25672       5.84767       3.85349       1.13474        10     
reco:pandoracalonosce:Calorimetry                   2.51216       3.90495       5.34787       3.96012      0.906908        10     
reco:pandorapid:Chi2ParticleID                    0.00590976    0.00697851    0.00922133    0.00676104    0.000934564      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0684269     0.0781621     0.107334      0.0766362     0.0107665       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00586186    0.00701981    0.00950757     0.0068032    0.00104021       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.04618       8.45708       12.3884       7.98377       2.41526        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.43164       8.19289       12.3905       7.60166       2.2498         10     
reco:emtrkmichelid:EmTrackMichelId                  204.976       242.331       282.302       238.496       24.2364        10     
reco:crttag:SingleCRTMatchingProducer              0.014964      0.020636      0.0359381     0.0193282    0.00554386       10     
reco:crtreco:TwoCRTMatchingProducer               0.00871036     0.0119989     0.0162973     0.0123033    0.00226723       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.92673       1.04653       1.15688       1.06793      0.0828788       10     
reco:pandora2Track:LArPandoraTrackCreation          17.3979       23.4337       32.2578       21.4979       5.24221        10     
reco:pandora2calo:Calorimetry                       7.27801       11.7275       16.4591       10.9427       2.83101        10     
reco:pandora2calonosce:Calorimetry                  6.62173       11.0609       15.845        10.9109       3.02087        10     
reco:pandora2pid:Chi2ParticleID                    0.0150907     0.0172808     0.0192333     0.0174826    0.00143136       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0876483     0.103999      0.136368      0.0985472     0.0148423       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0150352     0.0172615     0.0194035     0.0178999    0.00152816       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.96472       7.23289       10.3682       6.74469       1.64665        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.57159       9.18092       11.6289       9.08837       1.6284         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.35724       7.46647       10.0264       7.10979       1.44619        10     
[art]:TriggerResults:TriggerResultInserter        6.1342e-05    7.73891e-05   0.000150245   6.49875e-05   2.59019e-05      10     
end_path:out1:RootOutput                           1.335e-05    1.54972e-05   1.9559e-05    1.45815e-05   1.88725e-06      10     
end_path:out1:RootOutput(write)                     2.94602       3.43388       4.10807       3.40222      0.334425        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5603.29 MB
  Peak resident set size usage (VmHWM): 4773.91 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.53 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12342128
-rw-r--r-- 1 dune001 lbno 2815934644 Jan 12 12:18 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2668136175 Jan 12 12:15 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2567976320 Jan 12 13:39 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2296731206 Jan 12 10:06 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2198283160 Jan 12 09:50 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   90660040 Jan 12 13:32 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242884 Jan 12 13:39 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 12 13:39 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 12 13:39 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 12 13:39 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 12 12:18 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 10:06 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 12 13:39 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 12 09:42 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 12 13:39 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:24 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 12 12:18 debugprod.log
justIN time: 2026-02-06 10:21:39 UTC       justIN version: 01.06.00