justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 275407.0@dunegpschedd02.fnal.gov

Jobsub ID275407.0@dunegpschedd02.fnal.gov
Workflow ID12076
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-18 15:13:52
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2026-01-18 20:02:26
From worker nodeHostnameccwcondor0633
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-18 15:15:31
Input filesmonte-carlo-012076-000001
JobscriptExit code0
Real time4h (17098s)
CPU time4h (16896s = 98%)
Max RSS bytes4542312448 (4331 MiB)
Outputting started2026-01-18 20:00:30
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/12076/1/001/reco_stage1.root
Finished2026-01-18 20:02:26
Saved logsjustin-logs:275407.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 172535 traces, 0 summary
Retagger: tagging trace set: gauss with 129529 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 129529 traces tagged "gauss"
FrameSaver: q=4.52023e+06 n=2312868 tag=gauss
wclsFrameSaver: saving 172535 traces tagged "wiener"
FrameSaver: q=4.78474e+06 n=2044297 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 43621 XUs and 43970 XVs -> 684 XUVs
C:0 T:1 85946 XUs and 80211 XVs -> 8074 XUVs
C:0 T:2 111907 XUs and 166555 XVs -> 7641 XUVs
C:0 T:3 38956 XUs and 54796 XVs -> 415 XUVs
C:0 T:4 16557 XUs and 12625 XVs -> 239 XUVs
C:0 T:5 66082 XUs and 45707 XVs -> 8309 XUVs
C:0 T:6 118543 XUs and 154672 XVs -> 5662 XUVs
C:0 T:7 45794 XUs and 56757 XVs -> 465 XUVs
C:0 T:8 13334 XUs and 13204 XVs -> 160 XUVs
C:0 T:9 112986 XUs and 137929 XVs -> 8431 XUVs
C:0 T:10 46208 XUs and 40165 XVs -> 6177 XUVs
C:0 T:11 571 XUs and 650 XVs -> 72 XUVs
46329 XUVs total
16444 collection wire objects
42204 potential space points
Neighbour search...
3084844 tests to find 1632498 neighbours
Iterating with no regularization...
Begin: 9.17889e+07
0 7.56726e+07
1 7.4413e+07
2 7.42423e+07
3 7.41958e+07
Now with regularization...
Begin: 6.12403e+07
0 6.11745e+07
1 6.11439e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411818965
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411818965
fLifetime: 30000 [us]
18-Jan-2026 21:00:28 CET  Closed output file "reco_stage1.root"
18-Jan-2026 21:00:28 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          382.822       603.649       819.79        590.757       126.114        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00142454    0.00533466     0.009178      0.0049184    0.00255066       10     
reco:ophit:OpHitFinder                             0.377356      0.434487       0.48934      0.435983      0.0370858       10     
reco:opflash:OpFlashFinder                         0.217551      0.271628      0.316936      0.284865      0.0354854       10     
reco:opslicer:OpSlicer                             0.870229       1.06051       1.34232       1.08391      0.137936        10     
reco:caldata:DataPrepModule                          6.487        7.08251       7.46907       7.09054      0.265062        10     
reco:wclsdatasp:WireCellToolkit                     38.4857       41.5256       43.4792       42.3261       1.66976        10     
reco:gaushit:GausHitFinder                          1.32389       1.90128       2.47999       1.8785       0.377447        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00059658    0.000793441   0.00098783    0.000798421   0.000103175      10     
reco:reco3d:SpacePointSolver                        20.2729       48.2696       143.066       36.7426       34.4237        10     
reco:hitpdune:DisambigFromSpacePoints               1.9036        3.06396       4.75914       3.01889       0.81484        10     
reco:pandora:StandardPandora                        27.0767       67.7383       117.196       68.1723       26.3231        10     
reco:pandoraWriter:StandardPandora                 0.516758      0.708202      0.857589       0.72873      0.114704        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.60634       9.48519       12.6295       9.74871       2.05939        10     
reco:pandoraShower:LArPandoraShowerCreation         7.60281       12.1146       17.9562       11.9859       3.02372        10     
reco:pandoracalo:Calorimetry                        3.23422       5.46243        7.85         5.4177        1.15414        10     
reco:pandoracalonosce:Calorimetry                   3.17982       5.28824       7.14328       5.37301      0.940237        10     
reco:pandorapid:Chi2ParticleID                    0.00913756     0.0101313     0.0113299     0.0100346    0.000718621      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0636733     0.0911822      0.11208      0.0911668     0.0153577       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00744615    0.00963058     0.0109243    0.00950988    0.00103161       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.42291       12.9653       20.7375        13.14        3.56896        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.76251       12.3696       19.1919       12.2786       3.14324        10     
reco:emtrkmichelid:EmTrackMichelId                  196.865       268.526       334.467       269.604       42.4161        10     
reco:crttag:SingleCRTMatchingProducer               0.01972      0.0297948     0.0470712     0.0280226    0.00704984       10     
reco:crtreco:TwoCRTMatchingProducer                0.0125179     0.0196053     0.0278462     0.0201695     0.0039997       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.90927       1.26029       1.48103       1.29831      0.175351        10     
reco:pandora2Track:LArPandoraTrackCreation          15.0897       31.3733       43.7669        31.8         8.46632        10     
reco:pandora2calo:Calorimetry                       7.39299       16.7431       24.6057       16.7801       4.96346        10     
reco:pandora2calonosce:Calorimetry                  7.64387       15.3813       20.9809       14.9892       4.09612        10     
reco:pandora2pid:Chi2ParticleID                    0.0183746     0.0256337     0.0328589     0.0255355    0.00406522       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0854868     0.119236      0.146069       0.12125      0.0194673       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0182459     0.0248318     0.0309901     0.0247863    0.00351905       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.34558       10.4864       14.1804       10.2981       2.51519        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.89657       13.362        18.8646       13.5229       3.01196        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.6809        12.0442       15.0436       12.488        2.89408        10     
[art]:TriggerResults:TriggerResultInserter        7.1448e-05    8.83696e-05   0.000189827   7.6371e-05    3.43746e-05      10     
end_path:out1:RootOutput                          1.2292e-05    1.98001e-05   6.6488e-05    1.49115e-05   1.56126e-05      10     
end_path:out1:RootOutput(write)                     3.08411       4.17053       5.22411       4.13097      0.661959        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5373.61 MB
  Peak resident set size usage (VmHWM): 4542.31 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.50 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11695444
-rw-r--r-- 1 dune001 lbno 2678910502 Jan 18 19:17 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2533010961 Jan 18 19:14 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2437158515 Jan 18 21:00 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2168901226 Jan 18 16:41 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2065809376 Jan 18 16:23 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   91732430 Jan 18 20:52 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     243074 Jan 18 21:00 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Jan 18 21:00 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Jan 18 21:00 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Jan 18 21:00 time.db
-rw-r--r-- 1 dune001 lbno        519 Jan 18 19:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 16:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Jan 18 21:00 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Jan 18 16:15 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Jan 18 21:00 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Jan 18 19:23 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Jan 18 19:17 debugprod.log
justIN time: 2026-02-05 18:26:26 UTC       justIN version: 01.06.00