justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268869.0@dunegpschedd02.fnal.gov

Jobsub ID268869.0@dunegpschedd02.fnal.gov
Workflow ID11729
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 13:08:49
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2026-01-11 18:04:37
From worker nodeHostnamewn-la-22.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 13:09:33
Input filesmonte-carlo-011729-000001
JobscriptExit code0
Real time4h (17612s)
CPU time4h (17504s = 99%)
Max RSS bytes4600860672 (4387 MiB)
Outputting started2026-01-11 18:03:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11729/1/001/reco_stage1.root
Finished2026-01-11 18:04:37
Saved logsjustin-logs:268869.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

olBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 129259 traces, 0 summary
Retagger: tagging trace set: gauss with 98846 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 98846 traces tagged "gauss"
FrameSaver: q=3.48979e+06 n=1774310 tag=gauss
wclsFrameSaver: saving 129259 traces tagged "wiener"
FrameSaver: q=3.69222e+06 n=1570280 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29532 XUs and 32302 XVs -> 380 XUVs
C:0 T:1 51930 XUs and 55586 XVs -> 3564 XUVs
C:0 T:2 77661 XUs and 109237 XVs -> 4141 XUVs
C:0 T:3 41169 XUs and 46837 XVs -> 281 XUVs
C:0 T:4 12361 XUs and 14065 XVs -> 133 XUVs
C:0 T:5 47455 XUs and 52768 XVs -> 4010 XUVs
C:0 T:6 186254 XUs and 142305 XVs -> 13791 XUVs
C:0 T:7 53226 XUs and 45566 XVs -> 640 XUVs
C:0 T:8 6459 XUs and 7197 XVs -> 55 XUVs
C:0 T:9 37118 XUs and 52964 XVs -> 2569 XUVs
C:0 T:10 54336 XUs and 35949 XVs -> 5243 XUVs
C:0 T:11 41 XUs and 19 XVs -> 10 XUVs
34817 XUVs total
12480 collection wire objects
31719 potential space points
Neighbour search...
2617351 tests to find 1229030 neighbours
Iterating with no regularization...
Begin: 8.17589e+07
0 6.61178e+07
1 6.49431e+07
2 6.47318e+07
3 6.46615e+07
4 6.46298e+07
Now with regularization...
Begin: 5.35795e+07
0 5.35053e+07
1 5.34673e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411676371
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411676371
fLifetime: 30000 [us]
11-Jan-2026 19:03:03 CET  Closed output file "reco_stage1.root"
11-Jan-2026 19:03:03 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          342.747       517.92        743.505       472.844       112.546        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00116589    0.00655259     0.0121359    0.00680244    0.00345749       10     
reco:ophit:OpHitFinder                             0.393613      0.452336      0.514436      0.448266      0.0368917       10     
reco:opflash:OpFlashFinder                         0.269143      0.338556      0.424625      0.321345      0.0479983       10     
reco:opslicer:OpSlicer                             0.711618      0.898608       1.21387      0.868683      0.141229        10     
reco:caldata:DataPrepModule                         6.2897        6.46854       6.91715       6.41239      0.192497        10     
reco:wclsdatasp:WireCellToolkit                     43.3639       58.4195       64.8978       61.139        7.59055        10     
reco:gaushit:GausHitFinder                          1.08732       1.52362       2.20722       1.42622      0.324891        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000371285   0.000623049   0.000932351   0.000523701   0.000207206      10     
reco:reco3d:SpacePointSolver                         12.68        22.3981       34.017        19.8387       6.30391        10     
reco:hitpdune:DisambigFromSpacePoints               1.21504       2.05958       3.46983       1.76712      0.739154        10     
reco:pandora:StandardPandora                        20.8926       50.3969       117.435       41.1744       27.3187        10     
reco:pandoraWriter:StandardPandora                  0.39786      0.558352       0.77511      0.528387      0.114897        10     
reco:pandoraTrack:LArPandoraTrackCreation            5.114        10.1047       22.3287       8.45487       4.80668        10     
reco:pandoraShower:LArPandoraShowerCreation         3.7435        11.0481       20.3757       10.3359       4.5211         10     
reco:pandoracalo:Calorimetry                        2.55455       5.26553       9.8687        4.76312       1.89344        10     
reco:pandoracalonosce:Calorimetry                   2.54632       5.03738       9.25651       4.58569       1.86911        10     
reco:pandorapid:Chi2ParticleID                    0.00591409    0.00722303    0.00914792    0.00694671    0.000972252      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0468697     0.058891      0.0715538     0.0589084    0.00825345       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00579282    0.00700233    0.00890537    0.00683934    0.000859455      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.02075       10.6557       17.6026       9.20541       4.36671        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.3684        9.6939         16.06        8.01951       3.88001        10     
reco:emtrkmichelid:EmTrackMichelId                  170.256       228.396       291.254       217.928       35.1953        10     
reco:crttag:SingleCRTMatchingProducer              0.0132655     0.0211364     0.038464      0.0199313    0.00723681       10     
reco:crtreco:TwoCRTMatchingProducer                0.0118397     0.0207551     0.0377898     0.0188512    0.00842345       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.927752       1.10424       1.42319       1.08004      0.133837        10     
reco:pandora2Track:LArPandoraTrackCreation          15.8699       29.6003       46.9328       27.024        9.20569        10     
reco:pandora2calo:Calorimetry                       6.42096       14.8998       26.0293       13.9359       5.29765        10     
reco:pandora2calonosce:Calorimetry                  5.4732        13.0075       19.7575       12.3136       3.78487        10     
reco:pandora2pid:Chi2ParticleID                    0.0119789     0.0158042     0.0212621     0.0149085     0.0025719       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.057509      0.0756754     0.0976338     0.0735784     0.0127698       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0123571     0.0157843     0.020737      0.0150035    0.00244668       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.35259       9.92126       14.0269       9.76428       2.61736        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.55586       11.3419       17.6557       10.3035       3.24622        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.17478       10.2298       14.3184       10.6801       3.11429        10     
[art]:TriggerResults:TriggerResultInserter        4.1903e-05    6.18022e-05   0.000133139   5.7996e-05    2.47555e-05      10     
end_path:out1:RootOutput                           7.721e-06    8.9683e-06    1.4361e-05     8.307e-06    1.86455e-06      10     
end_path:out1:RootOutput(write)                     2.68978       3.60263       4.51795       3.40197      0.552554        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5399.57 MB
  Peak resident set size usage (VmHWM): 4600.86 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.03 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11100748
-rw-r--r--. 1 dune009 dune 2577895371 Jan 11 17:34 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2443104623 Jan 11 17:31 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2284611860 Jan 11 19:03 reco_stage1.root
-rw-r--r--. 1 dune009 dune 2032211646 Jan 11 14:34 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1948777462 Jan 11 14:17 g4_stage1.root
-rw-r--r--. 1 dune009 dune   79884880 Jan 11 18:57 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243201 Jan 11 19:03 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Jan 11 19:03 mem.db
-rw-r--r--. 1 dune009 dune      62376 Jan 11 19:03 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Jan 11 19:03 time.db
-rw-r--r--. 1 dune009 dune        519 Jan 11 17:34 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 11 14:34 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Jan 11 19:03 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Jan 11 14:09 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Jan 11 19:03 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Jan 11 17:42 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Jan 11 17:34 debugprod.log
justIN time: 2026-02-06 10:21:22 UTC       justIN version: 01.06.00