justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 234884.0@dunegpschedd02.fnal.gov

Jobsub ID234884.0@dunegpschedd02.fnal.gov
Workflow ID9450
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-29 17:10:41
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2025-10-29 21:40:05
From worker nodeHostnamewn-lb-12.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-29 17:12:04
Input filesmonte-carlo-009450-000001
JobscriptExit code0
Real time4h (15925s)
CPU time4h (15740s = 98%)
Max RSS bytes4151812096 (3959 MiB)
Outputting started2025-10-29 21:37:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09450/1/001/reco_stage1.root
Finished2025-10-29 21:40:05
Saved logsjustin-logs:234884.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 106001 traces, 0 summary
Retagger: tagging trace set: gauss with 80615 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80615 traces tagged "gauss"
FrameSaver: q=2.75807e+06 n=1422712 tag=gauss
wclsFrameSaver: saving 106001 traces tagged "wiener"
FrameSaver: q=2.92694e+06 n=1264716 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32106 XUs and 23764 XVs -> 529 XUVs
C:0 T:1 49850 XUs and 40527 XVs -> 3507 XUVs
C:0 T:2 40481 XUs and 40236 XVs -> 1101 XUVs
C:0 T:3 24300 XUs and 19984 XVs -> 80 XUVs
C:0 T:4 10835 XUs and 8001 XVs -> 68 XUVs
C:0 T:5 50711 XUs and 27430 XVs -> 2822 XUVs
C:0 T:6 101338 XUs and 139430 XVs -> 4585 XUVs
C:0 T:7 45168 XUs and 54055 XVs -> 439 XUVs
C:0 T:8 8086 XUs and 6789 XVs -> 212 XUVs
C:0 T:9 64123 XUs and 60975 XVs -> 6632 XUVs
C:0 T:10 35850 XUs and 146849 XVs -> 10654 XUVs
C:0 T:11 167 XUs and 136 XVs -> 22 XUVs
30651 XUVs total
9034 collection wire objects
28031 potential space points
Neighbour search...
3163527 tests to find 1429798 neighbours
Iterating with no regularization...
Begin: 7.1707e+07
0 5.74904e+07
1 5.67974e+07
2 5.6709e+07
3 5.66854e+07
Now with regularization...
Begin: 4.87815e+07
0 4.8755e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410191023
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410191023
fLifetime: 30000 [us]
29-Oct-2025 22:37:28 CET  Closed output file "reco_stage1.root"
29-Oct-2025 22:37:28 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          301.885       449.695       617.137       437.842       84.6579        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00125079    0.00582024     0.0120444    0.00528591    0.00311476       10     
reco:ophit:OpHitFinder                             0.369292       0.42256      0.505669      0.416002      0.0393337       10     
reco:opflash:OpFlashFinder                         0.211233       0.27548      0.340215      0.262393      0.0441142       10     
reco:opslicer:OpSlicer                             0.707325      0.820323      0.930535      0.813966      0.0696572       10     
reco:caldata:DataPrepModule                         6.3053        6.55032       7.02849       6.4344       0.229327        10     
reco:wclsdatasp:WireCellToolkit                     45.3089       60.7456       67.5988       63.8502       7.86273        10     
reco:gaushit:GausHitFinder                          1.00727       1.40536       1.87154       1.39395       0.23867        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000391366   0.000720074   0.00121977    0.000736453   0.000271625      10     
reco:reco3d:SpacePointSolver                        14.7923       21.1084       34.4686       20.3364       6.00498        10     
reco:hitpdune:DisambigFromSpacePoints               1.09104       1.79607       2.94697       1.68723      0.487483        10     
reco:pandora:StandardPandora                        22.5776       40.4588       76.9465       37.345        15.9303        10     
reco:pandoraWriter:StandardPandora                  0.38667       0.51949      0.688081      0.505093      0.0917101       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.22927       7.38135       10.7279       6.98026       2.07661        10     
reco:pandoraShower:LArPandoraShowerCreation         3.27559       8.50788       16.8543       7.24632       3.5047         10     
reco:pandoracalo:Calorimetry                        1.88022       3.88999       5.12062       3.77802      0.889896        10     
reco:pandoracalonosce:Calorimetry                   1.53819       3.60389       4.71233       3.85021      0.849102        10     
reco:pandorapid:Chi2ParticleID                    0.00579525    0.00658647    0.00736312    0.00668434    0.000519281      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0416408     0.057262      0.0700505     0.0569623    0.00786788       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00522431    0.00614173    0.00691251    0.00631689    0.000549244      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.82095       8.11154       14.8256       7.50492       3.17992        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.77298       7.59969       14.5174       7.57818       3.01174        10     
reco:emtrkmichelid:EmTrackMichelId                  153.555       203.562       263.789       200.451       30.239         10     
reco:crttag:SingleCRTMatchingProducer              0.0143086     0.0233811     0.0329599     0.0217491    0.00561923       10     
reco:crtreco:TwoCRTMatchingProducer                0.0120803     0.0177681     0.0476322     0.0144586     0.0102073       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.811902       1.03216       1.26577       1.04511      0.134622        10     
reco:pandora2Track:LArPandoraTrackCreation          11.2334       22.3769       39.884        21.3155       7.38249        10     
reco:pandora2calo:Calorimetry                       4.31111       11.0033       18.6611       10.3417       3.83825        10     
reco:pandora2calonosce:Calorimetry                  4.34074       10.0689       15.2601       9.61499       3.15079        10     
reco:pandora2pid:Chi2ParticleID                    0.0105384     0.013923      0.0195558     0.0133268    0.00227019       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0514668     0.0708897     0.0893243     0.0708072     0.0106041       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0102998     0.013988      0.0195727     0.0133911    0.00233398       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.71612       8.20574       11.9481       7.62344       2.4853         10     
reco:pandora2Showercalo:ShowerCalorimetry           3.80247       8.70612       12.5353       8.27338       2.6659         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.42442       7.57351       11.3774       7.44497       2.30002        10     
[art]:TriggerResults:TriggerResultInserter        3.8398e-05    6.02165e-05   0.000150886   4.96645e-05   3.10851e-05      10     
end_path:out1:RootOutput                           7.241e-06    9.4211e-06    1.4983e-05    8.3675e-06    2.15993e-06      10     
end_path:out1:RootOutput(write)                     2.76497       3.47984       4.46088       3.47467      0.502101        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4953.76 MB
  Peak resident set size usage (VmHWM): 4151.81 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.29 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.07 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10071460
-rw-r--r--. 1 dune009 dune 2366111601 Oct 29 21:20 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2243989457 Oct 29 21:18 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2058385998 Oct 29 22:37 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1824083999 Oct 29 18:34 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1743748437 Oct 29 18:19 g4_stage1.root
-rw-r--r--. 1 dune009 dune   76245970 Oct 29 22:33 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242958 Oct 29 22:37 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 29 22:37 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 29 22:37 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 29 22:37 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 29 21:20 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 18:34 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 22:37 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 29 18:12 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 29 22:37 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 29 21:25 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 29 21:20 debugprod.log
justIN time: 2025-11-03 18:40:30 UTC       justIN version: 01.05.01