justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 247173.0@dunegpschedd02.fnal.gov

Jobsub ID247173.0@dunegpschedd02.fnal.gov
Workflow ID10260
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 16:33:04
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-11-16 22:49:48
From worker nodeHostnamewn-lb-03.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 16:33:42
Input filesmonte-carlo-010260-000001
JobscriptExit code0
Real time6h (22470s)
CPU time6h (22327s = 99%)
Max RSS bytes4739772416 (4520 MiB)
Outputting started2025-11-16 22:48:13
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10260/1/001/reco_stage1.root
Finished2025-11-16 22:49:48
Saved logsjustin-logs:247173.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

llSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 119174 traces, 0 summary
Retagger: tagging trace set: gauss with 90873 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90873 traces tagged "gauss"
FrameSaver: q=3.17695e+06 n=1599261 tag=gauss
wclsFrameSaver: saving 119174 traces tagged "wiener"
FrameSaver: q=3.37711e+06 n=1417287 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39659 XUs and 42177 XVs -> 536 XUVs
C:0 T:1 81669 XUs and 93187 XVs -> 7109 XUVs
C:0 T:2 57594 XUs and 58488 XVs -> 1798 XUVs
C:0 T:3 29556 XUs and 26503 XVs -> 165 XUVs
C:0 T:4 5046 XUs and 12163 XVs -> 48 XUVs
C:0 T:5 26219 XUs and 42261 XVs -> 3246 XUVs
C:0 T:6 65797 XUs and 70331 XVs -> 2531 XUVs
C:0 T:7 31091 XUs and 28512 XVs -> 374 XUVs
C:0 T:8 7303 XUs and 10607 XVs -> 155 XUVs
C:0 T:9 46612 XUs and 81569 XVs -> 5406 XUVs
C:0 T:10 42749 XUs and 31910 XVs -> 4303 XUVs
C:0 T:11 313 XUs and 230 XVs -> 25 XUVs
25696 XUVs total
11198 collection wire objects
23736 potential space points
Neighbour search...
1170784 tests to find 524972 neighbours
Iterating with no regularization...
Begin: 1.02821e+08
0 7.54721e+07
1 7.42312e+07
2 7.40855e+07
3 7.40496e+07
Now with regularization...
Begin: 6.24581e+07
0 6.23299e+07
1 6.23158e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410552748
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410552748
fLifetime: 30000 [us]
16-Nov-2025 23:48:11 CET  Closed output file "reco_stage1.root"
16-Nov-2025 23:48:11 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          184.751       1177.44       6256.94       327.041       1836.39        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00103444    0.00535152    0.00935895    0.00567428    0.00245679       10     
reco:ophit:OpHitFinder                             0.232399       0.28909      0.353994      0.286384      0.0407987       10     
reco:opflash:OpFlashFinder                         0.115982      0.168725      0.235642      0.165337      0.0399766       10     
reco:opslicer:OpSlicer                             0.379699      0.559729      0.776822      0.555663       0.12082        10     
reco:caldata:DataPrepModule                         4.73593       5.17574       6.31632        4.967        0.52155        10     
reco:wclsdatasp:WireCellToolkit                     34.7032       49.0793       66.0786       49.3646       9.97853        10     
reco:gaushit:GausHitFinder                         0.602917       1.17555       2.20591       0.96189      0.544278        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000300712   0.000747162   0.00123038    0.00070227    0.000258415      10     
reco:reco3d:SpacePointSolver                        5.55077       46.7103       165.469       14.3549       59.6377        10     
reco:hitpdune:DisambigFromSpacePoints              0.423326       1.68349       3.39891       1.18957       1.07254        10     
reco:pandora:StandardPandora                        7.27905       760.836       5323.55       24.0975       1622.66        10     
reco:pandoraWriter:StandardPandora                 0.209439      0.539798       1.18095      0.418954      0.294127        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.02419       10.8108       46.7214       4.85026       13.1504        10     
reco:pandoraShower:LArPandoraShowerCreation        0.743128       9.40137       29.467        5.24242       8.89424        10     
reco:pandoracalo:Calorimetry                       0.614312       4.91504       19.4491       2.71174       5.39132        10     
reco:pandoracalonosce:Calorimetry                   0.56873       4.85207       19.5206       2.62419       5.49731        10     
reco:pandorapid:Chi2ParticleID                    0.00249277    0.00676283     0.0151501    0.00588772    0.00343937       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0228303     0.0501126     0.0781187     0.0476506     0.0147936       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00233264    0.00655009     0.0152588    0.00558671    0.00354169       10     
reco:pandoraShowercalo:ShowerCalorimetry           0.766612       8.0912        20.1413       5.65941       6.73554        10     
reco:pandoraShowercalonosce:ShowerCalorimetry      0.720037       8.02654       20.8998       5.46665       6.9505         10     
reco:emtrkmichelid:EmTrackMichelId                  83.6933       198.976       381.777       162.061       89.6643        10     
reco:crttag:SingleCRTMatchingProducer              0.0049844     0.0154063     0.0260772     0.0151496     0.0070331       10     
reco:crtreco:TwoCRTMatchingProducer               0.00414459     0.0114574     0.0183326    0.00982367    0.00480446       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.416441       0.84839       1.27036      0.748989      0.262588        10     
reco:pandora2Track:LArPandoraTrackCreation          2.38967       23.1115       68.6988       11.5378       20.4995        10     
reco:pandora2calo:Calorimetry                       1.27461       10.4975       30.2167       5.37471       9.44662        10     
reco:pandora2calonosce:Calorimetry                  1.18332       8.61542       22.3864       5.11938       7.01959        10     
reco:pandora2pid:Chi2ParticleID                   0.00493101     0.0128075     0.0288455     0.010683     0.00671064       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0273187     0.0608921      0.11247      0.0539795     0.0250219       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00483328     0.0129173     0.0291701     0.0107789    0.00676559       10     
reco:pandora2Shower:LArPandoraShowerCreation       0.919981       6.32903       16.3103       4.32531       4.63856        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.05688       7.04687       17.8863       5.20527       5.08888        10     
reco:pandora2Showercalonosce:ShowerCalorimetry     0.990561       6.38792       17.386        4.49072       4.68716        10     
[art]:TriggerResults:TriggerResultInserter         3.323e-05    5.04218e-05   0.000114212   4.04205e-05   2.30963e-05      10     
end_path:out1:RootOutput                           7.611e-06    8.6089e-06    1.2859e-05     8.097e-06    1.47733e-06      10     
end_path:out1:RootOutput(write)                     1.26954       2.88985       5.5076        2.55982       1.19553        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5543.73 MB
  Peak resident set size usage (VmHWM): 4739.77 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.19 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10763672
-rw-r--r--. 1 dune009 dune 2545807954 Nov 16 20:30 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2437180228 Nov 16 20:28 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2241618834 Nov 16 23:48 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1882799668 Nov 16 17:55 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1833845529 Nov 16 17:41 g4_stage1.root
-rw-r--r--. 1 dune009 dune   80130510 Nov 16 23:44 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243913 Nov 16 23:48 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 16 23:48 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 16 23:48 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 16 23:48 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 16 20:30 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 16 17:55 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 16 23:48 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 16 17:33 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 16 23:48 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 16 20:36 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 16 20:30 debugprod.log
justIN time: 2026-02-09 10:52:27 UTC       justIN version: 01.06.00