justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 14367.0@dunegpschedd02.fnal.gov

Jobsub ID14367.0@dunegpschedd02.fnal.gov
Workflow ID374
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-04 18:10:09
SiteUK_Oxford
EntryDUNE_UK_SGrid_Oxford_arc01
Last heartbeat2025-08-04 22:23:30
From worker nodeHostnamet2wn063.physics.ox.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-04 18:13:57
Input filesmonte-carlo-000374-000001
JobscriptExit code0
Real time4h (14854s)
CPU time3h (14135s = 95%)
Max RSS bytes4070629376 (3882 MiB)
Outputting started2025-08-04 22:21:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00374/1/001/reco_stage1.root
Finished2025-08-04 22:23:30
Saved logsjustin-logs:14367.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

are:   Running tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 127056 traces, 0 summary
Retagger: tagging trace set: gauss with 95759 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95759 traces tagged "gauss"
FrameSaver: q=3.14915e+06 n=1657043 tag=gauss
wclsFrameSaver: saving 127056 traces tagged "wiener"
FrameSaver: q=3.34724e+06 n=1454696 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 29832 XUs and 20285 XVs -> 281 XUVs
C:0 T:1 61154 XUs and 45706 XVs -> 3634 XUVs
C:0 T:2 110501 XUs and 114714 XVs -> 6895 XUVs
C:0 T:3 54598 XUs and 46447 XVs -> 1135 XUVs
C:0 T:4 13618 XUs and 12740 XVs -> 105 XUVs
C:0 T:5 49977 XUs and 39319 XVs -> 4791 XUVs
C:0 T:6 117807 XUs and 114650 XVs -> 5701 XUVs
C:0 T:7 41944 XUs and 44971 XVs -> 454 XUVs
C:0 T:8 6701 XUs and 6341 XVs -> 71 XUVs
C:0 T:9 45038 XUs and 62996 XVs -> 3160 XUVs
C:0 T:10 84574 XUs and 26745 XVs -> 12501 XUVs
C:0 T:11 27 XUs and 26 XVs -> 3 XUVs
38731 XUVs total
11220 collection wire objects
35447 potential space points
Neighbour search...
5304889 tests to find 2269706 neighbours
Iterating with no regularization...
Begin: 5.66588e+07
0 4.7243e+07
1 4.65323e+07
2 4.64267e+07
3 4.63955e+07
Now with regularization...
Begin: 3.78789e+07
0 3.78545e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408461866
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408461866
fLifetime: 30000 [us]
04-Aug-2025 23:21:18 BST  Closed output file "reco_stage1.root"
04-Aug-2025 23:21:18 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          311.365       380.401       466.006       354.666       52.1557        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0013475    0.00390125    0.00614114    0.00413302    0.00153985       10     
reco:ophit:OpHitFinder                             0.261075      0.330875      0.404867       0.32856      0.0438399       10     
reco:opflash:OpFlashFinder                         0.141746      0.183207       0.20616      0.186779      0.0176074       10     
reco:opslicer:OpSlicer                             0.558197      0.731092      0.941666      0.730187      0.105014        10     
reco:caldata:DataPrepModule                         5.60521       6.37469       7.02296       6.41365      0.439816        10     
reco:wclsdatasp:WireCellToolkit                     35.7477       38.6417       40.7366       38.9978       1.72756        10     
reco:gaushit:GausHitFinder                          1.25258       1.4467        1.70028       1.43439      0.146465        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000512808   0.000632597   0.000768536   0.00061783    9.31287e-05      10     
reco:reco3d:SpacePointSolver                        13.4876       19.4544       31.9149       18.1875       5.4359         10     
reco:hitpdune:DisambigFromSpacePoints               1.02624       1.69729       2.41262       1.7157       0.354539        10     
reco:pandora:StandardPandora                        20.0898       38.2898       109.867       28.6107       25.1027        10     
reco:pandoraWriter:StandardPandora                 0.374632      0.495453      0.603697      0.498109      0.0784829       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.09765       3.93155       4.97967       3.69778      0.659408        10     
reco:pandoraShower:LArPandoraShowerCreation         2.79323       4.37024       6.27331       4.32426       1.10088        10     
reco:pandoracalo:Calorimetry                        1.25005       2.40035       3.06365       2.48836      0.573363        10     
reco:pandoracalonosce:Calorimetry                   1.32221       2.1263        2.83222       2.19426      0.424572        10     
reco:pandorapid:Chi2ParticleID                    0.00405167    0.00668878    0.00837022    0.00716701    0.00146217       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0397145     0.0602056     0.0814346     0.0613111     0.0127811       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00436652    0.00625852    0.00775939    0.00643385    0.00105654       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.40428       4.17196       6.09286       3.97667       1.01677        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.01477       4.3863        6.69289       4.22548       1.31097        10     
reco:emtrkmichelid:EmTrackMichelId                  185.038       205.18        251.749       192.987       22.2212        10     
reco:crttag:SingleCRTMatchingProducer              0.0087532     0.0157704     0.0210576     0.0158529    0.00374569       10     
reco:crtreco:TwoCRTMatchingProducer               0.00600852    0.00986622     0.0142262     0.0089949    0.00250702       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.660188      0.921127       1.19811      0.898048      0.185038        10     
reco:pandora2Track:LArPandoraTrackCreation          8.99227       13.2158       18.5627       12.7562       3.25611        10     
reco:pandora2calo:Calorimetry                       3.87339       6.71328       9.82474       6.69409       1.74036        10     
reco:pandora2calonosce:Calorimetry                  4.01883       6.31101       9.64711       5.31399       1.77386        10     
reco:pandora2pid:Chi2ParticleID                   0.00997561     0.017427      0.0234354     0.0171792    0.00384002       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0495971     0.0825471     0.106462      0.0872733     0.0189455       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00984002     0.0166235     0.0221126     0.0172137    0.00392277       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.19396       4.60439       6.85663       4.35804       1.1418         10     
reco:pandora2Showercalo:ShowerCalorimetry           3.63119       5.87394       9.09199       5.29202       1.69149        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.24516       5.02531       8.37836       4.53745       1.56409        10     
[art]:TriggerResults:TriggerResultInserter        4.6087e-05    9.00073e-05   0.000212508   7.84465e-05   4.24602e-05      10     
end_path:out1:RootOutput                           1.07e-05     1.72682e-05   3.5526e-05    1.54785e-05   6.53212e-06      10     
end_path:out1:RootOutput(write)                     2.55566       3.17513       3.67302       3.20239       0.39061        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4941.62 MB
  Peak resident set size usage (VmHWM): 4070.63 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.01 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10254980
-rw-r--r-- 1 dune020 dune 2401313558 Aug  4 22:14 detsim_stage2.root
-rw-r--r-- 1 dune020 dune 2285854754 Aug  4 22:12 detsim_stage1.root
-rw-r--r-- 1 dune020 dune 2067941051 Aug  4 23:21 reco_stage1.root
-rw-r--r-- 1 dune020 dune 1878100923 Aug  4 19:37 g4_stage2.root
-rw-r--r-- 1 dune020 dune 1797304641 Aug  4 19:22 g4_stage1.root
-rw-r--r-- 1 dune020 dune   69944070 Aug  4 23:16 Pandora_Events.pndr
-rw-r--r-- 1 dune020 dune     242898 Aug  4 23:21 jobscript.log
-rw-r--r-- 1 dune020 dune     217088 Aug  4 23:21 mem.db
-rw-r--r-- 1 dune020 dune      62376 Aug  4 23:21 Pandora_Geometry.xml
-rw-r--r-- 1 dune020 dune      36864 Aug  4 23:21 time.db
-rw-r--r-- 1 dune020 dune        519 Aug  4 22:14 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Aug  4 19:37 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune020 dune        519 Aug  4 23:21 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune020 dune         52 Aug  4 19:13 all-input-dids.txt
-rw-r--r-- 1 dune020 dune          7 Aug  4 23:21 justin-processed-pfns.txt
-rw-r--r-- 1 dune020 dune          0 Aug  4 22:20 ProtoDUNE.log
-rw-r--r-- 1 dune020 dune          0 Aug  4 22:15 debugprod.log
justIN time: 2025-08-04 23:59:10 UTC       justIN version: 01.04.00