justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41105.0@dunegpschedd01.fnal.gov

Jobsub ID41105.0@dunegpschedd01.fnal.gov
Workflow ID2292
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-09-05 20:59:57
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-09-06 01:02:18
From worker nodeHostnamen226.dur.scotgrid.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-05 21:01:42
Input filesmonte-carlo-002292-000001
JobscriptExit code0
Real time3h (14327s)
CPU time3h (14063s = 98%)
Max RSS bytes4310085632 (4110 MiB)
Outputting started2025-09-06 01:00:30
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/02292/1/001/reco_stage1.root
Finished2025-09-06 01:02:18
Saved logsjustin-logs:41105.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 160774 traces, 0 summary
Retagger: tagging trace set: gauss with 115750 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115750 traces tagged "gauss"
FrameSaver: q=3.9498e+06 n=1974993 tag=gauss
wclsFrameSaver: saving 160774 traces tagged "wiener"
FrameSaver: q=4.20917e+06 n=1700674 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33005 XUs and 26382 XVs -> 325 XUVs
C:0 T:1 78041 XUs and 66392 XVs -> 4826 XUVs
C:0 T:2 94011 XUs and 157937 XVs -> 10859 XUVs
C:0 T:3 40057 XUs and 46871 XVs -> 557 XUVs
C:0 T:4 13897 XUs and 18088 XVs -> 152 XUVs
C:0 T:5 71381 XUs and 81328 XVs -> 8676 XUVs
C:0 T:6 180393 XUs and 188883 XVs -> 23797 XUVs
C:0 T:7 48455 XUs and 50879 XVs -> 975 XUVs
C:0 T:8 9774 XUs and 8674 XVs -> 91 XUVs
C:0 T:9 87406 XUs and 96450 XVs -> 5965 XUVs
C:0 T:10 22448 XUs and 26952 XVs -> 3154 XUVs
C:0 T:11 74 XUs and 47 XVs -> 4 XUVs
59381 XUVs total
15833 collection wire objects
53518 potential space points
Neighbour search...
8983442 tests to find 3876990 neighbours
Iterating with no regularization...
Begin: 7.72299e+07
0 5.60754e+07
1 5.47839e+07
2 5.45873e+07
3 5.45285e+07
4 5.45021e+07
Now with regularization...
Begin: 4.3175e+07
0 4.3104e+07
1 4.30677e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 409107709
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 409107709
fLifetime: 30000 [us]
06-Sep-2025 02:00:28 BST  Closed output file "reco_stage1.root"
06-Sep-2025 02:00:28 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          426.294       620.831       864.173       629.672       119.899        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00133701    0.00456571    0.00694103    0.00496097    0.00171984       10     
reco:ophit:OpHitFinder                             0.273043       0.3295        0.39456      0.332417      0.0351992       10     
reco:opflash:OpFlashFinder                         0.203758      0.268918      0.343041      0.276423      0.0437544       10     
reco:opslicer:OpSlicer                             0.585515      0.798469       1.0351       0.812727      0.114139        10     
reco:caldata:DataPrepModule                         6.4576        6.63086       6.82605       6.61332      0.134659        10     
reco:wclsdatasp:WireCellToolkit                     55.2812       60.7512       63.9876       61.6962       2.51222        10     
reco:gaushit:GausHitFinder                          1.33973       1.83062       2.28397       1.88633       0.26386        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000655625   0.000850513   0.00120845    0.000807088   0.000159059      10     
reco:reco3d:SpacePointSolver                        9.69494       20.6379       34.6095       21.1853       7.10315        10     
reco:hitpdune:DisambigFromSpacePoints               1.76417       2.92816       4.16487       2.93707      0.673014        10     
reco:pandora:StandardPandora                        32.588        60.4361       89.0566       63.4383       15.8292        10     
reco:pandoraWriter:StandardPandora                 0.476912      0.626441      0.808435      0.624636      0.0960593       10     
reco:pandoraTrack:LArPandoraTrackCreation           8.55043       13.078        18.5561       12.0573       3.16059        10     
reco:pandoraShower:LArPandoraShowerCreation         9.74304       16.546        28.7477       16.3081       5.48809        10     
reco:pandoracalo:Calorimetry                        4.05478       6.28778       9.27681       5.73969       1.77094        10     
reco:pandoracalonosce:Calorimetry                   3.75466       6.02041       9.06364       5.59138       1.74729        10     
reco:pandorapid:Chi2ParticleID                    0.00623157    0.00727493    0.00861081    0.00711038    0.000794328      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0605224     0.080503      0.107468      0.0792937     0.0126464       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00597949    0.00699501    0.00843318    0.00673944    0.000709961      10     
reco:pandoraShowercalo:ShowerCalorimetry            7.43431       15.9745       27.7148       15.3338       5.93984        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.29187       15.9032       27.1841       15.9064       5.73584        10     
reco:emtrkmichelid:EmTrackMichelId                  182.18        246.555       316.08        247.322       35.2471        10     
reco:crttag:SingleCRTMatchingProducer              0.0271919     0.0357723     0.0466134     0.0364844    0.00631285       10     
reco:crtreco:TwoCRTMatchingProducer                0.0233331     0.0307324     0.040777      0.0326535    0.00587312       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.1208        1.56627       1.9808        1.5548       0.245122        10     
reco:pandora2Track:LArPandoraTrackCreation          29.3008       47.2638       74.478        48.2456       12.6981        10     
reco:pandora2calo:Calorimetry                       11.4039       21.7417       35.3432       23.8726       6.84198        10     
reco:pandora2calonosce:Calorimetry                  10.8628       20.6813       33.3365       22.4539       6.49511        10     
reco:pandora2pid:Chi2ParticleID                    0.0139958     0.0180596     0.0222516     0.0182603    0.00223572       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0752524     0.102693      0.133826       0.1036       0.014841        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0134817     0.0175555     0.021227      0.0176326    0.00212358       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.6251        15.3764       24.8801       15.4464       4.49179        10     
reco:pandora2Showercalo:ShowerCalorimetry           10.2415       18.2364       29.7576       19.8222       5.99932        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.14303       16.3835       28.6034       17.5212       5.85774        10     
[art]:TriggerResults:TriggerResultInserter        5.5832e-05    7.40197e-05   0.000123807   6.5554e-05    2.01111e-05      10     
end_path:out1:RootOutput                           9.385e-06    1.17393e-05   1.5325e-05    1.04185e-05   2.26545e-06      10     
end_path:out1:RootOutput(write)                     2.46906       3.46926       4.48235       3.49853      0.564127        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5115.68 MB
  Peak resident set size usage (VmHWM): 4310.09 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.09 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10512540
-rw-r--r--. 1 dune001 dune 2455671103 Sep  6 00:14 detsim_stage2.root
-rw-r--r--. 1 dune001 dune 2327803944 Sep  6 00:12 detsim_stage1.root
-rw-r--r--. 1 dune001 dune 2181336644 Sep  6 02:00 reco_stage1.root
-rw-r--r--. 1 dune001 dune 1905783068 Sep  5 22:28 g4_stage2.root
-rw-r--r--. 1 dune001 dune 1808887626 Sep  5 22:08 g4_stage1.root
-rw-r--r--. 1 dune001 dune   84729720 Sep  6 01:51 Pandora_Events.pndr
-rw-r--r--. 1 dune001 dune     243121 Sep  6 02:00 jobscript.log
-rw-r--r--. 1 dune001 dune     217088 Sep  6 02:00 mem.db
-rw-r--r--. 1 dune001 dune      62376 Sep  6 02:00 Pandora_Geometry.xml
-rw-r--r--. 1 dune001 dune      36864 Sep  6 02:00 time.db
-rw-r--r--. 1 dune001 dune        519 Sep  6 00:14 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune001 dune        519 Sep  5 22:28 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune001 dune        519 Sep  6 02:00 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune001 dune         52 Sep  5 22:01 all-input-dids.txt
-rw-r--r--. 1 dune001 dune          7 Sep  6 02:00 justin-processed-pfns.txt
-rw-r--r--. 1 dune001 dune          0 Sep  6 00:22 ProtoDUNE.log
-rw-r--r--. 1 dune001 dune          0 Sep  6 00:14 debugprod.log
justIN time: 2025-09-18 21:55:25 UTC       justIN version: 01.05.00