justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242113.0@dunegpschedd01.fnal.gov

Jobsub ID242113.0@dunegpschedd01.fnal.gov
Workflow ID9533
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 23:12:06
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-03 06:37:21
From worker nodeHostnamewn2206240.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 23:13:39
Input filesmonte-carlo-009533-000001
JobscriptExit code0
Real time7h (26509s)
CPU time7h (25592s = 96%)
Max RSS bytes4429881344 (4224 MiB)
Outputting started2025-11-03 06:35:28
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09533/1/001/reco_stage1.root
Finished2025-11-03 06:37:21
Saved logsjustin-logs:242113.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 128428 traces, 0 summary
Retagger: tagging trace set: gauss with 99415 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 99415 traces tagged "gauss"
FrameSaver: q=3.27259e+06 n=1741483 tag=gauss
wclsFrameSaver: saving 128428 traces tagged "wiener"
FrameSaver: q=3.46013e+06 n=1543260 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 26170 XUs and 27154 XVs -> 196 XUVs
C:0 T:1 54807 XUs and 57330 XVs -> 5122 XUVs
C:0 T:2 111758 XUs and 128494 XVs -> 7179 XUVs
C:0 T:3 54029 XUs and 51293 XVs -> 435 XUVs
C:0 T:4 8579 XUs and 8990 XVs -> 127 XUVs
C:0 T:5 30113 XUs and 41359 XVs -> 4828 XUVs
C:0 T:6 140659 XUs and 117030 XVs -> 5653 XUVs
C:0 T:7 58046 XUs and 44887 XVs -> 602 XUVs
C:0 T:8 6112 XUs and 6152 XVs -> 86 XUVs
C:0 T:9 30751 XUs and 38347 XVs -> 1926 XUVs
C:0 T:10 61989 XUs and 50205 XVs -> 9820 XUVs
C:0 T:11 35 XUs and 14 XVs -> 1 XUVs
35975 XUVs total
11256 collection wire objects
33485 potential space points
Neighbour search...
3568095 tests to find 1722026 neighbours
Iterating with no regularization...
Begin: 6.42662e+07
0 5.51051e+07
1 5.44324e+07
2 5.43235e+07
3 5.42926e+07
Now with regularization...
Begin: 4.57264e+07
0 4.56904e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410276684
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410276684
fLifetime: 30000 [us]
03-Nov-2025 06:35:20 GMT  Closed output file "reco_stage1.root"
03-Nov-2025 06:35:20 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          689.683       983.403       1332.05       955.159       166.332        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00211123    0.00981643     0.0215244     0.0104626    0.00549691       10     
reco:ophit:OpHitFinder                             0.445827      0.528718      0.584516      0.526557      0.0440421       10     
reco:opflash:OpFlashFinder                         0.315473      0.380621      0.450048      0.386854      0.0463058       10     
reco:opslicer:OpSlicer                              1.00924       1.26155       1.47395       1.26573      0.123782        10     
reco:caldata:DataPrepModule                         9.66757       10.1266       10.785        10.0632      0.356776        10     
reco:wclsdatasp:WireCellToolkit                     74.2393       81.473        88.716        81.4085       4.86236        10     
reco:gaushit:GausHitFinder                          2.27023       2.81384       3.42027       2.79486      0.378201        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000867899   0.00113638    0.00152338    0.00109022    0.000186756      10     
reco:reco3d:SpacePointSolver                        24.3995       31.8054       43.8079       31.6786       6.50597        10     
reco:hitpdune:DisambigFromSpacePoints               3.63803       5.45268       8.07369       4.99693       1.45585        10     
reco:pandora:StandardPandora                        52.4298       100.837       137.968       97.1801       22.4244        10     
reco:pandoraWriter:StandardPandora                 0.878382       1.0499        1.2929        1.0077       0.139877        10     
reco:pandoraTrack:LArPandoraTrackCreation           15.2369       21.7289       28.0447       22.4349       3.40188        10     
reco:pandoraShower:LArPandoraShowerCreation         12.6657       23.6897       36.687        23.0562       5.85469        10     
reco:pandoracalo:Calorimetry                        7.57566       10.8639       14.0923       10.8904       1.68405        10     
reco:pandoracalonosce:Calorimetry                   7.18875       10.5053       13.5605       10.3359       1.67252        10     
reco:pandorapid:Chi2ParticleID                     0.0115565     0.0136329     0.0172772     0.0135021    0.00163239       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.100963      0.144582      0.186147      0.139875      0.0248847       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0110762     0.013259      0.0217982     0.0128165    0.00295979       10     
reco:pandoraShowercalo:ShowerCalorimetry            13.6059       25.6319       40.5197       24.783        6.53195        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       13.1219       25.5407       39.9667       24.7219       6.57879        10     
reco:emtrkmichelid:EmTrackMichelId                  311.669       402.741       524.446       395.107       61.8222        10     
reco:crttag:SingleCRTMatchingProducer              0.0418491     0.0930252     0.323922      0.0656079     0.0786559       10     
reco:crtreco:TwoCRTMatchingProducer                0.0303109     0.0432599     0.0557465     0.0424636    0.00710387       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.3342        1.99907       2.5078        2.10192      0.330472        10     
reco:pandora2Track:LArPandoraTrackCreation          40.8294       71.2249       111.511       69.752        17.2082        10     
reco:pandora2calo:Calorimetry                       19.8867       35.3585       52.7231       34.6276       8.10885        10     
reco:pandora2calonosce:Calorimetry                  18.8469       33.9595       50.7589       33.1823       7.90488        10     
reco:pandora2pid:Chi2ParticleID                    0.024046      0.0320451     0.0399257     0.0318768    0.00394354       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.120185      0.200108      0.444719      0.169863      0.0869724       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0226263     0.0307433     0.0354813     0.0304071    0.00368538       10     
reco:pandora2Shower:LArPandoraShowerCreation        13.1421       21.9016       33.8888       21.899        5.24188        10     
reco:pandora2Showercalo:ShowerCalorimetry           16.7809       29.6002       43.5635       29.2825       6.53563        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      15.343        26.6626       40.0436       26.8564       6.25202        10     
[art]:TriggerResults:TriggerResultInserter        9.1054e-05    0.000130792   0.000249904   0.000109751   4.49131e-05      10     
end_path:out1:RootOutput                          1.5266e-05    1.85426e-05   2.7831e-05    1.8035e-05    3.45252e-06      10     
end_path:out1:RootOutput(write)                     4.01841       5.41119       7.42535       5.07383      0.947722        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5251.75 MB
  Peak resident set size usage (VmHWM): 4429.88 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11377620
-rw-r--r-- 1 dune01 dune01 2634790788 Nov  3 03:47 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2497576321 Nov  3 03:42 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2372430697 Nov  3 06:35 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2067670185 Nov  2 23:52 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1990353077 Nov  2 23:24 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87229030 Nov  3 06:24 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243354 Nov  3 06:35 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  3 06:35 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  3 06:35 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  3 06:35 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  3 03:47 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 23:52 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  3 06:35 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  2 23:13 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  3 06:35 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  3 04:02 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  3 03:47 debugprod.log
justIN time: 2025-11-03 18:30:21 UTC       justIN version: 01.05.01