justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 14105.0@dunegpschedd02.fnal.gov

Jobsub ID14105.0@dunegpschedd02.fnal.gov
Workflow ID261
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-03 07:30:14
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-08-03 15:03:53
From worker nodeHostnamewn5916290.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-03 07:31:33
Input filesmonte-carlo-000261-000001
JobscriptExit code0
Real time7h (27022s)
CPU time7h (25652s = 94%)
Max RSS bytes4150976512 (3958 MiB)
Outputting started2025-08-03 15:01:57
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00261/1/001/reco_stage1.root
Finished2025-08-03 15:03:53
Saved logsjustin-logs:14105.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

BasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 125740 traces, 0 summary
Retagger: tagging trace set: gauss with 92921 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 92921 traces tagged "gauss"
FrameSaver: q=3.15912e+06 n=1622248 tag=gauss
wclsFrameSaver: saving 125740 traces tagged "wiener"
FrameSaver: q=3.36176e+06 n=1425238 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24405 XUs and 20868 XVs -> 379 XUVs
C:0 T:1 49697 XUs and 72333 XVs -> 7452 XUVs
C:0 T:2 59365 XUs and 90210 XVs -> 3925 XUVs
C:0 T:3 27970 XUs and 39477 XVs -> 156 XUVs
C:0 T:4 12508 XUs and 13146 XVs -> 123 XUVs
C:0 T:5 68404 XUs and 73374 XVs -> 8588 XUVs
C:0 T:6 105053 XUs and 97031 XVs -> 5214 XUVs
C:0 T:7 42708 XUs and 40667 XVs -> 581 XUVs
C:0 T:8 7472 XUs and 6725 XVs -> 122 XUVs
C:0 T:9 68784 XUs and 81195 XVs -> 6883 XUVs
C:0 T:10 45585 XUs and 30553 XVs -> 11232 XUVs
C:0 T:11 590 XUs and 448 XVs -> 346 XUVs
45001 XUVs total
12001 collection wire objects
42435 potential space points
Neighbour search...
5101101 tests to find 2622036 neighbours
Iterating with no regularization...
Begin: 6.17592e+07
0 4.69625e+07
1 4.58937e+07
2 4.57709e+07
3 4.57312e+07
Now with regularization...
Begin: 3.72826e+07
0 3.72202e+07
1 3.71814e+07
2 3.71531e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408432805
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408432805
fLifetime: 30000 [us]
03-Aug-2025 16:01:51 BST  Closed output file "reco_stage1.root"
03-Aug-2025 16:01:51 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          687.565       1041.95       1501.68       987.001       248.628        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00386256     0.0114895     0.0249424     0.0100074    0.00661126       10     
reco:ophit:OpHitFinder                             0.387468      0.466973      0.528695      0.465216      0.0393576       10     
reco:opflash:OpFlashFinder                         0.178021      0.278141      0.333732       0.28583      0.042804        10     
reco:opslicer:OpSlicer                              1.04642       1.25585       1.44622       1.27339      0.115021        10     
reco:caldata:DataPrepModule                         9.66259       10.1964       11.0769       10.1963      0.408798        10     
reco:wclsdatasp:WireCellToolkit                     70.3849       80.0511       95.528        79.1655       7.86518        10     
reco:gaushit:GausHitFinder                          2.19394       2.80212       3.88257       2.64914       0.58058        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00132029    0.00236842    0.00434939    0.00210724    0.000885269      10     
reco:reco3d:SpacePointSolver                        18.7383       36.2031       64.4819       32.9642       12.5494        10     
reco:hitpdune:DisambigFromSpacePoints               2.67777       4.19086       7.88518       3.39232       1.65333        10     
reco:pandora:StandardPandora                        50.2348       99.4891       208.59        78.9129       49.3189        10     
reco:pandoraWriter:StandardPandora                  0.93389       1.21661       1.73921       1.16112       0.26156        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.77555       18.993        27.1797       18.3177       4.97428        10     
reco:pandoraShower:LArPandoraShowerCreation         9.44233       24.865        47.2263       21.9343       11.6866        10     
reco:pandoracalo:Calorimetry                        5.54625       10.916        18.0064       10.8883       3.37298        10     
reco:pandoracalonosce:Calorimetry                   5.03303       9.93426       16.4798       10.0441       3.02796        10     
reco:pandorapid:Chi2ParticleID                    0.00848105     0.013699      0.0216103     0.0131348    0.00403784       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0597666      0.11768      0.180729      0.114218      0.0365205       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00672036     0.0124689     0.0189876     0.0123708    0.00349543       10     
reco:pandoraShowercalo:ShowerCalorimetry            11.8573       26.0181       49.944        24.9751       10.2639        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.29143       26.9684       47.5262       26.9742       10.8167        10     
reco:emtrkmichelid:EmTrackMichelId                  330.096       464.629       600.869       438.831       83.3152        10     
reco:crttag:SingleCRTMatchingProducer              0.0201246     0.0512498     0.0688099     0.0572772     0.0157975       10     
reco:crtreco:TwoCRTMatchingProducer                0.0197818     0.0476081     0.0639071     0.0515336     0.0125582       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.33745       1.70947       2.34179       1.74537      0.285575        10     
reco:pandora2Track:LArPandoraTrackCreation          25.8834       62.5805       93.5348       64.1038       20.9536        10     
reco:pandora2calo:Calorimetry                       13.9124       36.342        60.8826       35.659        14.9625        10     
reco:pandora2calonosce:Calorimetry                  15.8414       32.8767       58.4961       33.1012       13.0326        10     
reco:pandora2pid:Chi2ParticleID                    0.0211614     0.0306339     0.048002      0.0295466    0.00764084       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.120813      0.168012      0.270509      0.154264      0.0407103       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0236164     0.0333462     0.0514691     0.0333024    0.00760106       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.9613       21.5184       35.8992       21.1225       7.60603        10     
reco:pandora2Showercalo:ShowerCalorimetry           12.8973       32.0899       55.0241       32.9502       12.7211        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.89503       29.3721       51.0016       29.8247       12.8127        10     
[art]:TriggerResults:TriggerResultInserter        5.3011e-05    0.000107919   0.000204176   7.07035e-05   5.66434e-05      10     
end_path:out1:RootOutput                           8.256e-06    1.41228e-05   2.4306e-05    1.13415e-05   5.18033e-06      10     
end_path:out1:RootOutput(write)                     4.26179       6.05041       7.7722        5.87978       1.1564         10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4989.45 MB
  Peak resident set size usage (VmHWM): 4150.98 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.48 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.30 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10026224
-rw-r--r-- 1 dune01 dune01 2361511511 Aug  3 13:02 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2243213527 Aug  3 12:57 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2061691600 Aug  3 16:01 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1801166599 Aug  3 09:15 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1720590753 Aug  3 08:43 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   78050960 Aug  3 15:48 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243008 Aug  3 16:01 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Aug  3 16:01 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Aug  3 16:01 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Aug  3 16:01 time.db
-rw-r--r-- 1 dune01 dune01        519 Aug  3 13:02 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Aug  3 09:15 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Aug  3 16:01 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Aug  3 08:31 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Aug  3 16:01 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Aug  3 13:17 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Aug  3 13:02 debugprod.log
justIN time: 2025-08-04 14:14:00 UTC       justIN version: 01.04.00