justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238302.1@dunegpschedd01.fnal.gov

Jobsub ID238302.1@dunegpschedd01.fnal.gov
Workflow ID9324
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 18:49:24
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-10-25 01:10:29
From worker nodeHostnamewn5917301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 18:50:15
Input filesmonte-carlo-009324-000001
JobscriptExit code0
Real time6h (22701s)
CPU time6h (21768s = 95%)
Max RSS bytes4301680640 (4102 MiB)
Outputting started2025-10-25 01:08:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09324/1/001/reco_stage1.root
Finished2025-10-25 01:10:29
Saved logsjustin-logs:238302.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ning tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 101868 traces, 0 summary
Retagger: tagging trace set: gauss with 74684 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 74684 traces tagged "gauss"
FrameSaver: q=2.48093e+06 n=1258988 tag=gauss
wclsFrameSaver: saving 101868 traces tagged "wiener"
FrameSaver: q=2.63512e+06 n=1085057 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 19349 XUs and 19998 XVs -> 185 XUVs
C:0 T:1 30425 XUs and 27354 XVs -> 2596 XUVs
C:0 T:2 73751 XUs and 78035 XVs -> 5193 XUVs
C:0 T:3 37474 XUs and 29108 XVs -> 686 XUVs
C:0 T:4 7440 XUs and 9686 XVs -> 58 XUVs
C:0 T:5 31842 XUs and 29606 XVs -> 3502 XUVs
C:0 T:6 63544 XUs and 59128 XVs -> 2645 XUVs
C:0 T:7 28965 XUs and 22832 XVs -> 188 XUVs
C:0 T:8 3333 XUs and 4313 XVs -> 57 XUVs
C:0 T:9 20672 XUs and 35838 XVs -> 3651 XUVs
C:0 T:10 70927 XUs and 47888 XVs -> 6387 XUVs
C:0 T:11 332 XUs and 264 XVs -> 58 XUVs
25206 XUVs total
9753 collection wire objects
22698 potential space points
Neighbour search...
1396256 tests to find 717574 neighbours
Iterating with no regularization...
Begin: 4.73183e+07
0 3.61853e+07
1 3.55154e+07
2 3.54277e+07
3 3.54012e+07
Now with regularization...
Begin: 2.83957e+07
0 2.83795e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410091904
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410091904
fLifetime: 30000 [us]
25-Oct-2025 02:08:33 BST  Closed output file "reco_stage1.root"
25-Oct-2025 02:08:33 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          481.56        779.601       1093.42       811.222       182.904        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00184398    0.00724594     0.0136157    0.00759463    0.00328675       10     
reco:ophit:OpHitFinder                             0.408316      0.474699      0.548122      0.480582      0.0424635       10     
reco:opflash:OpFlashFinder                         0.233132      0.284291      0.367311      0.287036      0.0400976       10     
reco:opslicer:OpSlicer                              1.02668       1.32572       1.58827       1.32977      0.170431        10     
reco:caldata:DataPrepModule                         8.65081       9.19277       9.86593       9.17853      0.353508        10     
reco:wclsdatasp:WireCellToolkit                     54.8714       63.3348       70.0655       63.4043       5.32266        10     
reco:gaushit:GausHitFinder                          1.52796       2.2462        3.04353       2.25741       0.45315        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000724076   0.00102045    0.00138558    0.00102455    0.000212523      10     
reco:reco3d:SpacePointSolver                        12.0838       33.3315       50.764        36.7828       13.0461        10     
reco:hitpdune:DisambigFromSpacePoints               1.58179       3.1883        4.97124       3.16562       1.03735        10     
reco:pandora:StandardPandora                        34.0684       77.6122       139.327       78.1058       29.1672        10     
reco:pandoraWriter:StandardPandora                 0.618278      0.897446       1.16721      0.917539      0.180782        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.94308       11.1723       18.6481       11.6465       3.97316        10     
reco:pandoraShower:LArPandoraShowerCreation         4.86835       12.4808       22.6433       12.2294       5.30076        10     
reco:pandoracalo:Calorimetry                        2.82781       6.54296       9.48155       6.5864        2.19203        10     
reco:pandoracalonosce:Calorimetry                   2.54225       5.93256       9.5257        5.5222        2.17739        10     
reco:pandorapid:Chi2ParticleID                    0.00850216     0.0120347     0.014727      0.0128041    0.00225304       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0819286     0.118095      0.139849      0.120047      0.0176422       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00825517     0.0115101     0.0146329     0.0118011    0.00194455       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.71694       13.4759       24.245        12.3407       5.92749        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.57155       12.6228       22.4227       12.4773       5.13908        10     
reco:emtrkmichelid:EmTrackMichelId                  278.756       411.487       538.961       430.264       77.6248        10     
reco:crttag:SingleCRTMatchingProducer              0.0150181     0.0303984     0.0397852     0.0315487    0.00803064       10     
reco:crtreco:TwoCRTMatchingProducer                0.0133483     0.0227515     0.0272686     0.0255283    0.00531451       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.18562       1.63003       2.10739       1.5586       0.279233        10     
reco:pandora2Track:LArPandoraTrackCreation          14.0193       33.479        55.0337       33.8478       12.4222        10     
reco:pandora2calo:Calorimetry                       6.39239       17.5618       26.7898       18.1887       6.69835        10     
reco:pandora2calonosce:Calorimetry                  6.22424       16.8246       26.8086       17.0417       6.35561        10     
reco:pandora2pid:Chi2ParticleID                    0.018751      0.028439       0.03886      0.0290965    0.00578352       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0938746     0.149824      0.196327      0.153067      0.0299339       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0187834     0.0284507     0.0386265     0.027792     0.00566087       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.17326       11.8735       18.7727       12.4259       4.1917         10     
reco:pandora2Showercalo:ShowerCalorimetry           5.30243       14.2947       22.9711       14.7679       5.36014        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.78177       12.2638       19.6187       13.0408       4.29723        10     
[art]:TriggerResults:TriggerResultInserter        4.3472e-05    6.76525e-05   0.000183095   5.21685e-05   4.04706e-05      10     
end_path:out1:RootOutput                           7.294e-06    8.8156e-06    1.6982e-05    7.8445e-06    2.77589e-06      10     
end_path:out1:RootOutput(write)                     3.71041       5.42983       6.6882        5.61235      0.993149        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5154.78 MB
  Peak resident set size usage (VmHWM): 4301.68 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.92 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10677816
-rw-r--r-- 1 dune01 dune01 2480340940 Oct 24 23:54 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2355288056 Oct 24 23:50 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2181844707 Oct 25 02:08 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1967386617 Oct 24 20:27 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1870056128 Oct 24 20:00 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   78542880 Oct 25 02:02 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242970 Oct 25 02:08 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 25 02:08 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 25 02:08 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 25 02:08 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 24 23:54 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 24 20:27 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 25 02:08 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 24 19:50 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 25 02:08 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 25 00:08 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 24 23:55 debugprod.log
justIN time: 2025-11-03 18:41:14 UTC       justIN version: 01.05.01