justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263281.1@dunegpschedd02.fnal.gov

Jobsub ID263281.1@dunegpschedd02.fnal.gov
Workflow ID11257
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-18 16:44:58
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-12-18 21:08:49
From worker nodeHostnamewn2208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-18 16:46:07
Input filesmonte-carlo-011257-000001
JobscriptExit code0
Real time4h (15667s)
CPU time4h (15205s = 97%)
Max RSS bytes4362235904 (4160 MiB)
Outputting started2025-12-18 21:07:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11257/1/001/reco_stage1.root
Finished2025-12-18 21:08:49
Saved logsjustin-logs:263281.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 167492 traces, 0 summary
Retagger: tagging trace set: gauss with 127171 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 127171 traces tagged "gauss"
FrameSaver: q=4.41125e+06 n=2236621 tag=gauss
wclsFrameSaver: saving 167492 traces tagged "wiener"
FrameSaver: q=4.69943e+06 n=1976707 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 68844 XUs and 46843 XVs -> 930 XUVs
C:0 T:1 166701 XUs and 126178 XVs -> 13006 XUVs
C:0 T:2 135768 XUs and 118802 XVs -> 11181 XUVs
C:0 T:3 43828 XUs and 43629 XVs -> 249 XUVs
C:0 T:4 17976 XUs and 17825 XVs -> 240 XUVs
C:0 T:5 85700 XUs and 76508 XVs -> 10568 XUVs
C:0 T:6 78013 XUs and 77239 XVs -> 2721 XUVs
C:0 T:7 32331 XUs and 31708 XVs -> 131 XUVs
C:0 T:8 9298 XUs and 8873 XVs -> 179 XUVs
C:0 T:9 52639 XUs and 63331 XVs -> 4460 XUVs
C:0 T:10 174540 XUs and 119855 XVs -> 21952 XUVs
C:0 T:11 96 XUs and 98 XVs -> 5 XUVs
65622 XUVs total
15994 collection wire objects
61090 potential space points
Neighbour search...
10021668 tests to find 4343858 neighbours
Iterating with no regularization...
Begin: 9.6718e+07
0 7.0954e+07
1 6.93691e+07
2 6.90841e+07
3 6.89867e+07
4 6.89385e+07
Now with regularization...
Begin: 5.58775e+07
0 5.57441e+07
1 5.56433e+07
2 5.55623e+07
3 5.54945e+07
4 5.54384e+07
5 5.53906e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411196542
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411196542
fLifetime: 30000 [us]
18-Dec-2025 21:07:13 GMT  Closed output file "reco_stage1.root"
18-Dec-2025 21:07:13 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          405.493       495.467       627.832       469.884       77.7239        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00141196    0.00507748    0.00823954    0.00535089    0.00228957       10     
reco:ophit:OpHitFinder                             0.265278      0.395441      0.525605      0.414843      0.0912759       10     
reco:opflash:OpFlashFinder                         0.165867      0.243193      0.314103      0.242918      0.0495449       10     
reco:opslicer:OpSlicer                             0.510849      0.895919       1.23297      0.959864      0.227123        10     
reco:caldata:DataPrepModule                         4.53358       6.71315       7.98058       7.32496       1.33903        10     
reco:wclsdatasp:WireCellToolkit                     33.4383       41.3804       44.9848       42.8913       3.82912        10     
reco:gaushit:GausHitFinder                          1.08357       1.71944       2.25292       1.71239      0.391492        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000433355   0.000779113   0.00116718    0.000738233   0.000184835      10     
reco:reco3d:SpacePointSolver                        14.5711       31.1718       51.248        30.0321       10.5826        10     
reco:hitpdune:DisambigFromSpacePoints               1.56565       2.6838        3.96271       2.60409       0.79724        10     
reco:pandora:StandardPandora                        30.2662       51.5615       93.1613       41.1572       23.2504        10     
reco:pandoraWriter:StandardPandora                 0.374236      0.601822      0.879894      0.571043      0.154453        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.75682       5.48354       8.17255       4.77962       1.50605        10     
reco:pandoraShower:LArPandoraShowerCreation         3.96045       6.02508       10.7012       4.77639       2.28155        10     
reco:pandoracalo:Calorimetry                        2.33462       3.13038       4.18401       3.10876      0.665507        10     
reco:pandoracalonosce:Calorimetry                   1.96299       2.90726       4.26013       2.82371      0.705003        10     
reco:pandorapid:Chi2ParticleID                    0.00454026    0.00737452    0.00996966    0.00751515    0.00179758       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0472945     0.075279       0.11126      0.0719554     0.0219246       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00441624    0.00718105    0.00965339    0.00712937    0.00180642       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.49159       6.59185       11.3246       5.76242       2.11717        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.31926       6.40304       9.62289       6.17404       1.77548        10     
reco:emtrkmichelid:EmTrackMichelId                  222.944       261.952       314.523       251.109       29.476         10     
reco:crttag:SingleCRTMatchingProducer              0.0152963     0.023335      0.0508232     0.0201937    0.00979711       10     
reco:crtreco:TwoCRTMatchingProducer               0.00901615     0.0128043     0.0164947     0.0129453    0.00202726       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.987462       1.19854       1.48802       1.17644      0.146656        10     
reco:pandora2Track:LArPandoraTrackCreation          12.704        18.6285       27.2211       17.6924       4.19416        10     
reco:pandora2calo:Calorimetry                        6.502        9.72806       14.0829       9.26762       2.16244        10     
reco:pandora2calonosce:Calorimetry                  7.01888       9.2144        12.8683       8.75162       1.8085         10     
reco:pandora2pid:Chi2ParticleID                    0.0133519     0.0219926     0.0282935     0.0230242    0.00390103       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0694064     0.110851      0.130951      0.114468      0.017123        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0142609     0.0218947     0.0288382     0.0221593    0.00364606       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.95812       6.65786       8.87076       6.42696       1.10135        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.41406       8.1741        10.9852       7.96838       1.41732        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.62655       7.49685       9.72367       7.52345       1.30739        10     
[art]:TriggerResults:TriggerResultInserter        5.3691e-05    9.11564e-05   0.000181462   8.3192e-05    3.22116e-05      10     
end_path:out1:RootOutput                          1.1592e-05    1.55305e-05   2.3524e-05    1.5109e-05    2.88578e-06      10     
end_path:out1:RootOutput(write)                     3.44431       4.08255       4.73478       4.10428      0.348147        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5214.79 MB
  Peak resident set size usage (VmHWM): 4362.24 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.17 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10812092
-rw-r--r-- 1 dune01 dune01 2529845668 Dec 18 19:42 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2398318371 Dec 18 19:39 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2262142907 Dec 18 21:07 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1938694062 Dec 18 17:09 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1856594415 Dec 18 16:53 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   85363650 Dec 18 21:00 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243085 Dec 18 21:07 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec 18 21:07 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec 18 21:07 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec 18 21:07 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec 18 19:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 18 17:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec 18 21:07 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec 18 16:46 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec 18 21:07 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec 18 19:50 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec 18 19:42 debugprod.log
justIN time: 2025-12-19 05:13:39 UTC       justIN version: 01.05.03