justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255201.1@dunegpschedd01.fnal.gov

Jobsub ID255201.1@dunegpschedd01.fnal.gov
Workflow ID10508
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-22 03:26:07
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-22 09:08:59
From worker nodeHostnamewn1208241.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-22 03:26:48
Input filesmonte-carlo-010508-000001
JobscriptExit code0
Real time5h (19747s)
CPU time5h (19055s = 96%)
Max RSS bytes4168622080 (3975 MiB)
Outputting started2025-11-22 08:55:56
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10508/1/001/reco_stage1.root
Finished2025-11-22 09:08:59
Saved logsjustin-logs:255201.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

ol adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 130574 traces, 0 summary
Retagger: tagging trace set: gauss with 96996 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96996 traces tagged "gauss"
FrameSaver: q=3.54492e+06 n=1710178 tag=gauss
wclsFrameSaver: saving 130574 traces tagged "wiener"
FrameSaver: q=3.76645e+06 n=1507756 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31371 XUs and 22656 XVs -> 313 XUVs
C:0 T:1 60105 XUs and 45300 XVs -> 8004 XUVs
C:0 T:2 114088 XUs and 100545 XVs -> 5254 XUVs
C:0 T:3 56777 XUs and 44063 XVs -> 324 XUVs
C:0 T:4 7527 XUs and 9270 XVs -> 87 XUVs
C:0 T:5 24841 XUs and 34203 XVs -> 2879 XUVs
C:0 T:6 107642 XUs and 110817 XVs -> 4493 XUVs
C:0 T:7 44733 XUs and 41731 XVs -> 279 XUVs
C:0 T:8 13520 XUs and 10388 XVs -> 173 XUVs
C:0 T:9 99526 XUs and 106628 XVs -> 7828 XUVs
C:0 T:10 45101 XUs and 18387 XVs -> 7668 XUVs
C:0 T:11 23 XUs and 10 XVs -> 1 XUVs
37303 XUVs total
11741 collection wire objects
34475 potential space points
Neighbour search...
2541453 tests to find 1301628 neighbours
Iterating with no regularization...
Begin: 1.02891e+08
0 8.33845e+07
1 8.15991e+07
2 8.14173e+07
3 8.13651e+07
Now with regularization...
Begin: 6.94248e+07
0 6.93774e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410662422
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410662422
fLifetime: 30000 [us]
22-Nov-2025 08:55:51 GMT  Closed output file "reco_stage1.root"
22-Nov-2025 08:55:51 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          482.519       682.488       853.75        678.721       101.151        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00141059    0.00498902    0.00741115    0.00569638    0.00194803       10     
reco:ophit:OpHitFinder                              0.32961      0.444185      0.561008      0.443023      0.0639882       10     
reco:opflash:OpFlashFinder                         0.198406      0.263873      0.338363      0.249929      0.0489557       10     
reco:opslicer:OpSlicer                             0.877131       1.21211       1.43871       1.31202       0.20296        10     
reco:caldata:DataPrepModule                         7.94018       8.85965       9.38458       8.8019       0.394911        10     
reco:wclsdatasp:WireCellToolkit                      50.61        59.0427       65.6816       59.6843       4.43045        10     
reco:gaushit:GausHitFinder                          1.48615       2.03162       2.47854       2.04661      0.321336        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000661755   0.00112428    0.00182155    0.00110469    0.000310547      10     
reco:reco3d:SpacePointSolver                        11.2176       25.9054       37.5542       25.4982       6.44158        10     
reco:hitpdune:DisambigFromSpacePoints               1.81613       2.87971       3.97833       2.83844      0.613299        10     
reco:pandora:StandardPandora                        37.6268       63.5652       104.697       60.068        16.8379        10     
reco:pandoraWriter:StandardPandora                  0.65882      0.839513      0.988407      0.812991      0.110332        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.78956       9.11376       12.2364       8.67853       1.98326        10     
reco:pandoraShower:LArPandoraShowerCreation         4.70297       9.68086       13.2425       9.98739       2.56051        10     
reco:pandoracalo:Calorimetry                        2.80623       4.96677       5.96436       5.63243       1.12109        10     
reco:pandoracalonosce:Calorimetry                   2.38662       4.68757       6.60496       4.54171       1.3277         10     
reco:pandorapid:Chi2ParticleID                    0.00927507     0.0105822     0.0129377     0.0100032    0.00129714       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0799128     0.104827      0.124791      0.106357      0.013146        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00852711     0.0102822     0.0119082     0.010161     0.00117964       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.97159       10.3088       15.383        10.2242       3.32488        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.68258       10.446        16.7676       10.2212       4.08998        10     
reco:emtrkmichelid:EmTrackMichelId                  287.413       373.351       425.286       370.339       43.7016        10     
reco:crttag:SingleCRTMatchingProducer              0.0131081     0.0216314     0.0320095     0.0212106    0.00465182       10     
reco:crtreco:TwoCRTMatchingProducer                0.0127849     0.0196157     0.0244646     0.0199525    0.00391043       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.14205       1.43106       1.82812       1.42174      0.205942        10     
reco:pandora2Track:LArPandoraTrackCreation          12.1849       27.7531       40.0292       28.7619       7.17694        10     
reco:pandora2calo:Calorimetry                       7.12475       15.3527       21.9193       14.6981       4.16914        10     
reco:pandora2calonosce:Calorimetry                  6.53062       13.1381       20.2944       13.7767       3.98008        10     
reco:pandora2pid:Chi2ParticleID                    0.0203424     0.0253969     0.0348883     0.0250195    0.00389252       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.106351       0.13242      0.151453      0.135337      0.0142052       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0202583     0.0253939     0.0330334     0.0249561    0.00330196       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.16864       9.22524       11.7377       9.89381       2.04937        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.08532       11.9795       17.8133       12.2475       2.93858        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.37995       10.5395       15.1424       10.2722       2.63787        10     
[art]:TriggerResults:TriggerResultInserter        4.4022e-05    6.20947e-05   0.000161264   5.24335e-05   3.34454e-05      10     
end_path:out1:RootOutput                           6.943e-06    1.23663e-05   4.4965e-05     8.381e-06    1.10261e-05      10     
end_path:out1:RootOutput(write)                     3.62205       4.89686       5.99349       4.8423       0.689015        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5018.14 MB
  Peak resident set size usage (VmHWM): 4168.62 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10161636
-rw-r--r-- 1 dune01 dune01 2384754933 Nov 22 06:58 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2263516453 Nov 22 06:55 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2067459341 Nov 22 08:55 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1849684968 Nov 22 03:56 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1765128412 Nov 22 03:36 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   74338350 Nov 22 08:47 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242801 Nov 22 08:55 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 22 08:55 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 22 08:55 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 22 08:55 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 22 06:58 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 22 03:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 22 08:55 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 22 03:26 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 22 08:55 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 22 07:10 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 22 06:58 debugprod.log
justIN time: 2026-02-08 22:54:22 UTC       justIN version: 01.06.00