justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 251502.1@dunegpschedd01.fnal.gov

Jobsub ID251502.1@dunegpschedd01.fnal.gov
Workflow ID10211
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-15 08:06:33
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-15 14:06:32
From worker nodeHostnamewn2207251.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-15 08:07:13
Input filesmonte-carlo-010211-000001
JobscriptExit code0
Real time5h (21456s)
CPU time5h (20402s = 95%)
Max RSS bytes3972947968 (3788 MiB)
Outputting started2025-11-15 14:04:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10211/1/001/reco_stage1.root
Finished2025-11-15 14:06:32
Saved logsjustin-logs:251502.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

KeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 113119 traces, 0 summary
Retagger: tagging trace set: gauss with 88923 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88923 traces tagged "gauss"
FrameSaver: q=3.0962e+06 n=1579509 tag=gauss
wclsFrameSaver: saving 113119 traces tagged "wiener"
FrameSaver: q=3.28526e+06 n=1413302 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27766 XUs and 24644 XVs -> 505 XUVs
C:0 T:1 68288 XUs and 58889 XVs -> 5512 XUVs
C:0 T:2 42799 XUs and 74292 XVs -> 2923 XUVs
C:0 T:3 22450 XUs and 28779 XVs -> 243 XUVs
C:0 T:4 15154 XUs and 14680 XVs -> 167 XUVs
C:0 T:5 58126 XUs and 50458 XVs -> 7054 XUVs
C:0 T:6 65890 XUs and 70109 XVs -> 2783 XUVs
C:0 T:7 30766 XUs and 30642 XVs -> 556 XUVs
C:0 T:8 9383 XUs and 9638 XVs -> 203 XUVs
C:0 T:9 57604 XUs and 77654 XVs -> 6485 XUVs
C:0 T:10 43981 XUs and 20243 XVs -> 3749 XUVs
C:0 T:11 50 XUs and 19 XVs -> 3 XUVs
30183 XUVs total
10928 collection wire objects
27653 potential space points
Neighbour search...
3031393 tests to find 1230774 neighbours
Iterating with no regularization...
Begin: 7.86561e+07
0 6.23783e+07
1 6.09396e+07
2 6.06918e+07
3 6.0632e+07
Now with regularization...
Begin: 4.7847e+07
0 4.77522e+07
1 4.77213e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410525527
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410525527
fLifetime: 30000 [us]
15-Nov-2025 14:04:32 GMT  Closed output file "reco_stage1.root"
15-Nov-2025 14:04:32 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          401.61        727.37        1083.52       749.566       195.55         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00204275    0.00444278     0.0118252    0.00354915    0.00267586       10     
reco:ophit:OpHitFinder                              0.31625      0.467104      0.595155      0.443691      0.0858145       10     
reco:opflash:OpFlashFinder                         0.170228      0.253111      0.380313      0.219281      0.0744223       10     
reco:opslicer:OpSlicer                             0.784941       1.10807       1.55084       1.08876      0.244925        10     
reco:caldata:DataPrepModule                         9.74338       10.7412       11.7313       10.7815      0.520596        10     
reco:wclsdatasp:WireCellToolkit                     79.525        84.0669       89.9585       83.3448       3.39436        10     
reco:gaushit:GausHitFinder                          1.17503       2.23049       3.00685       2.29732      0.587655        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00062011    0.000914021   0.00113263    0.00092029    0.000153753      10     
reco:reco3d:SpacePointSolver                        11.0551       23.6588       44.7325       22.6007       10.4837        10     
reco:hitpdune:DisambigFromSpacePoints               1.39817       3.61694       6.69943       3.55144       1.55582        10     
reco:pandora:StandardPandora                        22.5528       61.0541       130.94        63.5147       29.7901        10     
reco:pandoraWriter:StandardPandora                  0.49449      0.783474       1.0566       0.767671      0.173183        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.92988       12.3396       17.7771       12.6367       3.50991        10     
reco:pandoraShower:LArPandoraShowerCreation         4.32679       13.0315       21.5624       13.9401       5.12117        10     
reco:pandoracalo:Calorimetry                        2.60804       5.93155       8.33643       6.15236       1.80526        10     
reco:pandoracalonosce:Calorimetry                   2.39576       5.48489       7.90622       5.51298       1.6955         10     
reco:pandorapid:Chi2ParticleID                    0.00887023     0.0114206     0.0147213     0.0113697    0.00193886       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0645753     0.109584      0.164259      0.106872      0.0294756       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00725321    0.00992245     0.0126205    0.00985657     0.0017096       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.82902       12.8868       23.4214       13.566        5.76544        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.69189       12.6265       22.5853       13.8628       5.55083        10     
reco:emtrkmichelid:EmTrackMichelId                  204.248       353.106       505.735       355.294       84.8825        10     
reco:crttag:SingleCRTMatchingProducer              0.0300185     0.138244       0.60738      0.0481121     0.175461        10     
reco:crtreco:TwoCRTMatchingProducer                0.0199437     0.0304588     0.0374283     0.0345056    0.00694234       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.880382       1.75442       2.65565       1.55736       0.54556        10     
reco:pandora2Track:LArPandoraTrackCreation          14.1874       40.7473       67.2191       42.2608       15.3938        10     
reco:pandora2calo:Calorimetry                       5.7102        18.0465       32.3759       18.6036       7.61082        10     
reco:pandora2calonosce:Calorimetry                  5.28497       16.9299       27.8949       17.7506       6.91428        10     
reco:pandora2pid:Chi2ParticleID                    0.0146744     0.0267437     0.0358532     0.0277121    0.00531282       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0762946     0.129623      0.181867      0.139993      0.0306611       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.015201      0.0243832     0.0323644     0.024695     0.00507561       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.47749       12.7884       21.1312       13.9614       4.84539        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.93694       14.9376       24.7158       15.8003       6.11907        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.12488       13.5191       24.6825       14.2195       5.91919        10     
[art]:TriggerResults:TriggerResultInserter        7.5799e-05    0.000122996   0.000212981   0.000114535   3.53845e-05      10     
end_path:out1:RootOutput                          1.1122e-05    1.52061e-05   2.3578e-05    1.4113e-05    3.32101e-06      10     
end_path:out1:RootOutput(write)                     2.97224       4.56207       6.64951       4.48719       1.10922        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4793.43 MB
  Peak resident set size usage (VmHWM): 3972.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.41 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.44 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.02 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9351584
-rw-r--r-- 1 dune01 dune01 2221625386 Nov 15 11:57 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2109692138 Nov 15 11:50 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1884652488 Nov 15 14:04 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1682201029 Nov 15 08:39 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1608130759 Nov 15 08:16 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   69084530 Nov 15 13:55 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242917 Nov 15 14:04 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 15 14:04 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 15 14:04 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 15 14:04 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 15 11:57 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 15 08:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 15 14:04 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 15 08:07 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 15 14:04 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 15 12:06 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 15 11:57 debugprod.log
justIN time: 2026-02-09 12:23:41 UTC       justIN version: 01.06.00