justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269133.0@dunegpschedd01.fnal.gov

Jobsub ID269133.0@dunegpschedd01.fnal.gov
Workflow ID11266
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-18 22:20:15
SiteUK_Bristol
EntryCMSHTPC_T2_UK_SGrid_Bristol_lcgce02
Last heartbeat2025-12-19 04:28:32
From worker nodeHostnamehd68.dice.priv
cpuinfoAMD EPYC 7551P 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit259200 (72 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-18 22:21:40
Input filesmonte-carlo-011266-000001
JobscriptExit code0
Real time6h (21921s)
CPU time5h (21384s = 97%)
Max RSS bytes4370948096 (4168 MiB)
Outputting started2025-12-19 04:27:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11266/1/001/reco_stage1.root
Finished2025-12-19 04:28:32
Saved logsjustin-logs:269133.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

KeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 83469 traces, 0 summary
Retagger: tagging trace set: gauss with 64651 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 64651 traces tagged "gauss"
FrameSaver: q=2.0913e+06 n=1111129 tag=gauss
wclsFrameSaver: saving 83469 traces tagged "wiener"
FrameSaver: q=2.21169e+06 n=977629 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 24034 XUs and 19138 XVs -> 238 XUVs
C:0 T:1 40423 XUs and 32640 XVs -> 2705 XUVs
C:0 T:2 47303 XUs and 95735 XVs -> 3769 XUVs
C:0 T:3 25290 XUs and 34903 XVs -> 255 XUVs
C:0 T:4 5587 XUs and 4454 XVs -> 17 XUVs
C:0 T:5 22934 XUs and 14211 XVs -> 2812 XUVs
C:0 T:6 66080 XUs and 76679 XVs -> 3216 XUVs
C:0 T:7 26718 XUs and 33214 XVs -> 144 XUVs
C:0 T:8 3940 XUs and 5535 XVs -> 136 XUVs
C:0 T:9 25380 XUs and 41537 XVs -> 2450 XUVs
C:0 T:10 15914 XUs and 10699 XVs -> 1726 XUVs
C:0 T:11 304 XUs and 204 XVs -> 38 XUVs
17506 XUVs total
7390 collection wire objects
16298 potential space points
Neighbour search...
874090 tests to find 467470 neighbours
Iterating with no regularization...
Begin: 5.17321e+07
0 4.33973e+07
1 4.30072e+07
2 4.29429e+07
3 4.29225e+07
Now with regularization...
Begin: 3.64902e+07
0 3.64834e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411200960
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411200960
fLifetime: 30000 [us]
19-Dec-2025 04:26:59 UTC  Closed output file "reco_stage1.root"
19-Dec-2025 04:26:59 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          311.987       633.648       910.212       660.748       171.51         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00110897    0.00476063    0.00738048    0.00533495     0.0020063       10     
reco:ophit:OpHitFinder                             0.296609      0.407309      0.474176      0.412381      0.0442975       10     
reco:opflash:OpFlashFinder                         0.178642       0.29922      0.364178       0.30069      0.047058        10     
reco:opslicer:OpSlicer                             0.867927       1.15979        1.343        1.16145      0.128533        10     
reco:caldata:DataPrepModule                         7.29359       7.61528       7.9869        7.6104       0.263961        10     
reco:wclsdatasp:WireCellToolkit                     45.672        51.2234       64.2225       48.2904       6.41837        10     
reco:gaushit:GausHitFinder                          1.13533       1.97806       2.58204       2.00796      0.404578        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000416285   0.000744202   0.00108724    0.000737355   0.00020784       10     
reco:reco3d:SpacePointSolver                        7.2965        27.8047       50.2806       24.9244       12.7047        10     
reco:hitpdune:DisambigFromSpacePoints               1.05393       2.97703       4.67456       3.31817       1.01281        10     
reco:pandora:StandardPandora                        20.3466       75.5778       130.695       62.7643       39.9644        10     
reco:pandoraWriter:StandardPandora                 0.405757      0.687935      0.908908       0.72072      0.143879        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.33659       9.11467       14.2076       9.45094       2.91741        10     
reco:pandoraShower:LArPandoraShowerCreation         2.88941       10.6229       18.7371       10.7543       4.35795        10     
reco:pandoracalo:Calorimetry                        1.73245       5.0056        7.68552       5.09613       1.66531        10     
reco:pandoracalonosce:Calorimetry                   1.59523       4.61023       7.28015       4.76217       1.51226        10     
reco:pandorapid:Chi2ParticleID                     0.0070322    0.00996447     0.0114701     0.0101449    0.00127979       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0563612     0.0988192     0.125779      0.101982      0.0206748       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00626188    0.00931488     0.0107574    0.00946876     0.0013173       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.82963       10.8216       19.3768       11.3016       4.44808        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.61866       10.3413       18.9136       10.8244       4.61076        10     
reco:emtrkmichelid:EmTrackMichelId                  180.808       315.939       415.352       330.353       65.5705        10     
reco:crttag:SingleCRTMatchingProducer              0.0128942     0.0253736     0.0340892     0.0274427     0.0060584       10     
reco:crtreco:TwoCRTMatchingProducer               0.00978026     0.0176048     0.0231704     0.0185493    0.00427317       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.971084       1.49217       1.91316       1.58065      0.321825        10     
reco:pandora2Track:LArPandoraTrackCreation          9.2064        28.8232       50.8091       30.2466       11.2337        10     
reco:pandora2calo:Calorimetry                       4.43587       14.4573       25.4129       15.0389       5.61465        10     
reco:pandora2calonosce:Calorimetry                  4.07255       14.1103       25.9377       14.0655       5.98246        10     
reco:pandora2pid:Chi2ParticleID                    0.0145566     0.0248625     0.0327227     0.025345     0.00509227       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0693988     0.125067      0.168525      0.130484      0.0274882       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0147166     0.0240987     0.0329496     0.0242065    0.00493693       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.31191       10.6289       18.4684       11.3517       4.21444        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.47521       12.0723       20.281        12.7018       4.70343        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.39295       10.8984       19.2055       11.5667       4.3726         10     
[art]:TriggerResults:TriggerResultInserter        6.8238e-05    8.9783e-05    0.000207461   7.1755e-05    4.02905e-05      10     
end_path:out1:RootOutput                          1.1913e-05    1.44414e-05   2.5789e-05    1.31495e-05   3.85619e-06      10     
end_path:out1:RootOutput(write)                     2.44522       4.46513       6.00486       4.70087       1.00523        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5186.58 MB
  Peak resident set size usage (VmHWM): 4370.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.55 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.73 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10960336
-rw-r--r-- 1 dnpil001 users 2540032484 Dec 19 02:38 detsim_stage2.root
-rw-r--r-- 1 dnpil001 users 2403574115 Dec 19 02:35 detsim_stage1.root
-rw-r--r-- 1 dnpil001 users 2270666475 Dec 19 04:26 reco_stage1.root
-rw-r--r-- 1 dnpil001 users 2005501817 Dec 18 22:52 g4_stage2.root
-rw-r--r-- 1 dnpil001 users 1917885786 Dec 18 22:29 g4_stage1.root
-rw-r--r-- 1 dnpil001 users   85107130 Dec 19 04:23 Pandora_Events.pndr
-rw-r--r-- 1 dnpil001 users     243058 Dec 19 04:27 jobscript.log
-rw-r--r-- 1 dnpil001 users     217088 Dec 19 04:26 mem.db
-rw-r--r-- 1 dnpil001 users      62376 Dec 19 04:26 Pandora_Geometry.xml
-rw-r--r-- 1 dnpil001 users      36864 Dec 19 04:26 time.db
-rw-r--r-- 1 dnpil001 users        519 Dec 19 02:38 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 18 22:52 g4_protoDUNE_hist.root
-rw-r--r-- 1 dnpil001 users        519 Dec 19 04:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dnpil001 users         52 Dec 18 22:21 all-input-dids.txt
-rw-r--r-- 1 dnpil001 users          7 Dec 19 04:27 justin-processed-pfns.txt
-rw-r--r-- 1 dnpil001 users          0 Dec 19 02:47 ProtoDUNE.log
-rw-r--r-- 1 dnpil001 users          0 Dec 19 02:38 debugprod.log
justIN time: 2026-02-07 08:00:11 UTC       justIN version: 01.06.00