justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270291.1@dunegpschedd01.fnal.gov

Jobsub ID270291.1@dunegpschedd01.fnal.gov
Workflow ID11347
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-26 18:47:47
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce2
Last heartbeat2025-12-26 22:43:37
From worker nodeHostnamewn012.hep
cpuinfoIntel(R) Core(TM) i7-5960X CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-26 18:48:37
Input filesmonte-carlo-011347-000001
JobscriptExit code0
Real time3h (13034s)
CPU time3h (12709s = 97%)
Max RSS bytes4533612544 (4323 MiB)
Outputting started2025-12-26 22:25:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11347/1/001/reco_stage1.root
Finished2025-12-26 22:43:37
Saved logsjustin-logs:270291.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148891 traces, 0 summary
Retagger: tagging trace set: gauss with 110994 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110994 traces tagged "gauss"
FrameSaver: q=3.92834e+06 n=1933819 tag=gauss
wclsFrameSaver: saving 148891 traces tagged "wiener"
FrameSaver: q=4.16544e+06 n=1704910 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 41152 XUs and 47627 XVs -> 868 XUVs
C:0 T:1 92663 XUs and 107450 XVs -> 10810 XUVs
C:0 T:2 76885 XUs and 106240 XVs -> 4633 XUVs
C:0 T:3 36536 XUs and 41975 XVs -> 191 XUVs
C:0 T:4 14612 XUs and 16001 XVs -> 195 XUVs
C:0 T:5 73674 XUs and 73934 XVs -> 5269 XUVs
C:0 T:6 43180 XUs and 47134 XVs -> 1882 XUVs
C:0 T:7 19788 XUs and 18676 XVs -> 128 XUVs
C:0 T:8 11222 XUs and 11257 XVs -> 240 XUVs
C:0 T:9 62316 XUs and 73292 XVs -> 4295 XUVs
C:0 T:10 51915 XUs and 39660 XVs -> 4810 XUVs
C:0 T:11 178 XUs and 170 XVs -> 13 XUVs
33334 XUVs total
12488 collection wire objects
30087 potential space points
Neighbour search...
1778239 tests to find 910734 neighbours
Iterating with no regularization...
Begin: 8.16471e+07
0 6.53196e+07
1 6.41983e+07
2 6.40744e+07
3 6.40424e+07
Now with regularization...
Begin: 5.29516e+07
0 5.2927e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411359252
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411359252
fLifetime: 30000 [us]
26-Dec-2025 22:25:49 GMT  Closed output file "reco_stage1.root"
26-Dec-2025 22:25:49 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          333.385       490.784       653.538       487.457       106.777        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00106274    0.00316979    0.00573004    0.00310387    0.00133124       10     
reco:ophit:OpHitFinder                             0.206066      0.284523      0.447583       0.24613      0.0875741       10     
reco:opflash:OpFlashFinder                          0.12995      0.195612      0.298396      0.196187      0.0488577       10     
reco:opslicer:OpSlicer                             0.519374      0.760095       1.37403      0.653116      0.267415        10     
reco:caldata:DataPrepModule                         4.78388       5.87404       8.2844        4.92473       1.48618        10     
reco:wclsdatasp:WireCellToolkit                     51.969        57.7765       75.3784       54.1813       7.55364        10     
reco:gaushit:GausHitFinder                         0.993005       1.65084       2.55123       1.55845      0.498629        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000456351   0.000686111   0.000993776   0.000645209   0.000165839      10     
reco:reco3d:SpacePointSolver                        9.12338       18.0232       27.0216       16.8128       6.74775        10     
reco:hitpdune:DisambigFromSpacePoints               1.49914       2.83761       5.86066       2.56189       1.18984        10     
reco:pandora:StandardPandora                        24.7654       55.4464       147.308       46.208        34.3112        10     
reco:pandoraWriter:StandardPandora                 0.305661      0.504577      0.855804      0.410791      0.198233        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.92469       7.38015       15.4924       6.47494       3.44849        10     
reco:pandoraShower:LArPandoraShowerCreation         2.59975       7.03698       14.9238       6.06426       3.88019        10     
reco:pandoracalo:Calorimetry                         2.166        3.64748       7.57565       3.07539       1.6327         10     
reco:pandoracalonosce:Calorimetry                   1.9502        3.34959       7.12638       2.77749       1.5656         10     
reco:pandorapid:Chi2ParticleID                    0.00476375    0.00890148     0.0243746    0.00616223    0.00570243       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0533814     0.0804004     0.132484      0.0670273     0.0278444       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0046001    0.00872975     0.0250102    0.00609016    0.00585102       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.61261       7.24278       15.6439       6.07952       4.02467        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.50907       6.63838       12.7202       5.95221       3.06897        10     
reco:emtrkmichelid:EmTrackMichelId                  168.573       239.347       334.785       237.103       51.689         10     
reco:crttag:SingleCRTMatchingProducer              0.0123042     0.0182465     0.0243661     0.0175197    0.00364442       10     
reco:crtreco:TwoCRTMatchingProducer               0.00987387     0.0159601     0.0254927     0.0153976    0.00392191       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.43735       2.03964       2.54418       1.95024       0.36601        10     
reco:pandora2Track:LArPandoraTrackCreation          12.8241       24.6723       37.0137       25.5043       7.29954        10     
reco:pandora2calo:Calorimetry                       5.36313       10.3285       15.6486       10.9448       3.03687        10     
reco:pandora2calonosce:Calorimetry                  4.68775       9.33728       12.0574       10.5301       2.60712        10     
reco:pandora2pid:Chi2ParticleID                    0.0110233     0.0182112     0.0274621     0.0172085    0.00494105       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0649046     0.0977117     0.136545      0.0961306     0.0238426       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0109369     0.0183007     0.0278449     0.0173823     0.0050139       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.56212       7.27255       9.04867       7.5065        1.73156        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.36901       8.31788       11.1018       8.73056       2.28121        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.11697       7.58376       10.9487       7.88892       2.22168        10     
[art]:TriggerResults:TriggerResultInserter        2.7827e-05    4.04387e-05   0.000120573   3.03905e-05   2.70546e-05      10     
end_path:out1:RootOutput                           3.758e-06    5.1726e-06    1.0679e-05    4.5265e-06    1.91174e-06      10     
end_path:out1:RootOutput(write)                     2.00754       2.86295       4.5242        2.58912      0.883811        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5336.54 MB
  Peak resident set size usage (VmHWM): 4533.61 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.85 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10705372
-rw-r--r--. 1 dune004 dune 2495373594 Dec 26 21:01 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2368682125 Dec 26 20:58 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 2203345591 Dec 26 22:25 reco_stage1.root
-rw-r--r--. 1 dune004 dune 1946973586 Dec 26 19:11 g4_stage2.root
-rw-r--r--. 1 dune004 dune 1867010981 Dec 26 18:57 g4_stage1.root
-rw-r--r--. 1 dune004 dune   80231490 Dec 26 22:20 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     243335 Dec 26 22:25 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Dec 26 22:25 mem.db
-rw-r--r--. 1 dune004 dune      62376 Dec 26 22:25 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Dec 26 22:25 time.db
-rw-r--r--. 1 dune004 dune        519 Dec 26 21:01 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 26 19:11 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 26 22:25 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Dec 26 18:48 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Dec 26 22:25 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Dec 26 21:10 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Dec 26 21:01 debugprod.log
justIN time: 2026-02-07 03:41:20 UTC       justIN version: 01.06.00