justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 232696.0@dunegpschedd02.fnal.gov

Jobsub ID232696.0@dunegpschedd02.fnal.gov
Workflow ID9320
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 18:47:25
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-10-24 23:33:41
From worker nodeHostnamewn2208240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 18:49:14
Input filesmonte-carlo-009320-000001
JobscriptExit code0
Real time4h (16966s)
CPU time4h (16259s = 95%)
Max RSS bytes4500287488 (4291 MiB)
Outputting started2025-10-24 23:32:01
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09320/1/001/reco_stage1.root
Finished2025-10-24 23:33:41
Saved logsjustin-logs:232696.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 162793 traces, 0 summary
Retagger: tagging trace set: gauss with 125924 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125924 traces tagged "gauss"
FrameSaver: q=4.08387e+06 n=2207529 tag=gauss
wclsFrameSaver: saving 162793 traces tagged "wiener"
FrameSaver: q=4.31589e+06 n=1935534 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37185 XUs and 43530 XVs -> 429 XUVs
C:0 T:1 85594 XUs and 90837 XVs -> 6408 XUVs
C:0 T:2 113738 XUs and 107917 XVs -> 6458 XUVs
C:0 T:3 52191 XUs and 41340 XVs -> 442 XUVs
C:0 T:4 12853 XUs and 14811 XVs -> 115 XUVs
C:0 T:5 69954 XUs and 68407 XVs -> 8414 XUVs
C:0 T:6 141447 XUs and 119857 XVs -> 6406 XUVs
C:0 T:7 55285 XUs and 43766 XVs -> 423 XUVs
C:0 T:8 9460 XUs and 10998 XVs -> 317 XUVs
C:0 T:9 113723 XUs and 149087 XVs -> 26034 XUVs
C:0 T:10 50151 XUs and 31186 XVs -> 4644 XUVs
C:0 T:11 98 XUs and 48 XVs -> 5 XUVs
60095 XUVs total
16325 collection wire objects
55194 potential space points
Neighbour search...
10850468 tests to find 4591594 neighbours
Iterating with no regularization...
Begin: 6.28571e+07
0 5.18358e+07
1 5.10544e+07
2 5.09324e+07
3 5.08945e+07
Now with regularization...
Begin: 4.02146e+07
0 4.01505e+07
1 4.01042e+07
2 4.0068e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410091949
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410091949
fLifetime: 30000 [us]
25-Oct-2025 00:31:58 BST  Closed output file "reco_stage1.root"
25-Oct-2025 00:31:58 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          425.503       559.415       724.405       554.982       79.7988        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00174479    0.00958368     0.0497819    0.00526957     0.0135723       10     
reco:ophit:OpHitFinder                             0.363405      0.408405      0.476746      0.411234      0.0311019       10     
reco:opflash:OpFlashFinder                          0.20965      0.245651      0.305036       0.22956      0.0328485       10     
reco:opslicer:OpSlicer                             0.805425      0.959594       1.17877      0.941147      0.126436        10     
reco:caldata:DataPrepModule                         7.41924       7.83367       8.68798       7.8297       0.342083        10     
reco:wclsdatasp:WireCellToolkit                     44.1865       47.5945       53.6317       46.9203       3.09698        10     
reco:gaushit:GausHitFinder                          1.51611       1.81249       2.10153       1.82956      0.151061        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000738197   0.000928984    0.0015235    0.000864839   0.00021742       10     
reco:reco3d:SpacePointSolver                        18.1189       25.849        40.8759       23.7531       6.28909        10     
reco:hitpdune:DisambigFromSpacePoints               1.73988       2.5292        2.97073       2.66745      0.399168        10     
reco:pandora:StandardPandora                        34.6521       62.3066       108.081       60.7146       20.0006        10     
reco:pandoraWriter:StandardPandora                 0.626414      0.752117      0.836719      0.760362      0.0581204       10     
reco:pandoraTrack:LArPandoraTrackCreation           6.64561       9.85533       14.5236       8.86813       2.33108        10     
reco:pandoraShower:LArPandoraShowerCreation         7.30833       11.3435       17.7127       10.5966       2.86975        10     
reco:pandoracalo:Calorimetry                        4.53044       5.72878       7.74081       5.23253       1.1398         10     
reco:pandoracalonosce:Calorimetry                   3.73302       5.42385       7.47228       4.99838       1.33502        10     
reco:pandorapid:Chi2ParticleID                    0.00791587    0.00925658     0.0115757    0.00923141    0.00102857       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0745159     0.090391      0.106893      0.0892312     0.0110713       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00746528    0.00911958     0.0116186     0.0090396    0.00126556       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.62111       11.994        18.5922       11.1705       3.22499        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.55268       11.4039       17.8666       10.7654       3.29064        10     
reco:emtrkmichelid:EmTrackMichelId                  211.682       250.291       289.754       254.348       22.6877        10     
reco:crttag:SingleCRTMatchingProducer              0.0257311     0.0389219     0.0565825     0.038653      0.0105698       10     
reco:crtreco:TwoCRTMatchingProducer                0.0143119     0.0206773     0.0285569     0.0196901    0.00394795       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.952534       1.20094       1.43606       1.18801      0.138365        10     
reco:pandora2Track:LArPandoraTrackCreation          17.8864       30.0518       44.2361       28.3798       7.19864        10     
reco:pandora2calo:Calorimetry                       9.73556       15.995        24.7938       14.938        4.04735        10     
reco:pandora2calonosce:Calorimetry                  8.96032       15.441        22.9657       14.9442       3.66537        10     
reco:pandora2pid:Chi2ParticleID                    0.0175638     0.0226172     0.0274095     0.0233669    0.00313673       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0929707      0.11286      0.132988       0.10991      0.0136728       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0163494     0.0213322     0.0261097     0.0216142    0.00307822       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.82862       10.6654       15.6233       10.5595       2.48311        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.32975       13.4729       19.6492       13.1286       3.27195        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.75663       11.5423       18.1417       11.2345       3.21444        10     
[art]:TriggerResults:TriggerResultInserter        7.7836e-05    0.000115881   0.000265028   9.6246e-05    5.1809e-05       10     
end_path:out1:RootOutput                          1.2243e-05    1.45612e-05   2.3123e-05     1.326e-05    3.09937e-06      10     
end_path:out1:RootOutput(write)                     3.4644        4.14118       4.83251       4.15561      0.379985        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5350.09 MB
  Peak resident set size usage (VmHWM): 4500.29 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.91 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11377528
-rw-r--r-- 1 dune01 dune01 2625356223 Oct 24 22:55 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2496248457 Oct 24 22:52 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2337622015 Oct 25 00:31 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2089841960 Oct 24 20:17 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2019786134 Oct 24 19:58 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   81102470 Oct 25 00:23 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243079 Oct 25 00:32 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 25 00:31 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 25 00:31 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 25 00:31 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 24 22:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 24 20:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 25 00:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 24 19:49 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 25 00:32 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 24 23:02 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 24 22:55 debugprod.log
justIN time: 2025-11-06 01:57:57 UTC       justIN version: 01.05.01