justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264373.1@dunegpschedd01.fnal.gov

Jobsub ID264373.1@dunegpschedd01.fnal.gov
Workflow ID11063
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-06 11:33:58
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-12-06 18:00:40
From worker nodeHostnamewn1208090.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-06 11:35:06
Input filesmonte-carlo-011063-000001
JobscriptExit code0
Real time6h (23035s)
CPU time6h (22045s = 95%)
Max RSS bytes4520128512 (4310 MiB)
Outputting started2025-12-06 17:59:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11063/1/001/reco_stage1.root
Finished2025-12-06 18:00:40
Saved logsjustin-logs:264373.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

gnalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 136312 traces, 0 summary
Retagger: tagging trace set: gauss with 102727 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102727 traces tagged "gauss"
FrameSaver: q=3.7339e+06 n=1780760 tag=gauss
wclsFrameSaver: saving 136312 traces tagged "wiener"
FrameSaver: q=3.99246e+06 n=1567371 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34175 XUs and 23794 XVs -> 281 XUVs
C:0 T:1 67462 XUs and 52518 XVs -> 4222 XUVs
C:0 T:2 128354 XUs and 108214 XVs -> 9065 XUVs
C:0 T:3 54727 XUs and 43352 XVs -> 655 XUVs
C:0 T:4 13214 XUs and 9525 XVs -> 65 XUVs
C:0 T:5 49165 XUs and 26956 XVs -> 4929 XUVs
C:0 T:6 143075 XUs and 283927 XVs -> 14889 XUVs
C:0 T:7 52444 XUs and 61689 XVs -> 1161 XUVs
C:0 T:8 7746 XUs and 4702 XVs -> 50 XUVs
C:0 T:9 58066 XUs and 45958 XVs -> 5368 XUVs
C:0 T:10 47637 XUs and 228699 XVs -> 18169 XUVs
C:0 T:11 160 XUs and 415 XVs -> 35 XUVs
58889 XUVs total
11972 collection wire objects
52498 potential space points
Neighbour search...
7605296 tests to find 3539328 neighbours
Iterating with no regularization...
Begin: 9.0879e+07
0 6.6745e+07
1 6.52136e+07
2 6.49982e+07
3 6.49356e+07
Now with regularization...
Begin: 5.3515e+07
0 5.3397e+07
1 5.33481e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410949889
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410949889
fLifetime: 30000 [us]
06-Dec-2025 17:58:59 GMT  Closed output file "reco_stage1.root"
06-Dec-2025 17:58:59 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          608.527       798.053       1268.09       781.697       173.616        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00140244    0.00638152     0.018313     0.00568876    0.00443797       10     
reco:ophit:OpHitFinder                             0.369198      0.480978      0.582284      0.471894      0.0620147       10     
reco:opflash:OpFlashFinder                         0.235186      0.294766      0.364937      0.272508      0.0439502       10     
reco:opslicer:OpSlicer                              1.01391       1.41183       1.70452       1.44202       0.22855        10     
reco:caldata:DataPrepModule                         7.80365       8.94665       9.74137       9.01356      0.596896        10     
reco:wclsdatasp:WireCellToolkit                     54.2142       59.5889       62.9044       59.861        2.39006        10     
reco:gaushit:GausHitFinder                          1.97872       2.46771       3.19807       2.45176      0.360616        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000720118   0.00107847    0.00177802    0.00101884    0.000284348      10     
reco:reco3d:SpacePointSolver                        19.844        31.6653       75.7708       26.6793       15.3209        10     
reco:hitpdune:DisambigFromSpacePoints               2.23969       3.36854       5.6285        3.25046      0.937945        10     
reco:pandora:StandardPandora                        51.574        89.9524       237.569       73.7214       51.5458        10     
reco:pandoraWriter:StandardPandora                 0.674374      0.883223       1.25811      0.884602      0.157291        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.54348       11.0509       21.2627       10.038        3.57244        10     
reco:pandoraShower:LArPandoraShowerCreation         6.51042       12.0647       23.7564       11.5769       4.40326        10     
reco:pandoracalo:Calorimetry                        4.16655       5.86024       9.91957       5.50769       1.50249        10     
reco:pandoracalonosce:Calorimetry                   3.81691       5.48412       8.89622       5.06248       1.36035        10     
reco:pandorapid:Chi2ParticleID                     0.0104768     0.0124911     0.0166524     0.0118728    0.00185663       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0950076     0.118546      0.158318       0.11613      0.0182498       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0103373     0.0121192     0.0167086     0.0115973    0.00172571       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.04933       12.5276       23.623        12.1998       4.42248        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.56105       11.6774       22.2086       10.1856       4.24787        10     
reco:emtrkmichelid:EmTrackMichelId                  339.84        428.831       585.681       432.669       64.1249        10     
reco:crttag:SingleCRTMatchingProducer              0.0161054     0.0332333     0.0578706     0.0331232     0.0129044       10     
reco:crtreco:TwoCRTMatchingProducer                0.0140111     0.0217091     0.0300119     0.0209364     0.0047427       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.32358       1.63469       2.04755       1.64005      0.212306        10     
reco:pandora2Track:LArPandoraTrackCreation          21.9284       34.0478        59.41        33.2526       9.66399        10     
reco:pandora2calo:Calorimetry                       10.7033       17.1134       27.3671       17.2642       4.44256        10     
reco:pandora2calonosce:Calorimetry                  10.2599       16.1088       24.4997       15.885        3.88603        10     
reco:pandora2pid:Chi2ParticleID                    0.0184152     0.028787      0.0388545     0.0289243    0.00507797       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0849564     0.145574      0.194222      0.143072      0.0300545       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0245489     0.0296772     0.0386563     0.0296858    0.00378461       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.53182       11.6135       18.1245       11.566        2.89227        10     
reco:pandora2Showercalo:ShowerCalorimetry            6.427        13.0781       18.455        13.0663       3.10461        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.64234        11.81        20.8655       11.837        3.84982        10     
[art]:TriggerResults:TriggerResultInserter        3.5617e-05    5.84704e-05   0.000150484   5.00295e-05   3.09929e-05      10     
end_path:out1:RootOutput                           7.053e-06    1.07443e-05    3.213e-05    7.6845e-06    7.36251e-06      10     
end_path:out1:RootOutput(write)                      3.556        5.46455       6.96826       5.63153      0.856067        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5367.83 MB
  Peak resident set size usage (VmHWM): 4520.13 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.78 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10959940
-rw-r--r-- 1 dune01 dune01 2562119046 Dec  6 15:41 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2434996040 Dec  6 15:37 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2282782386 Dec  6 17:58 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1961516094 Dec  6 12:09 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1897499700 Dec  6 11:46 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   83435570 Dec  6 17:50 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243127 Dec  6 17:59 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Dec  6 17:58 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Dec  6 17:58 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Dec  6 17:58 time.db
-rw-r--r-- 1 dune01 dune01        519 Dec  6 15:41 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec  6 12:09 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Dec  6 17:59 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Dec  6 11:35 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Dec  6 17:59 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Dec  6 15:54 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Dec  6 15:41 debugprod.log
justIN time: 2026-02-07 17:42:14 UTC       justIN version: 01.06.00