justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 243692.0@dunegpschedd01.fnal.gov

Jobsub ID243692.0@dunegpschedd01.fnal.gov
Workflow ID9644
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-05 20:01:47
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-06 02:45:07
From worker nodeHostnamewn1205240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-05 20:02:30
Input filesmonte-carlo-009644-000001
JobscriptExit code0
Real time6h (24043s)
CPU time6h (23130s = 96%)
Max RSS bytes4269326336 (4071 MiB)
Outputting started2025-11-06 02:43:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09644/1/001/reco_stage1.root
Finished2025-11-06 02:45:07
Saved logsjustin-logs:243692.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138153 traces, 0 summary
Retagger: tagging trace set: gauss with 107085 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 107085 traces tagged "gauss"
FrameSaver: q=3.67078e+06 n=1916815 tag=gauss
wclsFrameSaver: saving 138153 traces tagged "wiener"
FrameSaver: q=3.87948e+06 n=1702474 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38391 XUs and 34194 XVs -> 995 XUVs
C:0 T:1 99131 XUs and 77758 XVs -> 13764 XUVs
C:0 T:2 59722 XUs and 78351 XVs -> 2455 XUVs
C:0 T:3 32006 XUs and 33974 XVs -> 254 XUVs
C:0 T:4 9450 XUs and 9221 XVs -> 59 XUVs
C:0 T:5 54785 XUs and 28596 XVs -> 4360 XUVs
C:0 T:6 150542 XUs and 190586 XVs -> 15406 XUVs
C:0 T:7 48786 XUs and 55188 XVs -> 1041 XUVs
C:0 T:8 9690 XUs and 9406 XVs -> 85 XUVs
C:0 T:9 63732 XUs and 83671 XVs -> 4221 XUVs
C:0 T:10 36440 XUs and 27324 XVs -> 3963 XUVs
C:0 T:11 10 XUs and 4 XVs -> 1 XUVs
46604 XUVs total
12163 collection wire objects
41618 potential space points
Neighbour search...
5412314 tests to find 2644652 neighbours
Iterating with no regularization...
Begin: 8.26662e+07
0 7.17544e+07
1 7.11006e+07
2 7.10083e+07
3 7.09797e+07
Now with regularization...
Begin: 6.10974e+07
0 6.10398e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410334183
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410334183
fLifetime: 30000 [us]
06-Nov-2025 02:43:10 GMT  Closed output file "reco_stage1.root"
06-Nov-2025 02:43:10 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          661.292       877.301       1181.38       856.744       150.93         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00174062    0.00730896     0.0203893    0.00635747    0.00502133       10     
reco:ophit:OpHitFinder                             0.452401      0.497833       0.54982      0.492786      0.0286596       10     
reco:opflash:OpFlashFinder                         0.262233      0.346849      0.440474      0.336843      0.0497625       10     
reco:opslicer:OpSlicer                              1.14137       1.31818       1.43292       1.32097      0.0945283       10     
reco:caldata:DataPrepModule                         8.97447       9.4764        10.0913       9.42987      0.311612        10     
reco:wclsdatasp:WireCellToolkit                     62.9951       70.7915       80.6588       71.4224       4.69966        10     
reco:gaushit:GausHitFinder                          1.78248       2.43306       3.10717       2.40022      0.360002        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00107417    0.00125157    0.00165386    0.00124943    0.00015335       10     
reco:reco3d:SpacePointSolver                        16.0139       31.9819       50.2218       30.7248       8.18641        10     
reco:hitpdune:DisambigFromSpacePoints               2.39928       3.52856       4.93887       3.54759      0.741439        10     
reco:pandora:StandardPandora                        44.8375       82.8285       138.302       76.7403       24.454         10     
reco:pandoraWriter:StandardPandora                 0.730178      0.959084       1.15848      0.924805      0.123022        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.67264       13.6341       18.8257       13.2022       2.92063        10     
reco:pandoraShower:LArPandoraShowerCreation         9.2655        16.1549       26.8756       17.0135       4.93858        10     
reco:pandoracalo:Calorimetry                        5.03796       8.08559       10.4119       8.34377       1.83776        10     
reco:pandoracalonosce:Calorimetry                   5.11453       7.88917       10.844        8.11616       1.97907        10     
reco:pandorapid:Chi2ParticleID                     0.0119988     0.0143148     0.0166898     0.0142553    0.00147697       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.105493      0.130999      0.163911      0.128833      0.0201787       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.011693      0.0132102     0.0162513     0.0130456     0.0014359       10     
reco:pandoraShowercalo:ShowerCalorimetry            11.4298       18.6827       34.2165       18.4084       6.42386        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.49701       17.7132       30.8908       18.2087       5.71512        10     
reco:emtrkmichelid:EmTrackMichelId                  362.495       439.043       534.545       429.398       56.6232        10     
reco:crttag:SingleCRTMatchingProducer              0.0245572     0.0479627     0.0892521     0.0471099     0.0167463       10     
reco:crtreco:TwoCRTMatchingProducer                0.0170312     0.0261156     0.0335732     0.0267346    0.00535035       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.31886       1.76168       2.37091       1.70286      0.282158        10     
reco:pandora2Track:LArPandoraTrackCreation          27.2095       43.0577       68.8426       43.7134       11.8481        10     
reco:pandora2calo:Calorimetry                       13.9885       25.092        41.3676       25.7365       7.98109        10     
reco:pandora2calonosce:Calorimetry                  13.4212       23.0418       33.4329       23.9656       6.26094        10     
reco:pandora2pid:Chi2ParticleID                    0.0247489     0.0318235     0.0428002     0.0313431    0.00549704       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.125827      0.163231      0.222225      0.161038      0.0243133       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0255043     0.0310769     0.0431531     0.0292662    0.00522242       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.48948       14.7988       23.1192       15.2288       4.33178        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.02107       19.7338       29.4956       21.3047       6.32575        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.82639       17.994        32.0602       18.3694       6.84166        10     
[art]:TriggerResults:TriggerResultInserter        4.7079e-05    7.00012e-05   0.000204646   5.6166e-05    4.52398e-05      10     
end_path:out1:RootOutput                           7.094e-06    8.6475e-06    1.4929e-05    8.1455e-06    2.20722e-06      10     
end_path:out1:RootOutput(write)                     4.56982       5.69711       7.13445       5.66805      0.688295        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5126.58 MB
  Peak resident set size usage (VmHWM): 4269.33 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11026796
-rw-r--r-- 1 dune01 dune01 2559250362 Nov  6 00:13 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2429935660 Nov  6 00:08 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2273927713 Nov  6 02:43 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2020538749 Nov  5 20:38 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1925436572 Nov  5 20:12 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   81717150 Nov  6 02:32 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242629 Nov  6 02:43 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  6 02:43 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  6 02:43 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  6 02:43 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  6 00:13 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  5 20:38 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  6 02:43 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  5 20:02 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  6 02:43 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  6 00:25 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  6 00:13 debugprod.log
justIN time: 2026-02-11 06:12:54 UTC       justIN version: 01.06.00