justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 248320.1@dunegpschedd02.fnal.gov

Jobsub ID248320.1@dunegpschedd02.fnal.gov
Workflow ID10363
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-18 23:12:15
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-19 03:46:17
From worker nodeHostnamewn1905251.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-18 23:13:08
Input filesmonte-carlo-010363-000001
JobscriptExit code0
Real time4h (15428s)
CPU time4h (14848s = 96%)
Max RSS bytes4117278720 (3926 MiB)
Outputting started2025-11-19 03:30:17
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10363/1/001/reco_stage1.root
Finished2025-11-19 03:46:17
Saved logsjustin-logs:248320.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

l adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148303 traces, 0 summary
Retagger: tagging trace set: gauss with 115912 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 115912 traces tagged "gauss"
FrameSaver: q=4.0183e+06 n=2099225 tag=gauss
wclsFrameSaver: saving 148303 traces tagged "wiener"
FrameSaver: q=4.2468e+06 n=1879886 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34388 XUs and 30395 XVs -> 968 XUVs
C:0 T:1 68646 XUs and 76840 XVs -> 7162 XUVs
C:0 T:2 119035 XUs and 134841 XVs -> 4637 XUVs
C:0 T:3 57664 XUs and 54047 XVs -> 387 XUVs
C:0 T:4 10131 XUs and 12349 XVs -> 63 XUVs
C:0 T:5 37490 XUs and 34421 XVs -> 2843 XUVs
C:0 T:6 197739 XUs and 212262 XVs -> 10057 XUVs
C:0 T:7 73244 XUs and 75445 XVs -> 954 XUVs
C:0 T:8 6096 XUs and 7370 XVs -> 77 XUVs
C:0 T:9 45703 XUs and 67348 XVs -> 5472 XUVs
C:0 T:10 50076 XUs and 71323 XVs -> 7511 XUVs
C:0 T:11 262 XUs and 393 XVs -> 45 XUVs
40176 XUVs total
13088 collection wire objects
36221 potential space points
Neighbour search...
2460703 tests to find 1257056 neighbours
Iterating with no regularization...
Begin: 7.49711e+07
0 6.17418e+07
1 6.08413e+07
2 6.07066e+07
3 6.06664e+07
Now with regularization...
Begin: 5.01362e+07
0 5.01099e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410598547
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410598547
fLifetime: 30000 [us]
19-Nov-2025 03:30:14 GMT  Closed output file "reco_stage1.root"
19-Nov-2025 03:30:14 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          261.607       365.528       507.688       346.387       77.0399        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00116526    0.00424612    0.00701998    0.00413868    0.00185561       10     
reco:ophit:OpHitFinder                             0.223612      0.296237      0.367096      0.295609      0.0442275       10     
reco:opflash:OpFlashFinder                          0.12705      0.180084      0.292018      0.168605      0.0464966       10     
reco:opslicer:OpSlicer                              0.41006      0.687495      0.953075      0.703087       0.15168        10     
reco:caldata:DataPrepModule                         4.54227       5.55202       6.64291       5.68691      0.587902        10     
reco:wclsdatasp:WireCellToolkit                     33.6126       41.1199       47.1825       40.9887       3.48476        10     
reco:gaushit:GausHitFinder                         0.950761       1.22386       1.57729       1.18904      0.169415        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000360957   0.000896333   0.00327587    0.000630844   0.000815985      10     
reco:reco3d:SpacePointSolver                        6.76855       14.8925       25.2838       14.1155       5.03166        10     
reco:hitpdune:DisambigFromSpacePoints               1.08872       1.63441       2.47872       1.51518      0.403074        10     
reco:pandora:StandardPandora                        15.1139       30.8069       51.055        27.3001       11.0958        10     
reco:pandoraWriter:StandardPandora                 0.366514      0.481388      0.626438      0.452827      0.0714896       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.3859        4.45202       7.86166       4.12426       1.42746        10     
reco:pandoraShower:LArPandoraShowerCreation         2.84135       5.48398       9.52362       5.28091       2.14668        10     
reco:pandoracalo:Calorimetry                        1.42919       2.60461       5.13478       2.10567       1.14292        10     
reco:pandoracalonosce:Calorimetry                   1.07219       2.37685       4.28214       2.20876       0.85615        10     
reco:pandorapid:Chi2ParticleID                    0.00296168    0.00525329     0.0100597     0.0045236    0.00207013       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0315549     0.0512558     0.106566      0.0420393     0.023164        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00295703    0.00500596    0.00959028    0.00431421    0.00187421       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.63533       6.24699       13.0563       6.40166       3.12338        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.25319       5.1929        8.62195       5.23692       2.03184        10     
reco:emtrkmichelid:EmTrackMichelId                  141.302       191.119       247.719       187.778       35.0699        10     
reco:crttag:SingleCRTMatchingProducer             0.00865527     0.0165764     0.0244461     0.0163486    0.00504509       10     
reco:crtreco:TwoCRTMatchingProducer               0.00648225     0.0111428     0.0167712     0.0108849    0.00331965       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.665589       1.01381       1.93479       0.93657      0.354132        10     
reco:pandora2Track:LArPandoraTrackCreation          9.50979       15.0585       27.8888       12.9179       5.7994         10     
reco:pandora2calo:Calorimetry                       4.77453       7.55018       13.0116       6.44655       2.76295        10     
reco:pandora2calonosce:Calorimetry                  4.44169       7.47723       12.3585       6.35894       2.70768        10     
reco:pandora2pid:Chi2ParticleID                   0.00950456     0.0138554     0.0234214     0.0121118     0.0039147       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.049897      0.065443      0.0831935     0.0634107     0.0127733       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00880159     0.0122937     0.0168182     0.011834     0.00241169       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.44692       5.12926       8.06687       4.75134       1.47917        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.98079       6.33076       9.48096       6.15541       1.83277        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.37627       5.41541       7.70172       5.53186       1.35751        10     
[art]:TriggerResults:TriggerResultInserter        4.5345e-05    9.27811e-05   0.000172774   8.05515e-05   3.6463e-05       10     
end_path:out1:RootOutput                          1.0199e-05    1.47336e-05   2.6249e-05    1.33705e-05   4.76036e-06      10     
end_path:out1:RootOutput(write)                     2.14679       2.85427       3.65886       2.81142      0.461034        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4935.08 MB
  Peak resident set size usage (VmHWM): 4117.28 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.32 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9904040
-rw-r--r-- 1 dune01 dune01 2332802396 Nov 19 02:27 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2213182674 Nov 19 02:24 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2009802485 Nov 19 03:30 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1791104841 Nov 18 23:35 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1721303196 Nov 18 23:20 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   72910660 Nov 19 03:24 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243020 Nov 19 03:30 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov 19 03:30 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov 19 03:30 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov 19 03:30 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov 19 02:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 18 23:35 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov 19 03:30 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov 18 23:13 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov 19 03:30 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov 19 02:33 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov 19 02:27 debugprod.log
justIN time: 2026-02-09 05:00:05 UTC       justIN version: 01.06.00