justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 234562.0@dunegpschedd02.fnal.gov

Jobsub ID234562.0@dunegpschedd02.fnal.gov
Workflow ID9435
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-29 11:24:21
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-10-29 13:52:00
From worker nodeHostnamedunegli-7564038-0-fnpc22001.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-29 11:28:32
Input filesmonte-carlo-009435-000001
JobscriptExit code0
Real time2h (8581s)
CPU time2h (8483s = 98%)
Max RSS bytes4516401152 (4307 MiB)
Outputting started2025-10-29 13:51:34
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09435/1/001/reco_stage1.root
Finished2025-10-29 13:52:00
Saved logsjustin-logs:234562.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

l adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 137515 traces, 0 summary
Retagger: tagging trace set: gauss with 100870 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100870 traces tagged "gauss"
FrameSaver: q=3.4872e+06 n=1746851 tag=gauss
wclsFrameSaver: saving 137515 traces tagged "wiener"
FrameSaver: q=3.70954e+06 n=1527928 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42146 XUs and 40812 XVs -> 748 XUVs
C:0 T:1 115649 XUs and 92348 XVs -> 16659 XUVs
C:0 T:2 67936 XUs and 103456 XVs -> 4331 XUVs
C:0 T:3 37581 XUs and 42943 XVs -> 348 XUVs
C:0 T:4 12829 XUs and 10290 XVs -> 77 XUVs
C:0 T:5 53467 XUs and 31805 XVs -> 5209 XUVs
C:0 T:6 85849 XUs and 92091 XVs -> 5403 XUVs
C:0 T:7 33276 XUs and 35760 XVs -> 370 XUVs
C:0 T:8 8026 XUs and 7413 XVs -> 93 XUVs
C:0 T:9 60445 XUs and 66531 XVs -> 8777 XUVs
C:0 T:10 71763 XUs and 56605 XVs -> 11250 XUVs
C:0 T:11 262 XUs and 135 XVs -> 20 XUVs
53285 XUVs total
11916 collection wire objects
49996 potential space points
Neighbour search...
8610718 tests to find 3735164 neighbours
Iterating with no regularization...
Begin: 5.11916e+07
0 4.22623e+07
1 4.15218e+07
2 4.13852e+07
3 4.13408e+07
4 4.13201e+07
Now with regularization...
Begin: 3.141e+07
0 3.13667e+07
1 3.13353e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410186263
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410186263
fLifetime: 30000 [us]
29-Oct-2025 13:51:33 UTC  Closed output file "reco_stage1.root"
29-Oct-2025 13:51:33 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          146.326       257.866       324.931       266.862       53.767         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0006425    0.00293733    0.00465192    0.00289411     0.0012609       10     
reco:ophit:OpHitFinder                             0.183134      0.231107      0.263784       0.22902      0.0221585       10     
reco:opflash:OpFlashFinder                         0.118215       0.17109      0.201954      0.166744      0.0262023       10     
reco:opslicer:OpSlicer                             0.372814       0.51803      0.618741      0.525004      0.0663061       10     
reco:caldata:DataPrepModule                         3.30454       3.48947       3.72024       3.47229      0.0992362       10     
reco:wclsdatasp:WireCellToolkit                     23.1179       23.6966       24.8299       23.6206      0.436809        10     
reco:gaushit:GausHitFinder                         0.527519       0.88589       1.06619      0.960364      0.171836        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000227372   0.000407997   0.000632311   0.000393513   9.59583e-05      10     
reco:reco3d:SpacePointSolver                        5.80707       13.1305       17.8685       13.8283       3.60935        10     
reco:hitpdune:DisambigFromSpacePoints              0.602976       1.4335        1.94076       1.54845      0.400284        10     
reco:pandora:StandardPandora                        9.80518       30.6409       46.1194       34.2601       12.2111        10     
reco:pandoraWriter:StandardPandora                 0.183617       0.31595      0.396232      0.324149      0.0621652       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.29437       2.93832       4.45298       2.99651      0.861804        10     
reco:pandoraShower:LArPandoraShowerCreation         1.06112       3.52478       5.85753       3.76375       1.27388        10     
reco:pandoracalo:Calorimetry                       0.733481       1.73366       2.42784       1.77194      0.450258        10     
reco:pandoracalonosce:Calorimetry                  0.658563       1.57621       2.13761       1.61763      0.400606        10     
reco:pandorapid:Chi2ParticleID                    0.00255795    0.00391979    0.00495753    0.00397984    0.000625365      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0234191     0.0376645     0.0466326     0.0378276    0.00573068       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00251344    0.00374411    0.00426217    0.00393311    0.00053149       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.07193       3.81188       5.59045       4.05965       1.27385        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.03377       3.73587       5.52429       3.99927       1.27193        10     
reco:emtrkmichelid:EmTrackMichelId                  83.5101       130.668       161.216       133.444       21.8307        10     
reco:crttag:SingleCRTMatchingProducer             0.00750409    0.00914172     0.011502     0.00833642    0.00151581       10     
reco:crtreco:TwoCRTMatchingProducer               0.00360827    0.00576295     0.0074678    0.00585831    0.000933367      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.470843      0.698808      0.888265      0.694008      0.103543        10     
reco:pandora2Track:LArPandoraTrackCreation          3.34336       10.1322       16.4532       9.81153       3.74858        10     
reco:pandora2calo:Calorimetry                       1.70168       5.36644       8.55926       5.38108       1.88475        10     
reco:pandora2calonosce:Calorimetry                  1.59544       4.93574       7.17199       5.28157       1.53275        10     
reco:pandora2pid:Chi2ParticleID                   0.00580076    0.00954965     0.0116093     0.0100736    0.00185453       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0288252     0.0474252     0.0599818     0.0470638    0.00819555       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00535391    0.00943681     0.0113933     0.0101581    0.00190511       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.30552       3.42204       4.91965       3.40347       1.05385        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.49778       4.37501       6.01237       4.50624       1.28572        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.37975       4.01144       5.65977       4.05994       1.19352        10     
[art]:TriggerResults:TriggerResultInserter        2.5098e-05    4.19387e-05   9.1834e-05    3.68395e-05   1.74024e-05      10     
end_path:out1:RootOutput                           9.408e-06    1.15286e-05   1.4628e-05    1.13765e-05   1.4458e-06       10     
end_path:out1:RootOutput(write)                     1.36936       2.20859       2.83595       2.1902       0.406219        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5321.4 MB
  Peak resident set size usage (VmHWM): 4516.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.21 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11349172
-rw-r--r-- 1 dunegli fnalgrid 2616306697 Oct 29 13:07 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2478665214 Oct 29 13:05 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2363470577 Oct 29 13:51 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2072083770 Oct 29 11:41 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2000599087 Oct 29 11:32 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   89815240 Oct 29 13:49 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     243347 Oct 29 13:51 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Oct 29 13:51 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Oct 29 13:51 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Oct 29 13:51 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 29 13:07 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 29 11:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 29 13:51 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Oct 29 11:28 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Oct 29 13:51 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 29 13:12 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 29 13:07 debugprod.log
justIN time: 2025-11-03 18:30:23 UTC       justIN version: 01.05.01