justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241627.0@dunegpschedd01.fnal.gov

Jobsub ID241627.0@dunegpschedd01.fnal.gov
Workflow ID9510
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-01 09:36:05
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-11-01 13:03:50
From worker nodeHostnamehnode78.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-01 09:36:41
Input filesmonte-carlo-009510-000001
JobscriptExit code0
Real time3h (12356s)
CPU time3h (11888s = 96%)
Max RSS bytes4438953984 (4233 MiB)
Outputting started2025-11-01 13:02:38
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09510/1/001/reco_stage1.root
Finished2025-11-01 13:03:50
Saved logsjustin-logs:241627.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 164893 traces, 0 summary
Retagger: tagging trace set: gauss with 125178 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 125178 traces tagged "gauss"
FrameSaver: q=4.44196e+06 n=2219826 tag=gauss
wclsFrameSaver: saving 164893 traces tagged "wiener"
FrameSaver: q=4.71512e+06 n=1977731 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25336 XUs and 25514 XVs -> 1113 XUVs
C:0 T:1 52794 XUs and 47210 XVs -> 11992 XUVs
C:0 T:2 128156 XUs and 181078 XVs -> 6118 XUVs
C:0 T:3 63580 XUs and 77113 XVs -> 522 XUVs
C:0 T:4 15849 XUs and 16197 XVs -> 247 XUVs
C:0 T:5 79449 XUs and 61956 XVs -> 6984 XUVs
C:0 T:6 152187 XUs and 182116 XVs -> 19833 XUVs
C:0 T:7 53217 XUs and 54186 XVs -> 1687 XUVs
C:0 T:8 14842 XUs and 13563 XVs -> 275 XUVs
C:0 T:9 126687 XUs and 147637 XVs -> 13997 XUVs
C:0 T:10 45093 XUs and 27828 XVs -> 6593 XUVs
C:0 T:11 104 XUs and 68 XVs -> 20 XUVs
69381 XUVs total
13895 collection wire objects
62388 potential space points
Neighbour search...
9319436 tests to find 4238984 neighbours
Iterating with no regularization...
Begin: 9.23928e+07
0 7.90135e+07
1 7.7856e+07
2 7.76743e+07
3 7.7616e+07
Now with regularization...
Begin: 6.45944e+07
0 6.45321e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410245073
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410245073
fLifetime: 30000 [us]
01-Nov-2025 14:02:36 CET  Closed output file "reco_stage1.root"
01-Nov-2025 14:02:36 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          219.507       334.272       412.542       338.113       57.9173        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0008528    0.00299675    0.00460936    0.00290498    0.00120897       10     
reco:ophit:OpHitFinder                             0.213233      0.305792       0.40849      0.293638      0.0667287       10     
reco:opflash:OpFlashFinder                         0.143226      0.204978      0.259619      0.209008      0.0325233       10     
reco:opslicer:OpSlicer                             0.476997       0.70898      0.938288       0.71885      0.163042        10     
reco:caldata:DataPrepModule                         3.59103       4.84814       6.97131       4.17282       1.29736        10     
reco:wclsdatasp:WireCellToolkit                     23.6381       29.4532       37.1318       28.0125       4.54012        10     
reco:gaushit:GausHitFinder                         0.712435       1.11999       1.77243      0.974533      0.370017        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000291346   0.000477143   0.000836049   0.000396469   0.000168328      10     
reco:reco3d:SpacePointSolver                        7.16316       16.0237       22.6474       16.9014       4.91486        10     
reco:hitpdune:DisambigFromSpacePoints              0.863082       1.54195       1.96727       1.63187       0.30952        10     
reco:pandora:StandardPandora                        22.0331       33.0299       44.292        32.7611       6.32507        10     
reco:pandoraWriter:StandardPandora                 0.231037      0.391202      0.621423      0.365292      0.111281        10     
reco:pandoraTrack:LArPandoraTrackCreation            1.878        3.77783       5.91325       3.38176       1.20809        10     
reco:pandoraShower:LArPandoraShowerCreation         2.01713       4.32139       7.55773       3.90872       1.65738        10     
reco:pandoracalo:Calorimetry                        1.12685       2.17484       3.54282       2.0222       0.730502        10     
reco:pandoracalonosce:Calorimetry                   1.04384       1.93138       3.10903       1.83283      0.570617        10     
reco:pandorapid:Chi2ParticleID                    0.00291185    0.00508879    0.00774344    0.00444565    0.00162723       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0276427     0.0441732     0.0817163     0.039232      0.0154647       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0027642    0.00453555    0.00774438    0.00399084    0.00150547       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.30037       4.47441       6.17849       4.35154       1.23564        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.21919       4.46291       6.56141       4.46168        1.267         10     
reco:emtrkmichelid:EmTrackMichelId                  124.641       182.049       229.535       184.968       33.8215        10     
reco:crttag:SingleCRTMatchingProducer             0.00645458     0.0118472     0.0181063     0.0122863    0.00371828       10     
reco:crtreco:TwoCRTMatchingProducer               0.00449276      0.00952      0.0133597     0.010352     0.00294381       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.663973      0.995625       1.27648       0.99679      0.199384        10     
reco:pandora2Track:LArPandoraTrackCreation          6.54878       13.0931       17.7073       12.4747       3.26098        10     
reco:pandora2calo:Calorimetry                       3.52715       6.27149       8.8833        6.42686       1.24518        10     
reco:pandora2calonosce:Calorimetry                  3.48745       5.74099       8.49743       5.68702       1.25245        10     
reco:pandora2pid:Chi2ParticleID                   0.00770655     0.0105186     0.0167894     0.0102216    0.00231785       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0388404     0.0513012     0.0767883     0.0490728    0.00964357       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00774932     0.0106816     0.0169731     0.0105555    0.00233235       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.65095       4.09414       5.44045       4.06761      0.819409        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.24475       5.32048       7.12262       5.24638      0.990039        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.04454       5.04692       7.60029       4.86383       1.24007        10     
[art]:TriggerResults:TriggerResultInserter        2.1591e-05    3.70069e-05   0.000115243   2.79705e-05   2.64628e-05      10     
end_path:out1:RootOutput                           4.06e-06     5.4412e-06     1.151e-05     4.935e-06    2.07511e-06      10     
end_path:out1:RootOutput(write)                     1.79093       2.63775       3.74935       2.48559      0.639119        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5248.01 MB
  Peak resident set size usage (VmHWM): 4438.95 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.10 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11487588
-rw-r--r-- 1 dune001 dune 2651742724 Nov  1 13:04 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2514884084 Nov  1 13:02 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2384942657 Nov  1 14:02 reco_stage1.root
-rw-r--r-- 1 dune001 dune 2108176108 Nov  1 10:59 g4_stage2.root
-rw-r--r-- 1 dune001 dune 2016955221 Nov  1 10:45 g4_stage1.root
-rw-r--r-- 1 dune001 dune   85956330 Nov  1 13:58 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243247 Nov  1 14:02 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Nov  1 14:02 mem.db
-rw-r--r-- 1 dune001 dune      62376 Nov  1 14:02 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Nov  1 14:02 time.db
-rw-r--r-- 1 dune001 dune        519 Nov  1 13:04 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov  1 10:59 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov  1 14:02 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Nov  1 10:36 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Nov  1 14:02 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Nov  1 13:11 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Nov  1 13:04 debugprod.log
justIN time: 2025-11-05 04:51:32 UTC       justIN version: 01.05.01