justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252740.0@dunegpschedd01.fnal.gov

Jobsub ID252740.0@dunegpschedd01.fnal.gov
Workflow ID10255
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 10:29:48
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-16 14:45:49
From worker nodeHostnamewn-s03-06-09-04.cr.cnaf.infn.it
cpuinfoAMD EPYC 7351 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 10:30:54
Input filesmonte-carlo-010255-000001
JobscriptExit code0
Real time3h (14129s)
CPU time3h (13600s = 96%)
Max RSS bytes4190687232 (3996 MiB)
Outputting started2025-11-16 14:26:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10255/1/001/reco_stage1.root
Finished2025-11-16 14:45:49
Saved logsjustin-logs:252740.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

rocessing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 143071 traces, 0 summary
Retagger: tagging trace set: gauss with 106858 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106858 traces tagged "gauss"
FrameSaver: q=3.77714e+06 n=1933723 tag=gauss
wclsFrameSaver: saving 143071 traces tagged "wiener"
FrameSaver: q=4.02304e+06 n=1712862 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28169 XUs and 37557 XVs -> 507 XUVs
C:0 T:1 68619 XUs and 77356 XVs -> 8976 XUVs
C:0 T:2 47947 XUs and 53351 XVs -> 1874 XUVs
C:0 T:4 19186 XUs and 14685 XVs -> 265 XUVs
C:0 T:5 144270 XUs and 100116 XVs -> 21315 XUVs
C:0 T:6 90792 XUs and 72797 XVs -> 4191 XUVs
C:0 T:7 43170 XUs and 33929 XVs -> 484 XUVs
C:0 T:8 15311 XUs and 12556 XVs -> 302 XUVs
C:0 T:9 125826 XUs and 131360 XVs -> 8713 XUVs
C:0 T:10 52544 XUs and 51816 XVs -> 13347 XUVs
C:0 T:11 136 XUs and 90 XVs -> 39 XUVs
60013 XUVs total
14576 collection wire objects
56211 potential space points
Neighbour search...
8326683 tests to find 3883156 neighbours
Iterating with no regularization...
Begin: 6.81464e+07
0 5.61444e+07
1 5.51951e+07
2 5.50307e+07
3 5.49763e+07
Now with regularization...
Begin: 4.42252e+07
0 4.40702e+07
1 4.39645e+07
2 4.3893e+07
3 4.38405e+07
4 4.3798e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410547348
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410547348
fLifetime: 30000 [us]
16-Nov-2025 15:26:21 CET  Closed output file "reco_stage1.root"
16-Nov-2025 15:26:21 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          336.424       504.767       786.29        445.705       156.204        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0014985    0.00424143    0.00710901    0.00392179    0.00177591       10     
reco:ophit:OpHitFinder                             0.264553      0.352296      0.590699      0.322582      0.087575        10     
reco:opflash:OpFlashFinder                         0.130721      0.194807      0.330484      0.188207      0.0540773       10     
reco:opslicer:OpSlicer                             0.596637      0.922067       1.48354       0.84781      0.242864        10     
reco:caldata:DataPrepModule                         5.71851       6.62933       8.4027        6.06379       1.02635        10     
reco:wclsdatasp:WireCellToolkit                     38.7186       45.8516       52.7953       46.9027       4.06295        10     
reco:gaushit:GausHitFinder                          0.9101        1.64584       2.64005       1.56386      0.470341        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000518555   0.000686786   0.00105768    0.000625061   0.000162677      10     
reco:reco3d:SpacePointSolver                        9.05391       21.544        65.1396       17.195        15.5394        10     
reco:hitpdune:DisambigFromSpacePoints               1.01836       2.15785        3.314        2.24287      0.591841        10     
reco:pandora:StandardPandora                        18.0251       50.0418       92.6398       44.8808       22.9021        10     
reco:pandoraWriter:StandardPandora                 0.453815      0.644373       1.26259      0.564894      0.227361        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.44593       6.42553       9.39254       5.98982       1.93907        10     
reco:pandoraShower:LArPandoraShowerCreation         3.31589       7.77072       12.9166       7.58239       3.03691        10     
reco:pandoracalo:Calorimetry                        2.01571       3.61391       5.2147        3.81752       1.03742        10     
reco:pandoracalonosce:Calorimetry                   1.67193       3.08518       4.88914       3.03208       1.0433         10     
reco:pandorapid:Chi2ParticleID                    0.00552459    0.00746714     0.0119518    0.00677599    0.00223435       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0413433     0.0731778      0.11334      0.0676065     0.0223803       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00521074     0.007257      0.0110574    0.00670509    0.00200811       10     
reco:pandoraShowercalo:ShowerCalorimetry            2.70042       6.96499       11.8932       5.65536       3.23862        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.96213       6.73058       11.2134       5.87749       2.80723        10     
reco:emtrkmichelid:EmTrackMichelId                  192.371       270.968       425.946       248.679       82.9082        10     
reco:crttag:SingleCRTMatchingProducer              0.0129746     0.0240762     0.0660702     0.016514      0.015217        10     
reco:crtreco:TwoCRTMatchingProducer               0.00843074     0.0158004     0.0320242     0.0132886    0.00693345       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.611616       1.13316       1.68596       1.10328       0.33255        10     
reco:pandora2Track:LArPandoraTrackCreation          7.64444       20.5798       32.6674       17.3411       8.98637        10     
reco:pandora2calo:Calorimetry                       3.45042       10.2243       15.7577       10.2928       4.47088        10     
reco:pandora2calonosce:Calorimetry                  3.35755       10.8565       19.8994       10.0253       5.52836        10     
reco:pandora2pid:Chi2ParticleID                    0.0149934     0.0195521     0.027467      0.0196089    0.00374109       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0729575     0.103123      0.148047      0.105223      0.0230917       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0148651     0.0203031     0.0316063     0.0194191    0.00495332       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.08123       7.7577        13.309        6.67913       3.1613         10     
reco:pandora2Showercalo:ShowerCalorimetry           2.64586       7.66577       13.0771       6.3955        3.47447        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.44058       6.92421       11.9707       5.94502       3.24711        10     
[art]:TriggerResults:TriggerResultInserter        6.5551e-05    0.000113029   0.000302083   7.7711e-05    6.9439e-05       10     
end_path:out1:RootOutput                           1.257e-05    1.78501e-05    2.443e-05    1.6545e-05    3.87376e-06      10     
end_path:out1:RootOutput(write)                     2.49068       3.66228       5.61382       3.57857      0.924376        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4996.84 MB
  Peak resident set size usage (VmHWM): 4190.69 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9779940
-rw-r--r-- 1 neutrino002 neutrino 2303931673 Nov 16 13:59 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2186590450 Nov 16 13:56 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2001127406 Nov 16 15:26 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1751597834 Nov 16 11:53 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1693141885 Nov 16 11:39 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   77638900 Nov 16 15:19 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242810 Nov 16 15:26 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Nov 16 15:26 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Nov 16 15:26 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Nov 16 15:26 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 13:59 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 11:53 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Nov 16 15:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Nov 16 11:30 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Nov 16 15:26 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 16 14:11 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Nov 16 13:59 debugprod.log
justIN time: 2026-02-09 10:52:21 UTC       justIN version: 01.06.00