Jobsub ID 44175.11@dunegpschedd01.fnal.gov
| Jobsub ID | 44175.11@dunegpschedd01.fnal.gov | |
| Workflow ID | 2391 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 8387559424 (7999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-09-10 07:18:02 | |
| Site | US_FNAL-T1 | |
| Entry | CMSHTPC_T1_US_FNAL_condce_opp1_whole | |
| Last heartbeat | 2025-09-10 07:51:15 | |
| From worker node | Hostname | dunegli-45691-0-cmswn5006.fnal.gov |
| cpuinfo | AMD EPYC 7543 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 10485760000 (10000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | finished | |
| Started | 2025-09-10 07:19:23 | |
| Input files | ehn1-beam-np04:H4_v34b_1GeV_-27.7_004752_220006_24_20250614T012555.root | |
| Jobscript | Exit code | 0 |
| Real time | 31m (1874s) | |
| CPU time | 30m (1825s = 97%) | |
| Max RSS bytes | 4200202240 (4005 MiB) | |
| Outputting started | 2025-09-10 07:50:37 | |
| Output files | usertests:prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2.root https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1.log https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test/fnal/02391/1/001/prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2.log | |
| Finished | 2025-09-10 07:51:15 | |
| Saved logs | justin-logs:44175.11-dunegpschedd01.fnal.gov.logs.tgz | |
| List job events Cached HTCondor job logs | ||
Jobscript log (last 10,000 characters)
try depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25610 nodes/ 5537 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce channel summary named "threshold"
10-Sep-2025 07:46:04 UTC Initiating request to open input file "prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1.root"
10-Sep-2025 07:46:04 UTC Opened input file "prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 84389 traces, 84389 summary
Retagger: tagging trace set: gauss with 72266 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 72266 traces tagged "gauss"
FrameSaver: q=1.77854e+06 n=977318 tag=gauss
wclsFrameSaver: saving 84389 traces tagged "wiener"
FrameSaver: q=1.88613e+06 n=957814 tag=wiener
10-Sep-2025 07:47:29 UTC Opened output file with pattern "prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 82289 traces, 82289 summary
Retagger: tagging trace set: gauss with 73092 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 73092 traces tagged "gauss"
FrameSaver: q=2.05555e+06 n=1093104 tag=gauss
wclsFrameSaver: saving 82289 traces tagged "wiener"
FrameSaver: q=2.14334e+06 n=1070742 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 92091 traces, 92091 summary
Retagger: tagging trace set: gauss with 78799 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 78799 traces tagged "gauss"
FrameSaver: q=2.05069e+06 n=1120401 tag=gauss
wclsFrameSaver: saving 92091 traces tagged "wiener"
FrameSaver: q=2.17313e+06 n=1086191 tag=wiener
10-Sep-2025 07:50:36 UTC Closed output file "prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2.root"
10-Sep-2025 07:50:36 UTC Closed input file "prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1.root"
======================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
======================================================================================================================================
Full event 83.0825 86.3005 89.5662 86.253 2.64716 3
--------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000373228 0.00205406 0.00486787 0.000921071 0.0020022 3
reco:wclsdatahd:WireCellToolkit 26.6063 27.1196 27.6329 27.1197 0.419105 3
reco:gaushit:GausHitFinder 0.413867 0.480401 0.528486 0.49885 0.0485773 3
reco:nhitsfilter:NumberOfHitsFilter 0.000220784 0.000336924 0.00046801 0.000321979 0.000101481 3
reco:reco3d:SpacePointSolver 0.250618 0.296714 0.330249 0.309275 0.0337008 3
reco:hitpdune:DisambigFromSpacePoints 0.290638 0.333273 0.362673 0.346509 0.0308612 3
reco:pandora:StandardPandora 8.81116 9.80923 10.888 9.72857 0.849771 3
reco:pandoraWriter:StandardPandora 0.12449 0.154319 0.201886 0.13658 0.0339952 3
reco:pandoraTrack:LArPandoraTrackCreation 0.659859 0.764972 0.8667 0.768358 0.0844762 3
reco:pandoraShower:LArPandoraShowerCreation 0.787375 0.98421 1.08403 1.08123 0.139188 3
reco:pandoracalo:Calorimetry 0.36103 0.419492 0.483927 0.413521 0.0503497 3
reco:pandoracalonosce:Calorimetry 0.356482 0.412493 0.474571 0.406425 0.0483999 3
reco:pandoraShowercalo:ShowerCalorimetry 0.804042 0.995804 1.09964 1.08373 0.135751 3
reco:pandoraShowercalonosce:ShowerCalorimetry 0.771618 0.950787 1.04134 1.0394 0.126695 3
reco:pandoraGnocchiCalo:GnocchiCalorimetry 0.0336066 0.0343875 0.0356417 0.0339142 0.000895677 3
reco:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0304319 0.0324178 0.0346435 0.0321779 0.00172773 3
reco:emtrkmichelid:EmTrackMichelId 18.9701 20.9952 22.4176 21.598 1.47058 3
reco:opdec:Deconvolution 0.557838 0.588991 0.64468 0.564457 0.0394701 3
reco:ophitspe:OpHitFinderDeco 0.104285 0.105582 0.108121 0.104341 0.00179542 3
reco:opflash:OpFlashFinder 0.0603718 0.0615376 0.0634618 0.0607793 0.00137071 3
reco:opslicer:OpSlicer 0.138904 0.146276 0.153618 0.146305 0.00600698 3
reco:crtreco:TwoCRTMatchingProducer 0.00465213 0.00502472 0.00575691 0.00466513 0.00051776 3
reco:crttag:SingleCRTMatchingProducer 0.00254987 0.00294384 0.00351839 0.00276326 0.0004155 3
reco:pandora2Track:LArPandoraTrackCreation 1.82611 2.33452 2.64195 2.53552 0.362122 3
reco:pandora2calo:ShowerCalorimetry 0.682374 0.880708 1.01231 0.947446 0.142721 3
reco:pandora2calonosce:ShowerCalorimetry 0.674049 0.871019 1.00578 0.93323 0.142393 3
reco:pandora2GnocchiCalo:GnocchiCalorimetry 0.0283601 0.0320525 0.0342058 0.0335917 0.00262293 3
reco:pandora2GnocchiCalonosce:GnocchiCalorimetry 0.0277554 0.0291911 0.031754 0.0280639 0.00181661 3
reco:pandora2Shower:LArPandoraShowerCreation 0.825688 1.03457 1.14027 1.13774 0.147704 3
reco:pandora2Showercalo:ShowerCalorimetry 0.895655 1.11732 1.22821 1.2281 0.156743 3
reco:pandora2Showercalonosce:ShowerCalorimetry 0.882526 1.08923 1.19587 1.1893 0.146187 3
[art]:TriggerResults:TriggerResultInserter 2.5509e-05 5.6151e-05 0.00010429 3.8654e-05 3.44598e-05 3
end_path:out1:RootOutput 6.713e-06 1.88697e-05 3.5739e-05 1.4157e-05 1.23095e-05 3
end_path:michelremoving:michelremoving 0.121789 0.165799 0.199601 0.176007 0.0325766 3
end_path:michelremovinggnocchi:michelremoving 0.1157 0.155976 0.182242 0.169987 0.0289159 3
end_path:calibanagnocchi:CalibAnaTree 10.9629 12.4444 13.7757 12.5944 1.1532 3
end_path:out1:RootOutput(write) 1.19689 1.39848 1.52831 1.47023 0.144502 3
======================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 6588.83 MB
Peak resident set size usage (VmHWM): 4186.3 MB
Details saved in: 'mem.db'
====================================================================================================
{'art_out': 'prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2.root', 'tfile_out': 'prod_beam_p1GeV_protodunehd_1_44175_11_1757488767_gen_largeant_ionscint_pdext_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6_detsim1_detsim2_reco1_reco2_tfile.root', 'start_time': 1757490341.932074, 'end_time': 1757490637.5515752}