Jobsub ID 296289.4@dunegpschedd02.fnal.gov

Jobsub ID	296289.4@dunegpschedd02.fnal.gov
Workflow ID	12470
Stage ID	1
User name	skar@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2026-01-28 11:30:37
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2026-01-28 11:34:21
From worker node	Hostname	dunegli-8450302-0-fnpc18006.fnal.gov
	cpuinfo	AMD EPYC 7551 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-01-28 11:31:43
Input files	fardet-hd:nu_dune10kt_1x2x6_1408_143_20230826T093314Z_gen_g4_detsim_hitreco__20240227T211803Z_reco2.root
Jobscript	Exit code	0
	Real time	1m (95s)
	CPU time	1m (87s = 91%)
	Max RSS bytes	11545583616 (11010 MiB)
Outputting started	2026-01-28 11:33:19
Output files	usertests:nu_dune10kt_1x2x6_1408_143_20230826T093314Z_gen_g4_detsim_hitreco__20240227T211803Z_reco2_ana_2026-01-28T_113149Z.root
Finished	2026-01-28 11:34:21
Saved logs	justin-logs:296289.4-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Input PFN = root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/44/0e/nu_dune10kt_1x2x6_1408_143_20230826T093314Z_gen_g4_detsim_hitreco__20240227T211803Z_reco2.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e26-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 4th record. run: 1408 subRun: 1 event: 14304 at 28-Jan-2026 11:32:52 UTC
Analysing.

Warning: there was no track found for track-like PFParticle with ID 7
Found this many showers 7
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:32:53 UTC run: 1408 subRun: 1 event: 14304
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 5th record. run: 1408 subRun: 1 event: 14305 at 28-Jan-2026 11:32:55 UTC
Analysing.

Warning: there was no track found for track-like PFParticle with ID 5
Found this many showers 5
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:32:56 UTC run: 1408 subRun: 1 event: 14305
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 6th record. run: 1408 subRun: 1 event: 14306 at 28-Jan-2026 11:32:57 UTC
Analysing.

Warning: there was no vertex found for PFParticle with ID 3
Warning: there was no track found for track-like PFParticle with ID 3
Warning: there was no track found for track-like PFParticle with ID 5
Found this many showers 5
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:32:58 UTC run: 1408 subRun: 1 event: 14306
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 7th record. run: 1408 subRun: 1 event: 14307 at 28-Jan-2026 11:33:00 UTC
Analysing.

Warning: there was no track found for track-like PFParticle with ID 7
Found this many showers 7
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:33:01 UTC run: 1408 subRun: 1 event: 14307
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 8th record. run: 1408 subRun: 1 event: 14308 at 28-Jan-2026 11:33:03 UTC
Analysing.

Warning: there was no track found for track-like PFParticle with ID 5
Found this many showers 5
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:33:05 UTC run: 1408 subRun: 1 event: 14308
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 9th record. run: 1408 subRun: 1 event: 14309 at 28-Jan-2026 11:33:09 UTC
Analysing.

Warning: there was no track found for track-like PFParticle with ID 4
Found this many showers 4
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:33:10 UTC run: 1408 subRun: 1 event: 14309
Could not find shower with label emshower
%MSG
Found this many showers 0
Begin processing the 10th record. run: 1408 subRun: 1 event: 14310 at 28-Jan-2026 11:33:13 UTC
Analysing.

Warning: there were 11 reconstructed PFParticle daughters; only the first 10 being stored in tree
Warning: there was no track found for track-like PFParticle with ID 12
Found this many showers 12
%MSG-e NeutrinoShowerEff:  NeutrinoShowerEff:emshowershweff@BeginModule  28-Jan-2026 11:33:14 UTC run: 1408 subRun: 1 event: 14310
Could not find shower with label emshower
%MSG
Found this many showers 0
28-Jan-2026 11:33:18 UTC  Closed output file "nu_dune10kt_1x2x6_1408_143_20230826T093314Z_gen_g4_detsim_hitreco__20240227T211803Z_reco2_ana_2026-01-28T_113149Z.root"
28-Jan-2026 11:33:18 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/44/0e/nu_dune10kt_1x2x6_1408_143_20230826T093314Z_gen_g4_detsim_hitreco__20240227T211803Z_reco2.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        1.27951       4.44884       14.1672       3.39591       3.48772        10     
----------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                          0.00203248    0.00443976     0.0228063    0.00239056    0.00612724       10     
end_path:out1:RootOutput                         4.819e-06    1.15869e-05   5.9924e-05    5.9015e-06    1.61568e-05      10     
end_path:analysistree:AnalysisTree               0.217291       1.55613       6.10386       1.01634       1.60699        10     
end_path:pmtrackeff:NeutrinoTrackingEff         0.00130032     0.0423287     0.333141     0.00973419     0.0970666       10     
end_path:pmtracktceff:NeutrinoTrackingEff       0.00143295     0.0383837     0.302009     0.00922864     0.0879968       10     
end_path:pandoraeff:NeutrinoTrackingEff         0.00151579     0.0147091     0.0681963     0.0103134     0.0181736       10     
end_path:pmtrajfiteff:NeutrinoTrackingEff       0.00185927     0.0187844     0.105492      0.0101747     0.0291374       10     
end_path:pmtrajfittceff:NeutrinoTrackingEff     0.00459031     0.0554908     0.350554      0.0222661     0.0989677       10     
end_path:pandorashweff:NeutrinoShowerEff        0.00396173     0.0455981     0.228238      0.0201163     0.0635746       10     
end_path:emshowershweff:NeutrinoShowerEff       9.3728e-05    0.000304966   0.00100803    0.000202507   0.000264909      10     
end_path:trajclustershweff:NeutrinoShowerEff     0.002236      0.0029171    0.00377958     0.0028914    0.000404771      10     
end_path:runcheckereco:CheckRecoEnergy          2.0394e-05    5.41836e-05   0.00029215    2.36415e-05   8.02916e-05      10     
end_path:flashmatchAna:FlashMatchAna             0.020456      0.0853462     0.289626      0.0492251     0.078054        10     
end_path:out1:RootOutput(write)                   1.02184       2.58356       6.37874       2.24094       1.44111        10     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20240.2 MB
  Peak resident set size usage (VmHWM): 11545.6 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0