Jobsub ID 11798.0@dunegpschedd02.fnal.gov

Jobsub ID	11798.0@dunegpschedd02.fnal.gov
Workflow ID	80
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-27 20:08:11
Site	UK_RAL-Tier1
Entry	LIGO_UK_RAL_arc_ce05
Last heartbeat	2025-07-28 01:25:54
From worker node	Hostname	dune001-6997176.0-lcg2656.gridpp.rl.ac.uk
	cpuinfo	AMD EPYC 7763 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	216000 (60 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-27 20:10:20
Input files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	5h (18807s)
	CPU time	5h (18519s = 98%)
	Max RSS bytes	2512977920 (2396 MiB)
Outputting started	2025-07-28 01:23:48
Output files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root
Finished	2025-07-28 01:25:54
Saved logs	justin-logs:11798.0-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

t: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:09 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 01:20:10 UTC run: 5179 subRun: 0 event: 5800
Comparing two wires in the same plane: return failure
%MSG
28-Jul-2025 01:23:41 UTC  Closed output file "prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root"
28-Jul-2025 01:23:41 UTC  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/fardet-vd/4f/51/prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1224.83       1865.51       2341.33       1929.71       317.995        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00309262     0.0328291     0.114743      0.0124098     0.040407        10     
reco:pandora:StandardPandora                            1086.33       1630.5        1995.34       1697.91       277.435        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000438635   0.00797202     0.0499342    0.00402641     0.0142065       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.00053738    0.00205283    0.00963662    0.00118559    0.00255267       10     
reco:pandoracalo:Calorimetry                          0.00023924    0.00171445     0.011438     0.000384597    0.0033149       10     
reco:pandorapid:Chi2ParticleID                        7.7826e-05    0.000216957   0.00102701    0.000123988   0.000271681      10     
reco:linecluster:LineCluster                           0.244109       0.44666      0.667734      0.457943      0.134076        10     
reco:trajcluster:TrajCluster                           0.369131      0.422542       0.58439      0.421899      0.0607966       10     
reco:pmtracktc:PMAlgTrackMaker                          5.80946       9.34771       11.3647       10.3513       1.92532        10     
reco:emtrkmichelid:EmTrackMichelId                      130.084       223.684       332.747       218.267       49.2876        10     
reco:solarflash:SolarOpFlash                           0.466328       0.76237      0.923674      0.769146      0.130127        10     
[art]:TriggerResults:TriggerResultInserter            3.0117e-05    6.19425e-05   0.000117841   5.58955e-05   2.99786e-05      10     
end_path:out1:RootOutput                               6.693e-06    1.64601e-05   3.4104e-05    1.41965e-05   8.60638e-06      10     
end_path:out1:RootOutput(write)                        0.195707      0.307216      0.374947      0.308556      0.0464051       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4078.17 MB
  Peak resident set size usage (VmHWM): 2512.98 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root.ext.json
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T012944Z_gen_000580_supernova_g4stage1_g4stage2_detsim_reco_20250727T201026Z_reco2.root.json
reco2_hist.root