Jobsub ID 19842.64@dunegpschedd01.fnal.gov

Jobsub ID	19842.64@dunegpschedd01.fnal.gov
Workflow ID	166
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-31 14:56:57
Site	UK_Oxford
Entry	DUNE_UK_SGrid_Oxford_arc01
Last heartbeat	2025-07-31 20:38:07
From worker node	Hostname	t2wn004.physics.ox.ac.uk
	cpuinfo	AMD EPYC 7713P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-31 14:59:34
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-31 20:20:23
Output files
Finished	2025-07-31 20:38:07
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

n: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:06 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 21:16:07 BST run: 4985 subRun: 0 event: 43290
Flip, endpoint closer to vStart.
%MSG
31-Jul-2025 21:19:39 BST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco_20250731T145953Z_reco2.root"
31-Jul-2025 21:19:39 BST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e2/5d/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1625.55        1845         1965.35       1845.75       106.492        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00338741     0.0311802     0.119364     0.00958273     0.0436766       10     
reco:pandora:StandardPandora                            1402.95       1630.77       1741.3        1628.3        105.996        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000262476   0.00775946     0.056668     0.00280827     0.0164211       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000256966   0.00196642      0.01215     0.00114133     0.0034269       10     
reco:pandoracalo:Calorimetry                          0.000208042   0.00180075    0.00475638    0.000485937   0.00176969       10     
reco:pandorapid:Chi2ParticleID                         5.187e-05    0.000237597   0.00132188    9.57045e-05   0.000372485      10     
reco:linecluster:LineCluster                           0.997594       1.10568       1.53719       1.07065      0.150459        10     
reco:trajcluster:TrajCluster                           0.388687      0.442052      0.496957      0.435293      0.0303608       10     
reco:pmtracktc:PMAlgTrackMaker                          9.73558       11.3679       13.4456        11.03        1.19982        10     
reco:emtrkmichelid:EmTrackMichelId                      159.759       199.466       222.972       201.25        16.3669        10     
reco:solarflash:SolarOpFlash                            1.32423       1.48514       1.69447       1.46669       0.12064        10     
[art]:TriggerResults:TriggerResultInserter            1.7293e-05    3.04085e-05   6.0366e-05    2.4973e-05    1.49969e-05      10     
end_path:out1:RootOutput                               2.996e-06    5.9857e-06    2.1963e-05     4.574e-06    5.36396e-06      10     
end_path:out1:RootOutput(write)                         0.20276      0.317541      0.372213      0.335705      0.050371        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4046.36 MB
  Peak resident set size usage (VmHWM): 2518.79 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco_20250731T145953Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco_20250731T145953Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco_20250731T145953Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T224224Z_gen_004329_supernova_g4stage1_g4stage2_detsim_reco_20250731T145953Z_reco2.root.json
reco2_hist.root