Jobsub ID 13265.173@dunegpschedd02.fnal.gov

Jobsub ID	13265.173@dunegpschedd02.fnal.gov
Workflow ID	163
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-31 09:40:41
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce513
Last heartbeat	2025-07-31 14:28:07
From worker node	Hostname	b9p07p9496.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-31 11:17:04
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-31 14:14:21
Output files
Finished	2025-07-31 14:28:07
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

n failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:58 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:59 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 16:11:59 CEST run: 4969 subRun: 0 event: 28910
Comparing two wires in the same plane: return failure
%MSG
31-Jul-2025 16:14:16 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco_20250731T111737Z_reco2.root"
31-Jul-2025 16:14:16 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/4e/a9/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              888.773       1033.28       1188.76       1037.79       75.8112        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00236743     0.0295273     0.125968     0.00727815     0.0462611       10     
reco:pandora:StandardPandora                            782.355       907.716       1044.99       910.008       66.7068        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000218601    0.0041473     0.0309874     0.0015737    0.00897428       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000208391    0.0013131    0.00679848    0.000803009   0.00184725       10     
reco:pandoracalo:Calorimetry                          0.000236831   0.000708528   0.00190703    0.000335477   0.000648985      10     
reco:pandorapid:Chi2ParticleID                         4.663e-05    0.00014578    0.000691824   9.09305e-05   0.000183251      10     
reco:linecluster:LineCluster                           0.963684       1.06038       1.4165        1.00109      0.137324        10     
reco:trajcluster:TrajCluster                           0.302676      0.338979      0.380626      0.330645      0.0273995       10     
reco:pmtracktc:PMAlgTrackMaker                          4.45247       6.21427       9.49667       5.89325       1.59187        10     
reco:emtrkmichelid:EmTrackMichelId                      98.6246       116.422       133.733       115.702       10.7609        10     
reco:solarflash:SolarOpFlash                            1.08582       1.26267       1.47933       1.28792      0.108962        10     
[art]:TriggerResults:TriggerResultInserter             2.365e-05    3.2506e-05     4.99e-05      3.056e-05    7.04568e-06      10     
end_path:out1:RootOutput                               5.33e-06     1.19612e-05   3.1841e-05     8.24e-06     8.1757e-06       10     
end_path:out1:RootOutput(write)                        0.153499      0.226695      0.275718      0.252826      0.0463898       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4044.08 MB
  Peak resident set size usage (VmHWM): 2541.21 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco_20250731T111737Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco_20250731T111737Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco_20250731T111737Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T183424Z_gen_002891_supernova_g4stage1_g4stage2_detsim_reco_20250731T111737Z_reco2.root.json
reco2_hist.root