Jobsub ID 13319.89@dunegpschedd02.fnal.gov

Jobsub ID	13319.89@dunegpschedd02.fnal.gov
Workflow ID	166
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-31 14:12:56
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce514
Last heartbeat	2025-07-31 18:01:49
From worker node	Hostname	b9p04p9108.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-31 14:13:45
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-31 17:39:32
Output files
Finished	2025-07-31 18:01:49
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

n failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:58 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 19:36:59 CEST run: 5017 subRun: 0 event: 39830
Comparing two wires in the same plane: return failure
%MSG
31-Jul-2025 19:39:25 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco_20250731T141350Z_reco2.root"
31-Jul-2025 19:39:25 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/bb/83/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1098.06       1202.72       1309.47       1211.3        66.7938        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00320127     0.0290895      0.12297     0.00847138     0.0442846       10     
reco:pandora:StandardPandora                            953.417       1055.73       1157.11       1061.27       59.6618        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000195552   0.00494032     0.0385444    0.000654831    0.0112595       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000204042   0.00164348    0.00967861    0.000986425    0.0027142       10     
reco:pandoracalo:Calorimetry                          0.000302013   0.00119337    0.00271226    0.00103915    0.000865422      10     
reco:pandorapid:Chi2ParticleID                        4.1611e-05    0.000177173   0.00108288    8.6036e-05    0.000302654      10     
reco:linecluster:LineCluster                           0.989427       1.07843       1.45053       1.02767      0.135443        10     
reco:trajcluster:TrajCluster                           0.259547       0.32162      0.361977      0.327239      0.0299215       10     
reco:pmtracktc:PMAlgTrackMaker                          4.34572       6.87554       8.87727       7.38513       1.54706        10     
reco:emtrkmichelid:EmTrackMichelId                      113.089       137.142       158.307       135.348       10.9528        10     
reco:solarflash:SolarOpFlash                            1.08787       1.2943        1.51792       1.31834      0.114933        10     
[art]:TriggerResults:TriggerResultInserter             2.47e-05     3.60402e-05    6.652e-05    3.3625e-05    1.1645e-05       10     
end_path:out1:RootOutput                               5.79e-06      9.683e-06     2.447e-05     8.385e-06    5.12753e-06      10     
end_path:out1:RootOutput(write)                        0.166547      0.234182      0.271502      0.258729      0.0431713       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4037.37 MB
  Peak resident set size usage (VmHWM): 2529.53 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco_20250731T141350Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco_20250731T141350Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco_20250731T141350Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T030040Z_gen_003983_supernova_g4stage1_g4stage2_detsim_reco_20250731T141350Z_reco2.root.json
reco2_hist.root