Jobsub ID 19716.188@dunegpschedd01.fnal.gov

Jobsub ID	19716.188@dunegpschedd01.fnal.gov
Workflow ID	166
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-31 06:38:32
Site	IT_CNAF
Entry	CMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat	2025-07-31 16:34:31
From worker node	Hostname	wn-f01-14-02-03.cr.cnaf.infn.it
	cpuinfo	Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-31 06:52:20
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-31 16:16:55
Output files
Finished	2025-07-31 16:34:31
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

n failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:41 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  31-Jul-2025 18:11:42 CEST run: 5038 subRun: 0 event: 49230
Comparing two wires in the same plane: return failure
%MSG
31-Jul-2025 18:16:49 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco_20250731T065228Z_reco2.root"
31-Jul-2025 18:16:49 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/df/71/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2814.56       3323.26       3700.41       3459.62       318.206        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00625606     0.0387063     0.145461      0.0138966     0.0511632       10     
reco:pandora:StandardPandora                            2497.26       2945.98       3218.85       3051.23       266.11         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000646362    0.0182575     0.153432     0.00147889     0.0452106       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000549772   0.00688427     0.0383776    0.00287979     0.0109283       10     
reco:pandoracalo:Calorimetry                          0.000467004    0.0018459     0.0109783    0.000933839   0.00305527       10     
reco:pandorapid:Chi2ParticleID                        0.000114574   0.000665232   0.00419153    0.000207785   0.00120214       10     
reco:linecluster:LineCluster                            1.16932       1.31811       1.83091       1.27665      0.174059        10     
reco:trajcluster:TrajCluster                           0.470287      0.586576      0.727154      0.576821      0.0651597       10     
reco:pmtracktc:PMAlgTrackMaker                          10.2126       11.5092       12.6132       11.5444      0.825546        10     
reco:emtrkmichelid:EmTrackMichelId                      283.053       361.466       501.828       372.127       63.8621        10     
reco:solarflash:SolarOpFlash                            1.5619        1.89993       2.7282        1.84496      0.291869        10     
[art]:TriggerResults:TriggerResultInserter            5.3546e-05    0.000271319   0.00112925    8.17565e-05   0.000385757      10     
end_path:out1:RootOutput                               6.988e-06    1.84994e-05   4.6962e-05    1.3011e-05    1.23876e-05      10     
end_path:out1:RootOutput(write)                         0.25896      0.432629       0.49652      0.447128      0.0608908       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4029.83 MB
  Peak resident set size usage (VmHWM): 2559.71 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco_20250731T065228Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco_20250731T065228Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco_20250731T065228Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T235627Z_gen_004923_supernova_g4stage1_g4stage2_detsim_reco_20250731T065228Z_reco2.root.json
reco2_hist.root