Jobsub ID 11287.51@dunegpschedd02.fnal.gov

Jobsub ID	11287.51@dunegpschedd02.fnal.gov
Workflow ID	69
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 19:03:41
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce504
Last heartbeat	2025-07-26 02:02:01
From worker node	Hostname	b9p11p4111.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-25 21:19:03
Input files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-26 01:25:07
Output files
Finished	2025-07-26 02:02:01
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:14 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  26-Jul-2025 03:23:15 CEST run: 5183 subRun: 0 event: 5430
Comparing two wires in the same plane: return failure
%MSG
26-Jul-2025 03:25:02 CEST  Closed output file "prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco_20250725T211908Z_reco2.root"
26-Jul-2025 03:25:02 CEST  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/fardet-vd/87/4d/prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1059.26       1212.07       1365.74       1207.06       103.358        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00261527     0.042978      0.200038     0.00970107     0.0700112       10     
reco:pandora:StandardPandora                            945.805       1062.67       1196.49       1056.62       88.1802        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000391384    0.0059216     0.040139     0.00190024     0.0115054       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000360624   0.00148589    0.00543424    0.00103557    0.00141296       10     
reco:pandoracalo:Calorimetry                          0.000248362   0.000754131   0.00260259    0.000434385   0.000712621      10     
reco:pandorapid:Chi2ParticleID                        7.1271e-05    0.000165244   0.000486206   8.9971e-05    0.000149696      10     
reco:linecluster:LineCluster                           0.880583       1.58095       2.39458       1.53693      0.524342        10     
reco:trajcluster:TrajCluster                           0.242249      0.312901      0.384604      0.311444      0.0466662       10     
reco:pmtracktc:PMAlgTrackMaker                          5.25632       6.38636       9.54951       5.9964        1.21615        10     
reco:emtrkmichelid:EmTrackMichelId                      104.304       139.478       164.534       140.726       17.2777        10     
reco:solarflash:SolarOpFlash                           0.802811       1.3436        2.09957       1.29834      0.363074        10     
[art]:TriggerResults:TriggerResultInserter             2.185e-05    6.50517e-05   0.000289503   3.9596e-05    7.5758e-05       10     
end_path:out1:RootOutput                               7.94e-06      1.117e-05     1.882e-05     1.031e-05    3.28394e-06      10     
end_path:out1:RootOutput(write)                        0.161261      0.242016      0.319554      0.247275       0.04048        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4035.47 MB
  Peak resident set size usage (VmHWM): 2530.41 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco_20250725T211908Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco_20250725T211908Z_reco2.root
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco_20250725T211908Z_reco2.root.ext.json
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T024305Z_gen_000543_supernova_g4stage1_g4stage2_detsim_reco_20250725T211908Z_reco2.root.json
reco2_hist.root