Jobsub ID 15596.2@dunegpschedd02.fnal.gov

Jobsub ID	15596.2@dunegpschedd02.fnal.gov
Workflow ID	536
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-08 13:12:57
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2025-08-08 16:00:41
From worker node	Hostname	b9p07p8484.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	343800 (95 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-08 13:15:00
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	2h (9779s)
	CPU time	2h (9400s = 96%)
	Max RSS bytes	2441986048 (2328 MiB)
Outputting started	2025-08-08 15:58:00
Output files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root
Finished	2025-08-08 16:00:41
Saved logs	justin-logs:15596.2-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:35 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:36 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:36 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:55:36 CEST run: 4057 subRun: 0 event: 8890
Comparing two wires in the same plane: return failure
%MSG
08-Aug-2025 17:57:50 CEST  Closed output file "prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root"
08-Aug-2025 17:57:50 CEST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-vd/c8/96/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              869.622       954.686       1082.13       945.219       58.8499        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00274476     0.0116791     0.0499236    0.00657239     0.0137482       10     
reco:pandora:StandardPandora                            758.373       833.819       940.315       825.629       51.3013        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000190623   0.00391005     0.0369973    0.000233953    0.0110291       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000189072   0.000461083    0.0026166    0.000214288   0.000719386      10     
reco:pandoracalo:Calorimetry                          0.000153442   0.000950721   0.00202519    0.00110581    0.000697772      10     
reco:pandorapid:Chi2ParticleID                         3.632e-05    8.27683e-05   0.000434005   3.9626e-05    0.000117408      10     
reco:linecluster:LineCluster                           0.246018       0.29146       0.39862      0.281834      0.0404002       10     
reco:trajcluster:TrajCluster                           0.203214      0.268174      0.354498      0.244646      0.0573631       10     
reco:pmtracktc:PMAlgTrackMaker                          3.85187       5.59364       8.04627       5.50066       1.36455        10     
reco:emtrkmichelid:EmTrackMichelId                      102.734       113.872       132.175       113.362       8.72135        10     
reco:solarflash:SolarOpFlash                           0.458852       0.60951      0.733155      0.631086      0.0854871       10     
[art]:TriggerResults:TriggerResultInserter             1.745e-05    3.35135e-05   9.2091e-05    2.65655e-05   2.07734e-05      10     
end_path:out1:RootOutput                               5.15e-06     9.1873e-06    3.2021e-05     6.87e-06     7.65088e-06      10     
end_path:out1:RootOutput(write)                        0.157709      0.214431      0.260862      0.228035      0.0383424       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3952.5 MB
  Peak resident set size usage (VmHWM): 2441.99 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root.ext.json
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241112T181458Z_gen_000889_supernova_g4stage1_g4stage2_detsim_reco_20250808T131506Z_reco2.root.json
reco2_hist.root