Jobsub ID 19477.8@dunegpschedd01.fnal.gov

Jobsub ID	19477.8@dunegpschedd01.fnal.gov
Workflow ID	111
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-30 17:59:52
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce03
Last heartbeat	2025-07-30 21:45:41
From worker node	Hostname	ccwcondor0041
	cpuinfo	AMD EPYC 9334 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	106200 (29 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-30 18:00:43
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	3h (13089s)
	CPU time	3h (12491s = 95%)
	Max RSS bytes	2536439808 (2418 MiB)
Outputting started	2025-07-30 21:38:53
Output files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root
Finished	2025-07-30 21:45:41
Saved logs	justin-logs:19477.8-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

n failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 23:36:29 CEST run: 4984 subRun: 0 event: 45260
Comparing two wires in the same plane: return failure
%MSG
30-Jul-2025 23:38:48 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root"
30-Jul-2025 23:38:48 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/24/c7/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1032.72       1279.37       1457.65       1272.76       123.609        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00331854     0.0311784     0.118161      0.0107456     0.0435203       10     
reco:pandora:StandardPandora                            918.491       1135.59       1300.51       1131.12       112.739        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000311463    0.0071021     0.0509807    0.00213933     0.0147571       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000249999   0.00161269    0.00922519    0.00105655    0.00256769       10     
reco:pandoracalo:Calorimetry                          0.000250049    0.001368     0.00447603    0.000381474   0.00162423       10     
reco:pandorapid:Chi2ParticleID                        5.0857e-05    0.000186129   0.00110663    9.3923e-05    0.00030766       10     
reco:linecluster:LineCluster                            1.03236       1.19459       1.9713        1.06657      0.290965        10     
reco:trajcluster:TrajCluster                           0.291169      0.317787      0.359091       0.31702      0.0188635       10     
reco:pmtracktc:PMAlgTrackMaker                          5.15059       7.22221       8.4958        7.50692       1.1041         10     
reco:emtrkmichelid:EmTrackMichelId                      105.399       133.398       145.714       136.843       11.4354        10     
reco:solarflash:SolarOpFlash                            1.23228       1.3629        1.83414       1.29928      0.166428        10     
[art]:TriggerResults:TriggerResultInserter             3.94e-05     5.48714e-05   0.000122696   4.4317e-05    2.41903e-05      10     
end_path:out1:RootOutput                               7.021e-06    1.13463e-05   4.4057e-05     7.621e-06    1.09129e-05      10     
end_path:out1:RootOutput(write)                        0.211984      0.243652      0.318149      0.240129      0.0266358       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4037.15 MB
  Peak resident set size usage (VmHWM): 2536.44 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250129T204157Z_gen_004526_supernova_g4stage1_g4stage2_detsim_reco_20250730T180047Z_reco2.root.json
reco2_hist.root