Jobsub ID 22068.18@dunegpschedd01.fnal.gov

Jobsub ID	22068.18@dunegpschedd01.fnal.gov
Workflow ID	551
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-08 13:12:58
Site	US_Colorado
Entry	CMSHTPC_T3_US_Colorado_heposg01-colorado
Last heartbeat	2025-08-08 23:14:44
From worker node	Hostname	lnxfarm230.colorado.edu
	cpuinfo	Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-08 13:23:19
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (35292s)
	CPU time	9h (34634s = 98%)
	Max RSS bytes	2479423488 (2364 MiB)
Outputting started	2025-08-08 23:11:33
Output files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root
Finished	2025-08-08 23:14:44
Saved logs	justin-logs:22068.18-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ring two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:11 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:12 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:12 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:12 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  08-Aug-2025 17:05:12 MDT run: 81 subRun: 0 event: 68810
Comparing two wires in the same plane: return failure
%MSG
08-Aug-2025 17:11:19 MDT  Closed output file "prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root"
08-Aug-2025 17:11:19 MDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/07/2f/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3449.32       3489.73       3530.27       3488.06       23.7048        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00320765     0.0221783     0.0817509    0.00771795     0.0244369       10     
reco:pandora:StandardPandora                            3116.73       3205.71       3303.87       3206.38       61.8259        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000298626   0.00821996     0.0782426    0.000452619    0.0233411       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000296248   0.000814817   0.00465541    0.000388786   0.00128166       10     
reco:pandoracalo:Calorimetry                          0.000318443   0.00606348     0.0108892    0.00867268    0.00442492       10     
reco:pandorapid:Chi2ParticleID                        6.6287e-05    0.000158032   0.000833163   8.6517e-05    0.000225367      10     
reco:linecluster:LineCluster                            0.49141      0.590831       0.79015      0.545843      0.093907        10     
reco:trajcluster:TrajCluster                           0.412913      0.460857      0.506972      0.462385      0.0248412       10     
reco:pmtracktc:PMAlgTrackMaker                          7.0162        8.81981       11.1866       9.00026       1.20797        10     
reco:emtrkmichelid:EmTrackMichelId                      211.199       273.077       361.846       262.987       52.8851        10     
reco:solarflash:SolarOpFlash                           0.766476      0.814097      0.867089      0.815723      0.0298297       10     
[art]:TriggerResults:TriggerResultInserter            1.4141e-05    2.25658e-05   3.6739e-05    2.23105e-05   6.93568e-06      10     
end_path:out1:RootOutput                               2.641e-06    5.3356e-06    1.4902e-05      4.6e-06     3.39148e-06      10     
end_path:out1:RootOutput(write)                        0.211442      0.219892      0.237765       0.21805     0.00727773       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3964.98 MB
  Peak resident set size usage (VmHWM): 2479.42 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root.ext.json
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250727T114729Z_gen_006881_supernova_g4stage1_g4stage2_detsim_reco_20250808T132328Z_reco2.root.json
reco2_hist.root