Jobsub ID 15917.75@dunegpschedd02.fnal.gov

Jobsub ID	15917.75@dunegpschedd02.fnal.gov
Workflow ID	685
Stage ID	1
User name	epennacc@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-09 07:30:53
Site	US_Colorado
Entry	CMSHTPC_T3_US_Colorado_heposg01-colorado
Last heartbeat	2025-08-09 19:38:34
From worker node	Hostname	lnxfarm253.colorado.edu
	cpuinfo	Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-09 09:54:55
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34892s)
	CPU time	9h (34282s = 98%)
	Max RSS bytes	2446970880 (2333 MiB)
Outputting started	2025-08-09 19:36:28
Output files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root
Finished	2025-08-09 19:38:34
Saved logs	justin-logs:15917.75-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ring two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:11 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  09-Aug-2025 13:32:12 MDT run: 87 subRun: 0 event: 47520
Comparing two wires in the same plane: return failure
%MSG
09-Aug-2025 13:36:18 MDT  Closed output file "prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root"
09-Aug-2025 13:36:18 MDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/e0/c2/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              3216.84       3454.73       3531.22       3486.48       86.2012        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00352177     0.0528342     0.391403      0.0103088     0.113551        10     
reco:pandora:StandardPandora                            2964.99       3103.83       3163.85       3122.5        54.9357        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000339552    0.0151027     0.146976     0.000448923    0.0439579       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000292613   0.000913202   0.00562568    0.000418996   0.00157156       10     
reco:pandoracalo:Calorimetry                          0.000432759   0.00740576     0.011548     0.00963318    0.00408003       10     
reco:pandorapid:Chi2ParticleID                        5.2962e-05    0.000285377   0.00192755    8.25035e-05   0.000549071      10     
reco:linecluster:LineCluster                           0.436655      0.584032      0.822993      0.573223      0.0904242       10     
reco:trajcluster:TrajCluster                           0.431225      0.505222      0.619509      0.485133      0.0496639       10     
reco:pmtracktc:PMAlgTrackMaker                          7.81763       9.72348       11.0036       9.84733      0.942948        10     
reco:emtrkmichelid:EmTrackMichelId                      242.127       338.937       357.031       352.217       33.1992        10     
reco:solarflash:SolarOpFlash                            0.77905      0.834859      0.888584      0.834116      0.0332493       10     
[art]:TriggerResults:TriggerResultInserter            1.5085e-05    2.7382e-05    6.0281e-05    2.70975e-05   1.25531e-05      10     
end_path:out1:RootOutput                               2.678e-06    6.0488e-06    2.0073e-05    5.1325e-06    4.81304e-06      10     
end_path:out1:RootOutput(write)                        0.221871      0.231065       0.24473      0.230681     0.00577887       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4001.08 MB
  Peak resident set size usage (VmHWM): 2446.97 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root.ext.json
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250728T142949Z_gen_004752_supernova_g4stage1_g4stage2_detsim_reco_20250809T095458Z_reco2.root.json
reco2_hist.root