Jobsub ID 13528.23@dunegpschedd02.fnal.gov

Jobsub ID	13528.23@dunegpschedd02.fnal.gov
Workflow ID	166
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-31 23:51:23
Site	BR_CBPF
Entry	DUNE_BR_CBPF_ce01
Last heartbeat	2025-08-01 07:15:40
From worker node	Hostname	wn57
	cpuinfo	Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-31 23:52:43
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	7h (26342s)
	CPU time	6h (24813s = 94%)
	Max RSS bytes	2570657792 (2451 MiB)
Outputting started	2025-08-01 07:11:47
Output files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root
Finished	2025-08-01 07:15:40
Saved logs	justin-logs:13528.23-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:09 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 04:07:10 -03 run: 5030 subRun: 0 event: 33360
Comparing two wires in the same plane: return failure
%MSG
01-Aug-2025 04:11:39 -03  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root"
01-Aug-2025 04:11:39 -03  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/db/68/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2330.64       2510.48       2570.78       2552.71       77.9786        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00467825     0.0375503     0.162462      0.007517      0.0606373       10     
reco:pandora:StandardPandora                            2048.31       2234.13       2294.86       2276.94       81.062         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000260798   0.00709591     0.0465448    0.00397542     0.013256        10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000292173   0.00243695     0.0135403    0.00145762    0.00374816       10     
reco:pandoracalo:Calorimetry                          0.000428203   0.00165449    0.00393938    0.000656954   0.00141223       10     
reco:pandorapid:Chi2ParticleID                        6.2862e-05    0.00026165    0.00145133    0.000133712   0.000398362      10     
reco:linecluster:LineCluster                            1.35354       1.49777       2.14159       1.39749      0.226872        10     
reco:trajcluster:TrajCluster                           0.486266      0.512444      0.542232      0.510108      0.0204641       10     
reco:pmtracktc:PMAlgTrackMaker                          9.28929       10.2188       11.5408       10.1717      0.701168        10     
reco:emtrkmichelid:EmTrackMichelId                      247.648       261.58        267.996       263.023       5.76657        10     
reco:solarflash:SolarOpFlash                            1.97608       2.06646       2.42999       2.03316       0.12472        10     
[art]:TriggerResults:TriggerResultInserter            4.1743e-05    5.10716e-05   8.0567e-05    4.85345e-05   1.07959e-05      10     
end_path:out1:RootOutput                               5.888e-06    1.0838e-05    3.0548e-05     8.582e-06    6.81682e-06      10     
end_path:out1:RootOutput(write)                        0.351144      0.428947      0.516219      0.428832      0.0408219       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4042.35 MB
  Peak resident set size usage (VmHWM): 2570.66 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250203T050637Z_gen_003336_supernova_g4stage1_g4stage2_detsim_reco_20250731T235248Z_reco2.root.json
reco2_hist.root