Jobsub ID 20086.7@dunegpschedd01.fnal.gov

Jobsub ID	20086.7@dunegpschedd01.fnal.gov
Workflow ID	163
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-01 04:33:37
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc02
Last heartbeat	2025-08-01 09:06:27
From worker node	Hostname	wn-da-25.gina.surf.nl
	cpuinfo	AMD EPYC 7702P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-01 04:35:45
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	4h (16067s)
	CPU time	4h (15845s = 98%)
	Max RSS bytes	2617110528 (2495 MiB)
Outputting started	2025-08-01 09:03:33
Output files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root
Finished	2025-08-01 09:06:27
Saved logs	justin-logs:20086.7-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

res in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:14 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 11:01:15 CEST run: 4966 subRun: 0 event: 8260
Comparing two wires in the same plane: return failure
%MSG
01-Aug-2025 11:03:27 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root"
01-Aug-2025 11:03:27 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d4/df/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1523.53       1579.92       1653.15       1574.74       42.5082        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00268296     0.0257393     0.111106     0.00667436     0.039135        10     
reco:pandora:StandardPandora                            1384.12       1442.48       1516.25       1441.02       41.8246        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000177618   0.00538826      0.03909     0.00198578     0.0112807       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000171656   0.00147509    0.00517722    0.000795483   0.00182076       10     
reco:pandoracalo:Calorimetry                          0.000144755   0.000721649   0.00181844    0.000281882   0.000648299      10     
reco:pandorapid:Chi2ParticleID                        3.3402e-05    0.000134598   0.000490344   6.10605e-05   0.000166434      10     
reco:linecluster:LineCluster                           0.872111      0.939232       1.35689      0.885561      0.140485        10     
reco:trajcluster:TrajCluster                           0.251572      0.274929       0.29492       0.27337      0.0112901       10     
reco:pmtracktc:PMAlgTrackMaker                          4.46843       5.1974        5.89069       5.18354      0.416621        10     
reco:emtrkmichelid:EmTrackMichelId                      124.952       129.637       135.115       129.577       2.42507        10     
reco:solarflash:SolarOpFlash                            1.09534       1.15357       1.35529       1.12474      0.0709593       10     
[art]:TriggerResults:TriggerResultInserter            1.4908e-05    2.3727e-05    4.6629e-05    1.88615e-05   9.87126e-06      10     
end_path:out1:RootOutput                               3.747e-06    5.6809e-06    1.8004e-05    4.3285e-06    4.12253e-06      10     
end_path:out1:RootOutput(write)                        0.195709      0.204257      0.214731      0.204544     0.00624699       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4078.28 MB
  Peak resident set size usage (VmHWM): 2617.11 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T130320Z_gen_000826_supernova_g4stage1_g4stage2_detsim_reco_20250801T043553Z_reco2.root.json
reco2_hist.root