Jobsub ID 20095.5@dunegpschedd01.fnal.gov

Jobsub ID	20095.5@dunegpschedd01.fnal.gov
Workflow ID	163
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-01 05:43:40
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc03
Last heartbeat	2025-08-01 10:28:37
From worker node	Hostname	wn-da-15.gina.surf.nl
	cpuinfo	AMD EPYC 7702P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-01 05:44:20
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	4h (16864s)
	CPU time	4h (16591s = 98%)
	Max RSS bytes	2580107264 (2460 MiB)
Outputting started	2025-08-01 10:25:25
Output files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root
Finished	2025-08-01 10:28:37
Saved logs	justin-logs:20095.5-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

o wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Comparing two wires in the same plane: return failure
%MSG
%MSG-e pma::Track3D:  PMAlgTrackMaker:pmtracktc@BeginModule  01-Aug-2025 12:23:12 CEST run: 5041 subRun: 0 event: 46410
Flip, endpoint closer to vStart.
%MSG
01-Aug-2025 12:25:22 CEST  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root"
01-Aug-2025 12:25:23 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/bf/8e/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1600.48       1659.49       1728.24       1645.63       44.4276        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00308192     0.0289584     0.119901     0.00753427     0.0451291       10     
reco:pandora:StandardPandora                            1449.12       1512.58       1589.95       1493.08       49.5099        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000154123   0.00476821     0.0341545    0.00111565    0.00994116       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000169723   0.00139358    0.00775769    0.00053013    0.00220784       10     
reco:pandoracalo:Calorimetry                          0.000148613   0.00107969    0.00315318    0.000659772   0.000981111      10     
reco:pandorapid:Chi2ParticleID                        3.2222e-05    0.000137362   0.000515941   5.1864e-05    0.000178622      10     
reco:linecluster:LineCluster                           0.945853       1.02651       1.46804      0.959031       0.15892        10     
reco:trajcluster:TrajCluster                           0.274705      0.290276      0.310608      0.292142      0.0116462       10     
reco:pmtracktc:PMAlgTrackMaker                          4.94978       5.61831       6.05605        5.684       0.305941        10     
reco:emtrkmichelid:EmTrackMichelId                      127.385       138.495       150.062       139.087       6.45592        10     
reco:solarflash:SolarOpFlash                            1.17032       1.2239        1.45089       1.19483      0.0790075       10     
[art]:TriggerResults:TriggerResultInserter            1.3857e-05    1.99081e-05    4.724e-05    1.76245e-05   9.25273e-06      10     
end_path:out1:RootOutput                               3.808e-06    6.4544e-06    1.7683e-05    4.2535e-06    4.6559e-06       10     
end_path:out1:RootOutput(write)                        0.206062      0.216705      0.224465      0.219111     0.00699058       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4045.08 MB
  Peak resident set size usage (VmHWM): 2580.11 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250204T154326Z_gen_004641_supernova_g4stage1_g4stage2_detsim_reco_20250801T054426Z_reco2.root.json
reco2_hist.root