Jobsub ID 11848.1@dunegpschedd02.fnal.gov

Jobsub ID	11848.1@dunegpschedd02.fnal.gov
Workflow ID	77
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-27 23:14:21
Site	ES_PIC
Entry	DUNE_T1_ES_PIC_ce16-multicore
Last heartbeat	2025-07-28 06:45:07
From worker node	Hostname	td830.pic.es
	cpuinfo	AMD EPYC 7452 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	216000 (60 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-27 23:16:16
Input files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	7h (26771s)
	CPU time	7h (25683s = 95%)
	Max RSS bytes	2463404032 (2349 MiB)
Outputting started	2025-07-28 06:42:29
Output files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root
Finished	2025-07-28 06:45:07
Saved logs	justin-logs:11848.1-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:24 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  28-Jul-2025 08:38:25 CEST run: 5166 subRun: 0 event: 44970
Comparing two wires in the same plane: return failure
%MSG
28-Jul-2025 08:42:16 CEST  Closed output file "prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root"
28-Jul-2025 08:42:17 CEST  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/fardet-vd/0d/41/prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2130.43       2628.87       3129.34       2711.28       354.845        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00318368     0.0332399     0.132041      0.0147209     0.0414398       10     
reco:pandora:StandardPandora                            1945.5        2403.82       2875.25       2478.16       330.044        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000284764   0.00839828     0.0502223    0.00432707     0.0141454       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000309535   0.00924757     0.0551749    0.00180727     0.0164968       10     
reco:pandoracalo:Calorimetry                          0.000360414   0.00284089     0.0171318    0.000669834   0.00490459       10     
reco:pandorapid:Chi2ParticleID                        7.9731e-05    0.000272452   0.00106693    0.000159002   0.000292943      10     
reco:linecluster:LineCluster                           0.618348      0.781351       1.09241      0.726443      0.141314        10     
reco:trajcluster:TrajCluster                            0.45434      0.497207      0.580734      0.491208      0.0378152       10     
reco:pmtracktc:PMAlgTrackMaker                          6.82098       10.0954       12.6365       10.9077       2.02336        10     
reco:emtrkmichelid:EmTrackMichelId                      175.062       212.009        261.2        213.255       26.343         10     
reco:solarflash:SolarOpFlash                            1.02841       1.18074       1.3125        1.20527      0.0911787       10     
[art]:TriggerResults:TriggerResultInserter            3.4601e-05    5.37018e-05   9.1251e-05    5.02355e-05   1.65426e-05      10     
end_path:out1:RootOutput                               9.801e-06    1.58463e-05    4.379e-05    1.0685e-05    1.08708e-05      10     
end_path:out1:RootOutput(write)                        0.282655      0.430439      0.826967      0.386257      0.141724        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4037.58 MB
  Peak resident set size usage (VmHWM): 2463.4 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root.ext.json
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250214T074508Z_gen_004497_supernova_g4stage1_g4stage2_detsim_reco_20250727T231623Z_reco2.root.json
reco2_hist.root