Jobsub ID 19157.97@dunegpschedd01.fnal.gov

Jobsub ID	19157.97@dunegpschedd01.fnal.gov
Workflow ID	102
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-30 01:53:01
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat	2025-07-30 07:55:31
From worker node	Hostname	dunegli-6392088-0-fnpc22002.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-30 04:55:01
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	2h (10763s)
	CPU time	2h (10666s = 99%)
	Max RSS bytes	2592088064 (2472 MiB)
Outputting started	2025-07-30 07:54:25
Output files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root
Finished	2025-07-30 07:55:31
Saved logs	justin-logs:19157.97-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  30-Jul-2025 07:51:10 UTC run: 5021 subRun: 0 event: 38250
Comparing two wires in the same plane: return failure
%MSG
30-Jul-2025 07:54:21 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root"
30-Jul-2025 07:54:21 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/d4/62/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              979.107       1067.86       1211.85       1050.58       61.7708        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00357359     0.010507      0.0173666     0.010582     0.00382648       10     
reco:pandora:StandardPandora                            853.777       910.483       1040.24       898.667       55.0232        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000241819    0.0114999     0.058546      0.0064713     0.0162524       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000223365   0.00188017    0.00981054    0.00115911    0.00267538       10     
reco:pandoracalo:Calorimetry                          0.000287406   0.000985305   0.00339085    0.000504994   0.000958323      10     
reco:pandorapid:Chi2ParticleID                         3.704e-05    0.000197567   0.00120385    9.25115e-05   0.000336526      10     
reco:linecluster:LineCluster                           0.118746      0.133693      0.154689      0.126044      0.0131519       10     
reco:trajcluster:TrajCluster                            0.1952       0.214802       0.24473      0.208938      0.0154728       10     
reco:pmtracktc:PMAlgTrackMaker                          3.16641       3.90543       4.47911       3.91808      0.358484        10     
reco:emtrkmichelid:EmTrackMichelId                      118.091       152.622       189.42        149.033       19.703         10     
reco:solarflash:SolarOpFlash                           0.300225       0.31355      0.349724      0.310848      0.014001        10     
[art]:TriggerResults:TriggerResultInserter             2.134e-05    3.41907e-05   6.1997e-05    3.32575e-05   1.02823e-05      10     
end_path:out1:RootOutput                               7.585e-06    1.12094e-05   2.7061e-05     9.749e-06    5.40428e-06      10     
end_path:out1:RootOutput(write)                        0.151149       0.15677      0.169006      0.154354     0.00564642       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4056.62 MB
  Peak resident set size usage (VmHWM): 2592.09 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250202T100105Z_gen_003825_supernova_g4stage1_g4stage2_detsim_reco_20250730T045507Z_reco2.root.json
reco2_hist.root