Jobsub ID 17948.175@dunegpschedd01.fnal.gov

Jobsub ID	17948.175@dunegpschedd01.fnal.gov
Workflow ID	69
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-25 16:13:32
Site	UK_Imperial
Entry	DUNE_T2_UK_London_IC_ceprod02
Last heartbeat	2025-07-26 10:22:38
From worker node	Hostname	wk01.grid.hep.ph.ic.ac.uk
	cpuinfo	Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2025-07-25 17:00:27
Input files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	5h (20060s)
	CPU time	5h (18782s = 93%)
	Max RSS bytes	2200801280 (2098 MiB)
Outputting started	2025-07-25 22:34:49
Output files
Finished	2025-07-26 10:22:38
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  25-Jul-2025 23:31:09 BST run: 5178 subRun: 0 event: 23520
Comparing two wires in the same plane: return failure
%MSG
25-Jul-2025 23:34:44 BST  Closed output file "prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco_20250725T170032Z_reco2.root"
25-Jul-2025 23:34:44 BST  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/fardet-vd/81/98/prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1597.18       1983.52       3180.87       1870.98       444.69         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000803611    0.0220725     0.154293     0.00107344     0.0471101       10     
reco:pandora:StandardPandora                            1401.8        1749.89       2907.47       1617.62       423.271        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000199593   0.000556102   0.00313082    0.000253677   0.000860645      10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000189949   0.000550331   0.00309524    0.000250811   0.000850879      10     
reco:pandoracalo:Calorimetry                          0.00015331    0.000707285   0.00376046    0.000215172   0.00109439       10     
reco:pandorapid:Chi2ParticleID                        3.8862e-05    0.000121829   0.000735436   5.1873e-05    0.000204849      10     
reco:linecluster:LineCluster                           0.338964      0.597338       1.81476      0.428127       0.42567        10     
reco:trajcluster:TrajCluster                           0.296227      0.397327      0.514927      0.398407      0.0781689       10     
reco:pmtracktc:PMAlgTrackMaker                          4.79442       7.64538       11.5903       7.11361       2.46271        10     
reco:emtrkmichelid:EmTrackMichelId                      188.855       223.896       273.411       209.009       33.6112        10     
reco:solarflash:SolarOpFlash                           0.561305      0.759961       1.16219      0.686478      0.202529        10     
[art]:TriggerResults:TriggerResultInserter            1.7055e-05    2.83351e-05   6.0785e-05     2.425e-05    1.22259e-05      10     
end_path:out1:RootOutput                               6.706e-06    1.15674e-05   2.7702e-05    8.0235e-06    7.65993e-06      10     
end_path:out1:RootOutput(write)                        0.225696       0.30795      0.474587       0.24175      0.0939483       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3675.04 MB
  Peak resident set size usage (VmHWM): 2200.8 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco_20250725T170032Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco_20250725T170032Z_reco2.root
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco_20250725T170032Z_reco2.root.ext.json
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250215T022446Z_gen_002352_supernova_g4stage1_g4stage2_detsim_reco_20250725T170032Z_reco2.root.json
reco2_hist.root