Jobsub ID 18725.159@dunegpschedd01.fnal.gov

Jobsub ID	18725.159@dunegpschedd01.fnal.gov
Workflow ID	100
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-28 22:51:44
Site	UK_Imperial
Entry	DUNE_T2_UK_London_IC_ceprod03
Last heartbeat	2025-07-29 03:21:36
From worker node	Hostname	wm25.grid.hep.ph.ic.ac.uk
	cpuinfo	AMD EPYC 7742 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	aborted
Started	2025-07-28 22:53:47
Input files	fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco.root
Outputting started	2025-07-29 03:17:51
Output files
Finished	2025-07-29 03:21:36
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

nModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:39 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  29-Jul-2025 04:15:40 BST run: 5185 subRun: 0 event: 20
Comparing two wires in the same plane: return failure
%MSG
29-Jul-2025 04:17:47 BST  Closed output file "prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco_20250728T225351Z_reco2.root"
29-Jul-2025 04:17:47 BST  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/fardet-vd/87/cf/prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1453.2        1570.98       2346.34       1486.13       259.053        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000729447    0.012785      0.0750931    0.000823934    0.0247125       10     
reco:pandora:StandardPandora                            1329.14       1445.12       2216.35       1361.72       257.579        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000166354   0.000412851   0.00244822    0.000183973   0.000678641      10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000168899   0.000369139   0.00207138    0.00017781    0.000567487      10     
reco:pandoracalo:Calorimetry                          0.000135426   0.000340784   0.00130784    0.000142013   0.000406746      10     
reco:pandorapid:Chi2ParticleID                         3.155e-05    7.93666e-05   0.00048202    3.3939e-05    0.000134243      10     
reco:linecluster:LineCluster                           0.169307      0.284221      0.453527      0.264413      0.0806446       10     
reco:trajcluster:TrajCluster                            0.22138       0.24695      0.275253      0.242076      0.0190812       10     
reco:pmtracktc:PMAlgTrackMaker                          3.80003       4.58454       5.12943       4.50143      0.406102        10     
reco:emtrkmichelid:EmTrackMichelId                      115.179       120.062       125.222       119.048       3.1732         10     
reco:solarflash:SolarOpFlash                           0.447083      0.474498      0.506238      0.474864      0.021673        10     
[art]:TriggerResults:TriggerResultInserter            1.7052e-05    2.08043e-05   4.0466e-05     1.849e-05    6.65415e-06      10     
end_path:out1:RootOutput                               4.829e-06    7.2948e-06    1.9065e-05     6.552e-06    3.99038e-06      10     
end_path:out1:RootOutput(write)                        0.180656      0.199225      0.270667       0.18708      0.0264956       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3659.48 MB
  Peak resident set size usage (VmHWM): 2179.11 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco_20250728T225351Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco_20250728T225351Z_reco2.root
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco_20250728T225351Z_reco2.root.ext.json
prodmarley_nue_es_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250216T234316Z_gen_000002_supernova_g4stage1_g4stage2_detsim_reco_20250728T225351Z_reco2.root.json
reco2_hist.root