Jobsub ID 21366.29@dunegpschedd01.fnal.gov

Jobsub ID	21366.29@dunegpschedd01.fnal.gov
Workflow ID	545
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-08-06 16:02:39
Site	UK_QMUL
Entry	DUNE_UK_London_QMUL_arcce03
Last heartbeat	2025-08-06 20:49:29
From worker node	Hostname	cn299.htc.esc.qmul
	cpuinfo	AMD EPYC 7402 24-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-08-06 16:03:43
Input files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	4h (16932s)
	CPU time	4h (16549s = 97%)
	Max RSS bytes	2532716544 (2415 MiB)
Outputting started	2025-08-06 20:45:56
Output files	fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root
Finished	2025-08-06 20:49:29
Saved logs	justin-logs:21366.29-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ring two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:37 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  06-Aug-2025 21:43:38 BST run: 79 subRun: 0 event: 13540
Comparing two wires in the same plane: return failure
%MSG
06-Aug-2025 21:45:49 BST  Closed output file "prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root"
06-Aug-2025 21:45:50 BST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/da/2c/prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1533.7        1662.11       2125.07       1598.61       181.244        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0040498     0.0256008     0.112074     0.00613484     0.0400621       10     
reco:pandora:StandardPandora                            1399.59       1523.25       1943.43       1463.67       167.84         10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000212156   0.00555521     0.053265     0.000252119    0.0159033       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000228396   0.000527305    0.0030435    0.000248038   0.000838855      10     
reco:pandoracalo:Calorimetry                          0.000156993   0.00179554    0.00469093     0.0014689    0.00159936       10     
reco:pandorapid:Chi2ParticleID                        4.3642e-05    0.000108848   0.000624474   5.4296e-05    0.00017197       10     
reco:linecluster:LineCluster                           0.920792      0.994369       1.35944      0.931592      0.135575        10     
reco:trajcluster:TrajCluster                           0.473527      0.508539      0.545072      0.513241      0.0249294       10     
reco:pmtracktc:PMAlgTrackMaker                          4.94993       5.33542       6.13133       5.32328      0.319039        10     
reco:emtrkmichelid:EmTrackMichelId                      120.333       130.184       171.326       125.616       14.1183        10     
reco:solarflash:SolarOpFlash                            1.21484       1.4731        1.70138       1.47751      0.109343        10     
[art]:TriggerResults:TriggerResultInserter            1.9687e-05    3.49783e-05   8.2364e-05    3.29715e-05   1.6546e-05       10     
end_path:out1:RootOutput                               8.166e-06    1.1724e-05    3.6088e-05     9.097e-06    8.1417e-06       10     
end_path:out1:RootOutput(write)                        0.217964       0.33009      0.410229      0.336561      0.0441868       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4004.05 MB
  Peak resident set size usage (VmHWM): 2532.72 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root
Extracted metadata
checking file====== fardet-vd:prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root.ext.json
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20250726T164133Z_gen_001354_supernova_g4stage1_g4stage2_detsim_reco_20250806T160356Z_reco2.root.json
reco2_hist.root