Jobsub ID 273497.35@dunegpschedd02.fnal.gov

Jobsub ID	273497.35@dunegpschedd02.fnal.gov
Workflow ID	11974
Stage ID	1
User name	imawby@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	1048576000 (1000 MiB)
	Wall seconds limit	7200 (2 hours)
Submitted time	2026-01-15 20:18:16
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc03
Last heartbeat	2026-01-15 21:23:25
From worker node	Hostname	wn-da-03.gina.surf.nl
	cpuinfo	AMD EPYC 7702P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	1048576000 (1000 MiB)
	Wall seconds limit	129600 (36 hours)
	GPU
	Inner Apptainer?	True
Job state	outputting_failed
Started	2026-01-15 20:19:23
Input files	fardet-hd:nu_dune10kt_1x2x6_1102_1_20230826T063255Z_gen_g4_detsim_hitreco__20240229T180222Z_reco2.root
Jobscript	Exit code	0
	Real time	4m (277s)
	CPU time	4m (269s = 97%)
	Max RSS bytes	1193746432 (1138 MiB)
Outputting started	2026-01-15 20:24:01
Output files
Finished	2026-01-15 21:23:25
List job events Cached HTCondor job logs
Jobscript log (last 10,000 characters)

FN = root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/03/db/nu_dune10kt_1x2x6_1102_1_20230826T063255Z_gen_g4_detsim_hitreco__20240229T180222Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 29th record. run: 1102 subRun: 1 event: 129 at 15-Jan-2026 21:21:08 CET
Begin processing the 30th record. run: 1102 subRun: 1 event: 130 at 15-Jan-2026 21:21:10 CET
Begin processing the 31st record. run: 1102 subRun: 1 event: 131 at 15-Jan-2026 21:21:12 CET
Begin processing the 32nd record. run: 1102 subRun: 1 event: 132 at 15-Jan-2026 21:21:14 CET
Begin processing the 33rd record. run: 1102 subRun: 1 event: 133 at 15-Jan-2026 21:21:16 CET
Begin processing the 34th record. run: 1102 subRun: 1 event: 134 at 15-Jan-2026 21:21:18 CET
Begin processing the 35th record. run: 1102 subRun: 1 event: 135 at 15-Jan-2026 21:21:20 CET
Begin processing the 36th record. run: 1102 subRun: 1 event: 136 at 15-Jan-2026 21:21:23 CET
Begin processing the 37th record. run: 1102 subRun: 1 event: 137 at 15-Jan-2026 21:21:25 CET
Begin processing the 38th record. run: 1102 subRun: 1 event: 138 at 15-Jan-2026 21:21:27 CET
Begin processing the 39th record. run: 1102 subRun: 1 event: 139 at 15-Jan-2026 21:21:29 CET
Begin processing the 40th record. run: 1102 subRun: 1 event: 140 at 15-Jan-2026 21:21:31 CET
Begin processing the 41st record. run: 1102 subRun: 1 event: 141 at 15-Jan-2026 21:21:34 CET
Begin processing the 42nd record. run: 1102 subRun: 1 event: 142 at 15-Jan-2026 21:21:37 CET
Begin processing the 43rd record. run: 1102 subRun: 1 event: 143 at 15-Jan-2026 21:21:39 CET
Begin processing the 44th record. run: 1102 subRun: 1 event: 144 at 15-Jan-2026 21:21:41 CET
Begin processing the 45th record. run: 1102 subRun: 1 event: 145 at 15-Jan-2026 21:21:44 CET
Begin processing the 46th record. run: 1102 subRun: 1 event: 146 at 15-Jan-2026 21:21:46 CET
Begin processing the 47th record. run: 1102 subRun: 1 event: 147 at 15-Jan-2026 21:21:48 CET
Begin processing the 48th record. run: 1102 subRun: 1 event: 148 at 15-Jan-2026 21:21:50 CET
Begin processing the 49th record. run: 1102 subRun: 1 event: 149 at 15-Jan-2026 21:21:52 CET
Begin processing the 50th record. run: 1102 subRun: 1 event: 150 at 15-Jan-2026 21:21:55 CET
Begin processing the 51st record. run: 1102 subRun: 1 event: 151 at 15-Jan-2026 21:21:56 CET
Begin processing the 52nd record. run: 1102 subRun: 1 event: 152 at 15-Jan-2026 21:21:59 CET
Begin processing the 53rd record. run: 1102 subRun: 1 event: 153 at 15-Jan-2026 21:22:01 CET
Begin processing the 54th record. run: 1102 subRun: 1 event: 154 at 15-Jan-2026 21:22:04 CET
Begin processing the 55th record. run: 1102 subRun: 1 event: 155 at 15-Jan-2026 21:22:06 CET
Begin processing the 56th record. run: 1102 subRun: 1 event: 156 at 15-Jan-2026 21:22:08 CET
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0349703 0.0242327 0.0235996 0.0223819 0.0241225 0.0212329 0.0221395 0.0246726 0.023218 0.0228461 0.0227135 0.0230808 0.0258702 
----view 1: 0.000689089 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0242327 0.0235996 0.0223819 0.0241225 0.0212329 0.0221395 0.0246726 0.023218 0.0228461 0.0227135 0.0230808 0.0258702 0.0260888 
----view 1: 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 0.000689059 0.000689089 0.000689089 0.00102687 
Begin processing the 57th record. run: 1102 subRun: 1 event: 157 at 15-Jan-2026 21:22:11 CET
Begin processing the 58th record. run: 1102 subRun: 1 event: 158 at 15-Jan-2026 21:22:13 CET
Begin processing the 59th record. run: 1102 subRun: 1 event: 159 at 15-Jan-2026 21:22:15 CET
Begin processing the 60th record. run: 1102 subRun: 1 event: 160 at 15-Jan-2026 21:22:17 CET
Begin processing the 61st record. run: 1102 subRun: 1 event: 161 at 15-Jan-2026 21:22:19 CET
Begin processing the 62nd record. run: 1102 subRun: 1 event: 162 at 15-Jan-2026 21:22:22 CET
Begin processing the 63rd record. run: 1102 subRun: 1 event: 163 at 15-Jan-2026 21:22:24 CET
Begin processing the 64th record. run: 1102 subRun: 1 event: 164 at 15-Jan-2026 21:22:26 CET
Begin processing the 65th record. run: 1102 subRun: 1 event: 165 at 15-Jan-2026 21:22:32 CET
Begin processing the 66th record. run: 1102 subRun: 1 event: 166 at 15-Jan-2026 21:22:35 CET
Begin processing the 67th record. run: 1102 subRun: 1 event: 167 at 15-Jan-2026 21:22:37 CET
Begin processing the 68th record. run: 1102 subRun: 1 event: 168 at 15-Jan-2026 21:22:39 CET
Begin processing the 69th record. run: 1102 subRun: 1 event: 169 at 15-Jan-2026 21:22:42 CET
Begin processing the 70th record. run: 1102 subRun: 1 event: 170 at 15-Jan-2026 21:22:43 CET
Begin processing the 71st record. run: 1102 subRun: 1 event: 171 at 15-Jan-2026 21:22:45 CET
Begin processing the 72nd record. run: 1102 subRun: 1 event: 172 at 15-Jan-2026 21:22:47 CET
Begin processing the 73rd record. run: 1102 subRun: 1 event: 173 at 15-Jan-2026 21:22:50 CET
Begin processing the 74th record. run: 1102 subRun: 1 event: 174 at 15-Jan-2026 21:22:52 CET
Begin processing the 75th record. run: 1102 subRun: 1 event: 175 at 15-Jan-2026 21:22:54 CET
Begin processing the 76th record. run: 1102 subRun: 1 event: 176 at 15-Jan-2026 21:22:56 CET
Begin processing the 77th record. run: 1102 subRun: 1 event: 177 at 15-Jan-2026 21:22:57 CET
Begin processing the 78th record. run: 1102 subRun: 1 event: 178 at 15-Jan-2026 21:23:00 CET
Begin processing the 79th record. run: 1102 subRun: 1 event: 179 at 15-Jan-2026 21:23:02 CET
Begin processing the 80th record. run: 1102 subRun: 1 event: 180 at 15-Jan-2026 21:23:06 CET
Begin processing the 81st record. run: 1102 subRun: 1 event: 181 at 15-Jan-2026 21:23:08 CET
Begin processing the 82nd record. run: 1102 subRun: 1 event: 182 at 15-Jan-2026 21:23:11 CET
Begin processing the 83rd record. run: 1102 subRun: 1 event: 183 at 15-Jan-2026 21:23:13 CET
Begin processing the 84th record. run: 1102 subRun: 1 event: 184 at 15-Jan-2026 21:23:15 CET
Begin processing the 85th record. run: 1102 subRun: 1 event: 185 at 15-Jan-2026 21:23:17 CET
Begin processing the 86th record. run: 1102 subRun: 1 event: 186 at 15-Jan-2026 21:23:19 CET
Begin processing the 87th record. run: 1102 subRun: 1 event: 187 at 15-Jan-2026 21:23:27 CET
Begin processing the 88th record. run: 1102 subRun: 1 event: 188 at 15-Jan-2026 21:23:29 CET
Begin processing the 89th record. run: 1102 subRun: 1 event: 189 at 15-Jan-2026 21:23:31 CET
Begin processing the 90th record. run: 1102 subRun: 1 event: 190 at 15-Jan-2026 21:23:33 CET
Begin processing the 91st record. run: 1102 subRun: 1 event: 191 at 15-Jan-2026 21:23:36 CET
Begin processing the 92nd record. run: 1102 subRun: 1 event: 192 at 15-Jan-2026 21:23:38 CET
Begin processing the 93rd record. run: 1102 subRun: 1 event: 193 at 15-Jan-2026 21:23:41 CET
Begin processing the 94th record. run: 1102 subRun: 1 event: 194 at 15-Jan-2026 21:23:43 CET
Begin processing the 95th record. run: 1102 subRun: 1 event: 195 at 15-Jan-2026 21:23:46 CET
Begin processing the 96th record. run: 1102 subRun: 1 event: 196 at 15-Jan-2026 21:23:48 CET
Begin processing the 97th record. run: 1102 subRun: 1 event: 197 at 15-Jan-2026 21:23:51 CET
Begin processing the 98th record. run: 1102 subRun: 1 event: 198 at 15-Jan-2026 21:23:53 CET
Begin processing the 99th record. run: 1102 subRun: 1 event: 199 at 15-Jan-2026 21:23:55 CET
Begin processing the 100th record. run: 1102 subRun: 1 event: 200 at 15-Jan-2026 21:23:57 CET
15-Jan-2026 21:24:00 CET  Closed output file "nu_dune10kt_1x2x6_1102_1_20230826T063255Z_gen_g4_detsim_hitreco__20240229T180222Z_reco2_reco2.root"
15-Jan-2026 21:24:00 CET  Closed input file "root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/03/db/nu_dune10kt_1x2x6_1102_1_20230826T063255Z_gen_g4_detsim_hitreco__20240229T180222Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      1.75558       2.27527       7.63426       2.07866       0.74126        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00192887     0.0125756     0.0357705     0.0139789    0.00668205       100    
reco:pandora:StandardPandora                    1.75081       2.25454       7.59874       2.06353      0.736365        100    
[art]:TriggerResults:TriggerResultInserter     9.728e-06    1.87229e-05   6.9001e-05    1.7082e-05    9.08875e-06      100    
end_path:out1:RootOutput                       3.115e-06    3.93365e-06   1.6841e-05     3.707e-06    1.6178e-06       100    
end_path:out1:RootOutput(write)               0.00093939    0.00795901     0.043951     0.00619372    0.00712327       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2173.45 MB
  Peak resident set size usage (VmHWM): 1193.75 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===