justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242001.45@dunegpschedd02.fnal.gov

Jobsub ID242001.45@dunegpschedd02.fnal.gov
Workflow ID9966
Stage ID1
User namesmanthey@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-11-12 06:14:51
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-11-13 08:58:21
From worker nodeHostnamedunegli-7572200-0-fnpc19114.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Started2025-11-13 05:14:19
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T042404Z_gen_001829_supernova_g4stage1_g4stage2_detsim_reco_20250627T113925Z_reco2.root
JobscriptExit code0
Real time3h (13433s)
CPU time3h (13075s = 97%)
Max RSS bytes1910988800 (1822 MiB)
Outputting started2025-11-13 08:58:13
Output files
Finished2025-11-13 08:58:21
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

y		EndPosition		TrackID	MotherID
--------------------------------------------------------------------------------
marley	22		2.22	(-465.04, -572.10, 377.89)	133541	133361
marley	1000010020	0.00	(-465.04, -572.10, 377.89)	133540	133361
marley	2212		0.00	(-461.94, -553.61, 316.99)	133514	133361
marley	2212		0.00	(-328.30, -644.54, 302.15)	133507	133361
marley	1000180360	0.00	(-230.77, -622.65, 177.46)	133498	133361
marley	1000180360	0.00	(-288.59, -650.48, 225.38)	133487	133361
marley	1000180400	0.00	(-277.06, -668.03, 203.78)	133485	133361
marley	1000180360	0.00	(-307.31, -665.64, 267.60)	133482	133361
marley	1000180360	0.00	(-301.95, -663.97, 278.85)	133481	133361
marley	1000180400	0.00	(-309.01, -631.59, 259.98)	133479	133361
marley	1000180400	0.00	(-203.88, -564.82, 249.06)	133475	133361
marley	1000180360	0.00	(-199.57, -595.11, 251.35)	133474	133361
marley	1000180400	0.00	(-226.28, -576.88, 322.94)	133473	133361
marley	1000180360	0.00	(-221.05, -607.52, 302.49)	133472	133361
marley	1000180400	0.00	(-217.75, -595.89, 299.29)	133471	133361
marley	1000180400	0.00	(-218.97, -606.53, 303.03)	133470	133361
marley	1000180400	0.00	(-231.50, -609.74, 277.87)	133469	133361
marley	1000180360	0.00	(-212.14, -604.94, 294.82)	133467	133361
marley	1000180360	0.00	(-220.03, -569.42, 344.45)	133466	133361
marley	1000180400	0.00	(-241.20, -560.37, 343.26)	133465	133361
marley	1000180360	0.00	(-221.71, -549.36, 316.16)	133463	133361
marley	1000180400	0.00	(-221.95, -549.45, 316.29)	133462	133361
marley	1000180360	0.00	(-219.93, -552.53, 310.58)	133461	133361
marley	1000180360	0.00	(-245.59, -537.17, 342.34)	133460	133361
marley	1000180360	0.00	(-291.85, -540.05, 292.81)	133459	133361
marley	1000180360	0.00	(-293.40, -539.37, 303.03)	133458	133361
marley	1000180360	0.00	(-287.52, -537.67, 293.25)	133457	133361
marley	1000180400	0.00	(-281.30, -544.18, 284.47)	133456	133361
marley	1000180360	0.00	(-274.53, -545.62, 285.25)	133455	133361
marley	1000180360	0.00	(-228.49, -517.18, 320.89)	133454	133361
marley	1000180400	0.00	(-230.30, -513.62, 314.56)	133453	133361
marley	1000180360	0.00	(-216.20, -487.26, 328.96)	133452	133361
marley	1000180360	0.00	(-193.43, -537.20, 334.41)	133451	133361
marley	1000180360	0.00	(-179.13, -561.87, 373.36)	133450	133361
marley	1000180360	0.00	(-190.36, -556.13, 375.93)	133449	133361
marley	1000180360	0.00	(-191.42, -557.63, 382.27)	133448	133361
marley	1000180360	0.00	(-159.48, -617.03, 388.92)	133447	133361
marley	1000180360	0.00	(-163.98, -617.77, 392.36)	133446	133361
marley	1000180400	0.00	(-154.02, -605.67, 401.77)	133445	133361
marley	1000180400	0.00	(-166.44, -601.54, 405.46)	133444	133361
marley	1000180360	0.00	(-196.91, -636.05, 380.39)	133443	133361
marley	1000180360	0.00	(-195.78, -632.92, 383.49)	133442	133361
marley	1000180360	0.00	(-225.59, -631.56, 368.76)	133441	133361
marley	1000180360	0.00	(-223.14, -634.20, 367.42)	133440	133361
marley	1000180360	0.00	(-203.11, -618.73, 335.48)	133439	133361
marley	1000180360	0.00	(-228.65, -605.97, 309.24)	133438	133361
marley	1000180360	0.00	(-232.71, -607.71, 314.80)	133437	133361
marley	1000180400	0.00	(-247.61, -621.25, 297.75)	133436	133361
marley	1000180360	0.00	(-288.82, -602.25, 314.26)	133435	133361
marley	1000180400	0.00	(-164.48, -613.13, 360.68)	133434	133361
marley	1000180400	0.00	(-164.31, -656.44, 349.93)	133432	133361
marley	1000180380	0.00	(-167.60, -651.77, 350.79)	133431	133361
marley	1000180400	0.00	(-163.80, -647.99, 309.57)	133430	133361
marley	1000180400	0.00	(-150.30, -725.22, 411.66)	133429	133361
marley	1000180400	0.00	(-128.30, -724.39, 419.07)	133428	133361
marley	1000180360	0.00	(-149.31, -721.90, 451.94)	133427	133361
marley	1000180360	0.00	(-145.17, -722.49, 556.23)	133426	133361
marley	1000180360	0.00	(-132.90, -750.71, 542.70)	133425	133361
marley	1000260570	0.00	(-139.47, -753.89, 548.07)	133424	133361
marley	2212		0.02	(-134.58, -755.61, 545.05)	133423	133361
marley	2212		0.04	(-127.89, -753.98, 548.17)	133422	133361
marley	2212		0.21	(-127.36, -754.42, 548.61)	133421	133361
marley	1000180360	0.00	(-124.52, -749.61, 575.06)	133420	133361
marley	1000180400	0.03	(-124.26, -750.15, 576.78)	133419	133361
marley	1000180400	0.01	(-124.89, -750.26, 575.11)	133418	133361
marley	1000180400	0.01	(-117.58, -750.91, 569.62)	133417	133361
marley	1000180400	0.01	(-113.27, -705.76, 510.82)	133416	133361
marley	1000180400	0.02	(-119.51, -681.62, 507.10)	133415	133361
marley	1000180400	0.02	(-128.60, -694.85, 486.49)	133414	133361
marley	1000180400	0.03	(-135.20, -718.15, 505.71)	133413	133361
marley	1000180400	0.04	(-137.20, -717.34, 504.93)	133412	133361
marley	1000180360	0.00	(-135.30, -717.76, 508.21)	133411	133361
marley	1000180400	0.01	(-133.85, -718.24, 509.60)	133410	133361
marley	1000180360	0.05	(-139.15, -720.64, 527.46)	133409	133361
marley	1000180400	0.05	(-126.95, -714.38, 455.44)	133408	133361
marley	1000180400	0.05	(-224.81, -725.28, 581.28)	133407	133361
marley	1000130270	0.03	(-220.55, -675.28, 528.09)	133406	133361
marley	1000180400	0.00	(-212.84, -671.39, 527.68)	133405	133361
marley	1000180400	0.06	(-211.07, -670.26, 527.35)	133404	133361
marley	1000130270	0.10	(-214.70, -674.93, 524.18)	133403	133361
marley	1000180400	0.00	(-209.36, -663.88, 521.02)	133402	133361
marley	1000180400	0.03	(-200.08, -637.79, 522.34)	133401	133361
marley	1000180400	0.01	(-190.44, -637.33, 547.71)	133400	133361
marley	1000180400	0.07	(-191.96, -630.62, 558.15)	133399	133361
marley	1000180400	0.09	(-192.63, -631.63, 557.93)	133398	133361
marley	1000180400	0.04	(-190.14, -601.11, 556.65)	133397	133361
marley	1000180400	0.00	(-188.31, -600.92, 555.56)	133396	133361
marley	1000180400	0.03	(-174.87, -602.30, 547.22)	133395	133361
marley	1000180400	0.09	(-168.63, -571.02, 520.48)	133394	133361
marley	1000180400	0.00	(-196.73, -576.40, 560.04)	133393	133361
marley	1000180400	0.02	(-197.10, -576.68, 560.76)	133392	133361
marley	1000180400	0.09	(-200.02, -577.30, 561.75)	133391	133361
marley	1000180400	0.01	(-195.46, -579.31, 565.34)	133390	133361
marley	1000180400	0.02	(-193.93, -581.97, 571.06)	133389	133361
marley	1000180400	0.03	(-171.40, -590.90, 580.95)	133388	133361
marley	1000180400	0.00	(-168.17, -593.35, 578.28)	133387	133361
marley	22		7.18	(-159.98, -595.14, 572.50)	133362	0
marley	2112		1.33	(-159.98, -595.14, 572.50)	133361	0
marley	1000190390	0.05	(-159.98, -595.14, 572.50)	133360	0
marley	11		46.03	(-159.98, -595.14, 572.50)	133359	0
Signal Track IDs: 133359; 133360; 133361; 133362; 133387; 133388; 133389; 133390; 133391; 133392; 133393; 133394; 133395; 133396; 133397; 133398; 133399; 133400; 133401; 133402; 133403; 133404; 133405; 133406; 133407; 133408; 133409; 133410; 133411; 133412; 133413; 133414; 133415; 133416; 133417; 133418; 133419; 133420; 133421; 133422; 133423; 133424; 133425; 133426; 133427; 133428; 133429; 133430; 133431; 133432; 133434; 133435; 133436; 133437; 133438; 133439; 133440; 133441; 133442; 133443; 133444; 133445; 133446; 133447; 133448; 133449; 133450; 133451; 133452; 133453; 133454; 133455; 133456; 133457; 133458; 133459; 133460; 133461; 133462; 133463; 133465; 133466; 133467; 133469; 133470; 133471; 133472; 133473; 133474; 133475; 133479; 133481; 133482; 133485; 133487; 133498; 133507; 133514; 133540; 133541; 
%MSG
%MSG-w ProducerUtils:  SolarNuAna:solarnuana@BeginModule  13-Nov-2025 08:58:12 UTC run: 4961 subRun: 0 event: 18290

Kinetic energy of signal particle is above threshold of 30.00 MeV. Skipping event.

%MSG
13-Nov-2025 08:58:12 UTC  Closed input file "root://fndcadoor.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/59/16/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20250128T042404Z_gen_001829_supernova_g4stage1_g4stage2_detsim_reco_20250627T113925Z_reco2.root"
%MSG-i ProducerUtils:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 08:58:12 UTC  ModuleEndJob
Finished running the SolarNuAna module
%MSG

====================================================================================================================
TimeTracker printout (sec)            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================
Full event                          1.18032       1337.65       3321.56       1104.49       1368.21        10     
--------------------------------------------------------------------------------------------------------------------
source:RootInput(read)            0.000781127   0.00644977     0.0145345    0.00540564     0.0055806       10     
end_path:solarnuana:SolarNuAna      1.1793        1337.65       3321.56       1104.48       1368.21        10     
====================================================================================================================
%MSG-i NuRandomService:  SolarNuAna:solarnuana@EndJob 13-Nov-2025 08:58:12 UTC  ModuleEndJob

Summary of seeds computed by the NuRandomService
Random policy: 'perEvent'
  algorithm version: EventTimestamp_v1

%MSG

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2530.09 MB
  Peak resident set size usage (VmHWM): 1910.99 MB
====================================================================================================

TrigReport ---------- Event summary -------------
TrigReport Events total = 10 passed = 10 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport         10         10          0 solarnuana

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 13060.141919 Real = 13414.750144

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 2530.09 VmHWM = 1910.99

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2026-02-10 06:23:43 UTC       justIN version: 01.06.00