Jobsub ID 297939.0@dunegpschedd02.fnal.gov
| Jobsub ID | 297939.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12718 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 23:38:25 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 02:11:00 |
| From worker node | Hostname | gpu05.pic.es |
| cpuinfo | Intel(R) Xeon(R) Gold 6230R CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-04 00:14:21 |
| Input files | vd-protodune:np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (6869s) |
| CPU time | 1h (6326s = 92%) |
| Max RSS bytes | 3525513216 (3362 MiB) |
| Outputting started | 2026-02-04 02:08:53 |
| Output files | |
| Finished | 2026-02-04 02:11:00 |
| Saved logs | justin-logs:297939.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
x5e3b7ae0
Caught exception in looking up space point ptr... 0xb8796f0, 0xf867db0
Caught exception in looking up space point ptr... 0x37ff3bd0, 0x6bfc9f30
Caught exception in looking up space point ptr... 0xb2ba1c0, 0x37ff3bd0
Caught exception in looking up space point ptr... 0x12022480, 0x405f7a80
Caught exception in looking up space point ptr... 0x405f7a80, 0x2219e660
Caught exception in looking up space point ptr... 0x2219e660, 0x38670f30
Caught exception in looking up space point ptr... 0x38670f30, 0x12022480
Caught exception in looking up space point ptr... 0x45428c80, 0x454279c0
Caught exception in looking up space point ptr... 0x454279c0, 0x45431550
Caught exception in looking up space point ptr... 0x45431550, 0x45431870
Caught exception in looking up space point ptr... 0x48625e50, 0x3e149390
Caught exception in looking up space point ptr... 0x3e149390, 0x1605d220
Caught exception in looking up space point ptr... 0x40720fa0, 0x40720230
Caught exception in looking up space point ptr... 0x40720230, 0x413d8190
Caught exception in looking up space point ptr... 0x61b79020, 0x61b7ab00
Caught exception in looking up space point ptr... 0x61b7ab00, 0x61b79020
Caught exception in looking up space point ptr... 0x11482380, 0x11481fa0
Caught exception in looking up space point ptr... 0x11481fa0, 0x11482120
Caught exception in looking up space point ptr... 0x11482120, 0x11481bc0
Caught exception in looking up space point ptr... 0x43c708f0, 0x43c70660
Caught exception in looking up space point ptr... 0x43c70660, 0x33bd8f60
Caught exception in looking up space point ptr... 0x43c70e10, 0x43c708f0
Caught exception in looking up space point ptr... 0xb1cae60, 0xb1caff0
Caught exception in looking up space point ptr... 0xb1caff0, 0xacc12c0
Caught exception in looking up space point ptr... 0x44406bf0, 0x444085b0
Caught exception in looking up space point ptr... 0x444085b0, 0x4440e5d0
Caught exception in looking up space point ptr... 0x43c38f80, 0xdf5a2a0
Caught exception in looking up space point ptr... 0xdf5a2a0, 0x43c38f80
Caught exception in looking up space point ptr... 0x2a61dc20, 0xfa61030
Caught exception in looking up space point ptr... 0xfa61030, 0x2b4a6510
Caught exception in looking up space point ptr... 0x2b305780, 0x25f50590
Caught exception in looking up space point ptr... 0x624bd0f0, 0x2b305780
Caught exception in looking up space point ptr... 0x21f6b0b0, 0x21f6b200
Caught exception in looking up space point ptr... 0x21f6b200, 0x21f6b350
Caught exception in looking up space point ptr... 0x34af7450, 0x34af76f0
Caught exception in looking up space point ptr... 0x34af76f0, 0x34af7450
Caught exception in looking up space point ptr... 0x2e4ec590, 0x44410bb0
Caught exception in looking up space point ptr... 0x44410bb0, 0x44411510
Caught exception in looking up space point ptr... 0x102e45d0, 0x25fae160
Caught exception in looking up space point ptr... 0x25fae160, 0x102e45d0
Caught exception in looking up space point ptr... 0x2db01ad0, 0x2db00570
Caught exception in looking up space point ptr... 0x2db00570, 0x2db01ad0
Caught exception in looking up space point ptr... 0x34660e20, 0x6178c690
Caught exception in looking up space point ptr... 0x6178c690, 0x44b57690
Caught exception in looking up space point ptr... 0x412899d0, 0x2b14f230
Caught exception in looking up space point ptr... 0x2b14f230, 0x412874b0
Caught exception in looking up space point ptr... 0x412874b0, 0x48621dc0
Caught exception in looking up space point ptr... 0x315d4600, 0x315d4560
Caught exception in looking up space point ptr... 0x315d4560, 0x315d4600
Caught exception in looking up space point ptr... 0xb8bd440, 0x376c0ca0
Caught exception in looking up space point ptr... 0x376c0ca0, 0x4e796c40
Caught exception in looking up space point ptr... 0x460dcd00, 0x26f2f120
Caught exception in looking up space point ptr... 0x26f2f120, 0x460dcd00
Caught exception in looking up space point ptr... 0x460dcd00, 0x4f579f20
Caught exception in looking up space point ptr... 0x4f579f20, 0x460dcd00
Caught exception in looking up space point ptr... 0x31dc77d0, 0x5af1ff80
Caught exception in looking up space point ptr... 0x5af1ff80, 0x5af200f0
Caught exception in looking up space point ptr... 0x5af200f0, 0x4a0f6570
Caught exception in looking up space point ptr... 0x4a0f6570, 0x31dc77d0
Caught exception in looking up space point ptr... 0x91e44a0, 0x91e3ff0
Caught exception in looking up space point ptr... 0x91e3ff0, 0x91e44a0
Caught exception in looking up space point ptr... 0x26adf8f0, 0x26adeaa0
Caught exception in looking up space point ptr... 0x26adeaa0, 0x26adebe0
Caught exception in looking up space point ptr... 0x2fd433b0, 0x2fd1b5a0
Caught exception in looking up space point ptr... 0x2fd1b5a0, 0x2fd433b0
++++>>>> total num hits: 50169, num free: 48086
04-Feb-2026 03:08:39 CET Closed output file "np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714_reco_stage0_20260204T020839_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 160.928 248.833 569.618 201.66 115.757 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 9.5302e-05 0.000149032 0.000456402 0.000131256 6.50132e-05 27
produce:tpcrawdecoder:PDVDTPCReader 18.5677 19.5261 28.8394 19.213 1.84034 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.887714 0.928228 1.01281 0.921824 0.0283314 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0299547 0.0361807 0.0726313 0.0301073 0.0113801 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.147862 0.151741 0.178076 0.148414 0.00925833 27
produce:beamevent:BeamEvent 0.00016272 0.00026882 0.00198854 0.000195973 0.000338924 27
produce:pdvddaphne:DAPHNEReaderPDVD 8.74771 10.2156 12.1888 10.0375 0.847725 27
produce:ophit:OpHitFinder 0.0559553 0.090917 0.113693 0.0928517 0.0130858 27
produce:wclsdatavd:WireCellToolkit 112.503 139.995 160.365 141.641 11.8672 27
produce:gaushit:GausHitFinder 1.01822 2.47203 5.46979 2.24 1.10965 27
produce:nhitsfilter:NumberOfHitsFilter 0.000416552 0.000728465 0.00157328 0.000653616 0.00028961 27
produce:reco3d:SpacePointSolver 8.1671 17.461 37.8195 15.3327 8.05728 27
produce:hitpdune:DisambigFromSpacePoints 0.138519 0.415735 1.75118 0.249915 0.414249 27
produce:cluster3d:Cluster3D 1.41776 49.1717 327.203 2.94506 101.313 27
[art]:TriggerResults:TriggerResultInserter 3.0922e-05 7.86763e-05 0.000429575 5.2687e-05 7.45356e-05 27
end_path:out1:RootOutput 5.299e-06 1.20924e-05 4.3176e-05 9.114e-06 8.33767e-06 27
end_path:out1:RootOutput(write) 4.8764 8.29914 17.9205 7.63784 2.96229 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5803.22 MB
Peak resident set size usage (VmHWM): 3525.51 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714_reco_stage0_20260204T020839_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714_reco_stage0_20260204T020839_offline.root
Ran successfully
{
"name": "np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714_reco_stage0_20260204T020839_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770164079.0,
"core.end_time": 1770170922.0,
"core.events": [
15732,
15738,
15744,
15750,
15756,
15762,
15768,
15774,
15780,
15786,
15792,
15798,
15804,
15810,
15816,
15822,
15828,
15834,
15840,
15846,
15852,
15858,
15864,
15870,
15876,
15882,
15888
],
"core.event_count": 27,
"core.first_event_number": 15732,
"core.last_event_number": 15888,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42422
],
"core.runs_subruns": [
4242200001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042422_0097_df-s02-d0_dw_0_20260203T160714.hdf5"
}
]
} 