Jobsub ID 303703.0@dunegpschedd01.fnal.gov
| Jobsub ID | 303703.0@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-04 00:32:28 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 03:18:37 |
| From worker node | Hostname | tdm003.pic.es |
| cpuinfo | AMD EPYC 7662 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-04 00:33:28 |
| Input files | vd-protodune:np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9739s) |
| CPU time | 2h (9126s = 93%) |
| Max RSS bytes | 11475308544 (10943 MiB) |
| Outputting started | 2026-02-04 03:15:52 |
| Output files | |
| Finished | 2026-02-04 03:18:37 |
| Saved logs | justin-logs:303703.0-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
p space point ptr... 0xb589ce50, 0xb589ccc0
Caught exception in looking up space point ptr... 0x3e750db0, 0xd648210
Caught exception in looking up space point ptr... 0x3e750550, 0x3e750db0
Caught exception in looking up space point ptr... 0x913a4860, 0x98540f0
Caught exception in looking up space point ptr... 0x98540f0, 0x9854200
Caught exception in looking up space point ptr... 0x9854200, 0x9854330
Caught exception in looking up space point ptr... 0x13aa76670, 0x13aa76540
Caught exception in looking up space point ptr... 0x90ac0d30, 0x13aa76670
Caught exception in looking up space point ptr... 0x29aaf140, 0x5bd84930
Caught exception in looking up space point ptr... 0x5bd84930, 0x5bd849d0
Caught exception in looking up space point ptr... 0x1146028b0, 0x2b8ab220
Caught exception in looking up space point ptr... 0xe0520350, 0xe0520510
Caught exception in looking up space point ptr... 0xe0520510, 0x114601590
Caught exception in looking up space point ptr... 0x114601590, 0x1146028b0
Caught exception in looking up space point ptr... 0x32997cf0, 0x989ad380
Caught exception in looking up space point ptr... 0x11362550, 0x32997cf0
Caught exception in looking up space point ptr... 0x301e0a30, 0x15999ce0
Caught exception in looking up space point ptr... 0x15999ce0, 0x33742ce0
Caught exception in looking up space point ptr... 0x11360350, 0x14eabf1c0
Caught exception in looking up space point ptr... 0x14eabf1c0, 0x14eabf8a0
Caught exception in looking up space point ptr... 0xcda7a540, 0xcda7a3b0
Caught exception in looking up space point ptr... 0xcda7a3b0, 0x1454a39d0
Caught exception in looking up space point ptr... 0xdabb8920, 0xdabb8ee0
Caught exception in looking up space point ptr... 0x754a3e00, 0xdabb8920
Caught exception in looking up space point ptr... 0x90ed05b0, 0x2fcc61e0
Caught exception in looking up space point ptr... 0x2fcc61e0, 0x580b1ee0
Caught exception in looking up space point ptr... 0xf6cf5320, 0x2b8e4fa0
Caught exception in looking up space point ptr... 0x2b8e4fa0, 0xf6cf5320
Caught exception in looking up space point ptr... 0x7e1d3b90, 0x7e1d3a30
Caught exception in looking up space point ptr... 0x7e1d3a30, 0xf6cf4fe0
Caught exception in looking up space point ptr... 0x175aae440, 0x10df57630
Caught exception in looking up space point ptr... 0xca474fa0, 0x175aae440
Caught exception in looking up space point ptr... 0x107bc3e50, 0x3acacf00
Caught exception in looking up space point ptr... 0xefb94020, 0x107bc3e50
Caught exception in looking up space point ptr... 0xa9d1a070, 0xbcf85740
Caught exception in looking up space point ptr... 0xbcf85740, 0xa9d1a070
Caught exception in looking up space point ptr... 0x3ac65900, 0x3ac65800
Caught exception in looking up space point ptr... 0x3ac64e00, 0x3ac65900
Caught exception in looking up space point ptr... 0x29caaea0, 0x2d90c620
Caught exception in looking up space point ptr... 0x2d90c620, 0x3c65ed00
Caught exception in looking up space point ptr... 0xa9c95390, 0x29caaea0
Caught exception in looking up space point ptr... 0x1475d2e0, 0x1475cf70
Caught exception in looking up space point ptr... 0x1475cf70, 0x1475d470
Caught exception in looking up space point ptr... 0x89b1a990, 0x89b1a800
Caught exception in looking up space point ptr... 0x89b1a800, 0x89b1a990
Caught exception in looking up space point ptr... 0x74d27e40, 0x3378fca0
Caught exception in looking up space point ptr... 0x3378fca0, 0x74d27e40
Caught exception in looking up space point ptr... 0x3a2e66b0, 0x1475f4a0
Caught exception in looking up space point ptr... 0x1475f4a0, 0x3a2e66b0
Caught exception in looking up space point ptr... 0x32cef430, 0xb1acd60
Caught exception in looking up space point ptr... 0xb1acd60, 0x32cef340
Caught exception in looking up space point ptr... 0x30bef0b0, 0x3afad7e0
Caught exception in looking up space point ptr... 0x3afad7e0, 0x30bef0b0
Caught exception in looking up space point ptr... 0x13648580, 0x5982df80
Caught exception in looking up space point ptr... 0x5982df80, 0x13648580
Caught exception in looking up space point ptr... 0x12717780, 0x121166c0
Caught exception in looking up space point ptr... 0x121166c0, 0x12717780
Caught exception in looking up space point ptr... 0xe734f7e0, 0x11020b590
Caught exception in looking up space point ptr... 0x11020b590, 0xe734f550
Caught exception in looking up space point ptr... 0x38db0bc0, 0xa8290ce0
Caught exception in looking up space point ptr... 0xa8290ce0, 0x38db0bc0
Caught exception in looking up space point ptr... 0x338b56d0, 0x2422dc90
Caught exception in looking up space point ptr... 0x338b5c50, 0x338b56d0
Caught exception in looking up space point ptr... 0x2b76fb80, 0x2b76f8d0
Caught exception in looking up space point ptr... 0x2b76f8d0, 0x2b76fb80
++++>>>> total num hits: 65256, num free: 63531
04-Feb-2026 04:15:40 CET Closed output file "np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224_reco_stage0_20260204T031540_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 95.181 344.489 1196.52 115.747 318.631 28
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.7691e-05 0.000112636 0.000216644 0.000100772 3.728e-05 28
produce:tpcrawdecoder:PDVDTPCReader 15.4029 15.9663 17.6999 15.8205 0.516227 28
produce:triggerrawdecoder:PDVDTriggerReader4 0.878602 0.907593 0.942268 0.909529 0.0157712 28
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0292431 0.0296939 0.0300786 0.0297672 0.000268385 28
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.145956 0.149486 0.181 0.147501 0.00825036 28
produce:beamevent:BeamEvent 5.8431e-05 0.000151253 0.00123385 0.000117997 0.00021029 28
produce:pdvddaphne:DAPHNEReaderPDVD 3.27866 4.33118 5.12214 4.47813 0.547609 28
produce:ophit:OpHitFinder 0.0319186 0.0363517 0.0433962 0.0358951 0.00334 28
produce:wclsdatavd:WireCellToolkit 65.7675 74.1411 86.0253 72.1625 5.36286 28
produce:gaushit:GausHitFinder 0.630598 4.31644 15.099 1.26775 4.42297 28
produce:nhitsfilter:NumberOfHitsFilter 0.000190653 0.000783873 0.00234031 0.000594561 0.000529668 28
produce:reco3d:SpacePointSolver 4.17458 37.7189 146.781 15.6419 37.2509 28
produce:hitpdune:DisambigFromSpacePoints 0.0577065 1.64112 6.58399 0.155648 2.05634 28
produce:cluster3d:Cluster3D 0.674853 196.403 906.068 2.12389 266.566 28
[art]:TriggerResults:TriggerResultInserter 2.0061e-05 4.00268e-05 0.000104182 3.37455e-05 1.94292e-05 28
end_path:out1:RootOutput 7.2e-06 1.05189e-05 3.144e-05 8.5755e-06 5.08539e-06 28
end_path:out1:RootOutput(write) 3.53795 8.76726 22.3774 4.73618 6.12367 28
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 13726.4 MB
Peak resident set size usage (VmHWM): 11475.3 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224_reco_stage0_20260204T031540_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224_reco_stage0_20260204T031540_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224_reco_stage0_20260204T031540_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770165218.0,
"core.end_time": 1770174942.0,
"core.events": [
15458,
15464,
15470,
15476,
15482,
15488,
15494,
15500,
15506,
15512,
15518,
15524,
15530,
15536,
15542,
15548,
15554,
15560,
15566,
15572,
15578,
15584,
15590,
15596,
15602,
15608,
15614,
15620
],
"core.event_count": 28,
"core.first_event_number": 15458,
"core.last_event_number": 15620,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0095_df-s03-d0_dw_0_20260203T142224.hdf5"
}
]
} 