Jobsub ID 303653.63@dunegpschedd01.fnal.gov
| Jobsub ID | 303653.63@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:52:24 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 00:51:14 |
| From worker node | Hostname | td108.pic.es |
| cpuinfo | AMD EPYC 9825 144-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:40:44 |
| Input files | vd-protodune:np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (4121s) |
| CPU time | 59m (3599s = 87%) |
| Max RSS bytes | 8002011136 (7631 MiB) |
| Outputting started | 2026-02-04 00:49:28 |
| Output files | |
| Finished | 2026-02-04 00:51:14 |
| Saved logs | justin-logs:303653.63-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
g up space point ptr... 0x2b291e70, 0x2f6b6300
Caught exception in looking up space point ptr... 0x271a7bf0, 0x271a7280
Caught exception in looking up space point ptr... 0x271a7cf0, 0x271a7bf0
Caught exception in looking up space point ptr... 0x3ddb8e00, 0x47881aa0
Caught exception in looking up space point ptr... 0x47881aa0, 0x3de70860
Caught exception in looking up space point ptr... 0x34c66240, 0x34c66390
Caught exception in looking up space point ptr... 0x34c66390, 0x34c66650
Caught exception in looking up space point ptr... 0x92db1130, 0x6ba79a30
Caught exception in looking up space point ptr... 0x3f249330, 0x92db1130
Caught exception in looking up space point ptr... 0x3b2e0860, 0x58005ae0
Caught exception in looking up space point ptr... 0x58005ae0, 0x3b2e0860
Caught exception in looking up space point ptr... 0x2fb76e90, 0x2fb76110
Caught exception in looking up space point ptr... 0x2fb76110, 0x2fb76e90
Caught exception in looking up space point ptr... 0x8ef8e30, 0x8ef8fc0
Caught exception in looking up space point ptr... 0x8ef8fc0, 0x8ef8e30
Caught exception in looking up space point ptr... 0x61f85d70, 0x24c903f0
Caught exception in looking up space point ptr... 0x61f85be0, 0x61f85d70
Caught exception in looking up space point ptr... 0x5fa31cc0, 0x488e5060
Caught exception in looking up space point ptr... 0x488e5060, 0x5fa31cc0
Caught exception in looking up space point ptr... 0x62042e60, 0x8bb8960
Caught exception in looking up space point ptr... 0x8bb8960, 0x60b19d00
Caught exception in looking up space point ptr... 0x529b8c80, 0x34a813d0
Caught exception in looking up space point ptr... 0x34a813d0, 0x8cb5ed0
Caught exception in looking up space point ptr... 0x628275c0, 0x2772d520
Caught exception in looking up space point ptr... 0x2772d520, 0x628275c0
Caught exception in looking up space point ptr... 0x8ccb980, 0x8ccb880
Caught exception in looking up space point ptr... 0x8ccb880, 0x8ccb980
Caught exception in looking up space point ptr... 0x13b95860, 0x13b96180
Caught exception in looking up space point ptr... 0x6805d380, 0x13b95860
Caught exception in looking up space point ptr... 0x41f15400, 0x229e7830
Caught exception in looking up space point ptr... 0x4a500e40, 0x41f15400
Caught exception in looking up space point ptr... 0x3c7900c0, 0xdd79590
Caught exception in looking up space point ptr... 0xdd79590, 0x3c7900c0
Caught exception in looking up space point ptr... 0x233ea760, 0x22fa3740
Caught exception in looking up space point ptr... 0x22fa3740, 0x233ea760
Caught exception in looking up space point ptr... 0xa8bb970, 0x2f1dd590
Caught exception in looking up space point ptr... 0x2f1dd590, 0xa8bb970
Caught exception in looking up space point ptr... 0xa8bb970, 0x2f1dd7f0
Caught exception in looking up space point ptr... 0x2f1dd7f0, 0xa8bb970
Caught exception in looking up space point ptr... 0x56adb2d0, 0x31833e80
Caught exception in looking up space point ptr... 0x56adadf0, 0x56adb2d0
Caught exception in looking up space point ptr... 0x529391e0, 0x317b8390
Caught exception in looking up space point ptr... 0x317b8390, 0x27cb03b0
Caught exception in looking up space point ptr... 0x2b9d28a0, 0x529391e0
Caught exception in looking up space point ptr... 0x22be7dc0, 0x165e5bc0
Caught exception in looking up space point ptr... 0x165e5bc0, 0x278b78f0
Caught exception in looking up space point ptr... 0x115195c0, 0x2ca09590
Caught exception in looking up space point ptr... 0x2ca09590, 0x115195c0
Caught exception in looking up space point ptr... 0x115195c0, 0x115192a0
Caught exception in looking up space point ptr... 0x115192a0, 0x115195c0
Caught exception in looking up space point ptr... 0x115195c0, 0x3a4d7af0
Caught exception in looking up space point ptr... 0x3a4d7af0, 0x115195c0
Caught exception in looking up space point ptr... 0x115195c0, 0x11519430
Caught exception in looking up space point ptr... 0x11519430, 0x115195c0
Caught exception in looking up space point ptr... 0x1671b340, 0x1671a9a0
Caught exception in looking up space point ptr... 0x1671a9a0, 0x1671b160
Caught exception in looking up space point ptr... 0x4e249160, 0xac64140
Caught exception in looking up space point ptr... 0x424d9450, 0x4e249160
Caught exception in looking up space point ptr... 0x3a06eb60, 0x611879c0
Caught exception in looking up space point ptr... 0x38126480, 0x3a06eb60
Caught exception in looking up space point ptr... 0xb831d30, 0x34d4b8e0
Caught exception in looking up space point ptr... 0x34d4b8e0, 0xb831d30
Caught exception in looking up space point ptr... 0x8ef8470, 0x8ef82d0
Caught exception in looking up space point ptr... 0x8ef82d0, 0x8ef8710
Caught exception in looking up space point ptr... 0x3182c9c0, 0x4f6d1e50
Caught exception in looking up space point ptr... 0x4f6d1e50, 0x3182c9c0
++++>>>> total num hits: 47876, num free: 45891
04-Feb-2026 01:49:19 CET Closed output file "np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729_reco_stage0_20260204T004919_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 75.2601 148.902 654.563 82.901 150.478 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.729e-05 9.90622e-05 0.000198881 8.389e-05 3.50671e-05 27
produce:tpcrawdecoder:PDVDTPCReader 15.9642 16.4049 16.7817 16.3794 0.236411 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.886406 0.946546 1.05474 0.933668 0.0435081 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0296622 0.0342807 0.0644734 0.0300633 0.00813852 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.146985 0.152234 0.178216 0.148009 0.01056 27
produce:beamevent:BeamEvent 0.000110661 0.000163915 0.000885784 0.000134701 0.000142923 27
produce:pdvddaphne:DAPHNEReaderPDVD 2.19321 2.45202 2.72173 2.43935 0.133458 27
produce:ophit:OpHitFinder 0.0251938 0.0321997 0.0384893 0.0331107 0.00333746 27
produce:wclsdatavd:WireCellToolkit 44.3594 47.4354 52.3258 46.9365 2.32716 27
produce:gaushit:GausHitFinder 0.480646 1.29176 4.61092 0.73824 1.2235 27
produce:nhitsfilter:NumberOfHitsFilter 0.000163341 0.000279529 0.000638722 0.000217541 0.000141592 27
produce:reco3d:SpacePointSolver 4.66303 16.4201 62.1227 9.86556 15.5377 27
produce:hitpdune:DisambigFromSpacePoints 0.0424963 0.579693 3.69373 0.0867871 1.09074 27
produce:cluster3d:Cluster3D 0.757612 58.932 521.26 1.74409 129.75 27
[art]:TriggerResults:TriggerResultInserter 2.178e-05 4.06682e-05 9.1e-05 3.646e-05 1.40829e-05 27
end_path:out1:RootOutput 5.01e-06 1.00908e-05 2.851e-05 8.85e-06 5.29651e-06 27
end_path:out1:RootOutput(write) 2.7181 4.17228 11.1182 2.9689 2.62629 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 10162.1 MB
Peak resident set size usage (VmHWM): 8002.01 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729_reco_stage0_20260204T004919_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729_reco_stage0_20260204T004919_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729_reco_stage0_20260204T004919_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770162065.0,
"core.end_time": 1770166161.0,
"core.events": [
1977,
1983,
1989,
1995,
2001,
2007,
2013,
2019,
2025,
2031,
2037,
2043,
2049,
2055,
2061,
2067,
2073,
2079,
2085,
2091,
2097,
2103,
2109,
2115,
2121,
2127,
2133
],
"core.event_count": 27,
"core.first_event_number": 1977,
"core.last_event_number": 2133,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0012_df-s03-d1_dw_0_20260203T133729.hdf5"
}
]
} 