Jobsub ID 303653.79@dunegpschedd01.fnal.gov
| Jobsub ID | 303653.79@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:52:24 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 00:33:45 |
| From worker node | Hostname | td108.pic.es |
| cpuinfo | AMD EPYC 9825 144-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:41:33 |
| Input files | vd-protodune:np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 50m (3032s) |
| CPU time | 42m (2564s = 84%) |
| Max RSS bytes | 2710294528 (2584 MiB) |
| Outputting started | 2026-02-04 00:32:07 |
| Output files | |
| Finished | 2026-02-04 00:33:45 |
| Saved logs | justin-logs:303653.79-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
x542b2180
Caught exception in looking up space point ptr... 0x542b2180, 0x542c3480
Caught exception in looking up space point ptr... 0x1fd1fe10, 0x359839e0
Caught exception in looking up space point ptr... 0x359839e0, 0x1fd1fe10
Caught exception in looking up space point ptr... 0x2ac41010, 0x43281560
Caught exception in looking up space point ptr... 0x43281560, 0x2ac41010
Caught exception in looking up space point ptr... 0x2ac41010, 0x3412ea70
Caught exception in looking up space point ptr... 0x3412ea70, 0x2ac41010
Caught exception in looking up space point ptr... 0x2ac41010, 0x43281410
Caught exception in looking up space point ptr... 0x43281410, 0x2ac41010
Caught exception in looking up space point ptr... 0x28325ea0, 0xbe748b0
Caught exception in looking up space point ptr... 0xbe748b0, 0x2064e810
Caught exception in looking up space point ptr... 0x2832d240, 0xbe7bc50
Caught exception in looking up space point ptr... 0xbe7bc50, 0xbe69500
Caught exception in looking up space point ptr... 0x2c92cdc0, 0x2c92cec0
Caught exception in looking up space point ptr... 0x2c92cec0, 0x3fee5a70
Caught exception in looking up space point ptr... 0x5f82b0c0, 0x5f82af30
Caught exception in looking up space point ptr... 0x5f82af30, 0x30905c40
Caught exception in looking up space point ptr... 0x2bfdd3d0, 0x2bfdd500
Caught exception in looking up space point ptr... 0x2bfdd500, 0x2bfdd3d0
Caught exception in looking up space point ptr... 0x39cbdce0, 0x28562590
Caught exception in looking up space point ptr... 0x28562590, 0x39cbdce0
Caught exception in looking up space point ptr... 0x4359f010, 0x4a65c7b0
Caught exception in looking up space point ptr... 0x4a65c7b0, 0x4a65c6c0
Caught exception in looking up space point ptr... 0x39e16be0, 0x14be3240
Caught exception in looking up space point ptr... 0x14be3240, 0x39e16be0
Caught exception in looking up space point ptr... 0x1430b2a0, 0x2d3fde00
Caught exception in looking up space point ptr... 0x2d3fde00, 0x1430b3a0
Caught exception in looking up space point ptr... 0x4b4348b0, 0x2a255170
Caught exception in looking up space point ptr... 0x2a255170, 0x4b4348b0
Caught exception in looking up space point ptr... 0x43eb5180, 0x35879fc0
Caught exception in looking up space point ptr... 0x35879fc0, 0x43eb5180
Caught exception in looking up space point ptr... 0x4a1dc4b0, 0x24df3f10
Caught exception in looking up space point ptr... 0x24df3f10, 0x5ee12f50
Caught exception in looking up space point ptr... 0xe4de350, 0x2f8a8840
Caught exception in looking up space point ptr... 0x2f8a8840, 0xe4de350
Caught exception in looking up space point ptr... 0x4eabe0b0, 0x3bf685e0
Caught exception in looking up space point ptr... 0x3bf685e0, 0x4eabe0b0
Caught exception in looking up space point ptr... 0x421a3780, 0x421a35f0
Caught exception in looking up space point ptr... 0x421a35f0, 0x421a3780
Caught exception in looking up space point ptr... 0x10b91730, 0x425439f0
Caught exception in looking up space point ptr... 0x425439f0, 0x10b91730
Caught exception in looking up space point ptr... 0x304a16e0, 0xde43680
Caught exception in looking up space point ptr... 0xde43680, 0x10b92c80
Caught exception in looking up space point ptr... 0xfbcabe0, 0xfbca720
Caught exception in looking up space point ptr... 0xfbca720, 0xfbcabe0
Caught exception in looking up space point ptr... 0xfbca850, 0x465b6cb0
Caught exception in looking up space point ptr... 0x465b6cb0, 0x20003db0
Caught exception in looking up space point ptr... 0x5f411900, 0x10b92b50
Caught exception in looking up space point ptr... 0x10b92b50, 0x5f411900
Caught exception in looking up space point ptr... 0x421a4400, 0x421a4270
Caught exception in looking up space point ptr... 0x421a4270, 0x421a4400
Caught exception in looking up space point ptr... 0xfbca5f0, 0x10b91f20
Caught exception in looking up space point ptr... 0x10b91f20, 0xfbca5f0
Caught exception in looking up space point ptr... 0x3e04dd00, 0xe4dd9f0
Caught exception in looking up space point ptr... 0xe4dd9f0, 0x3e04dd00
Caught exception in looking up space point ptr... 0x5b80f1c0, 0x5b80f6e0
Caught exception in looking up space point ptr... 0x5b80f6e0, 0x5b80f1c0
Caught exception in looking up space point ptr... 0x304a1de0, 0x304a0640
Caught exception in looking up space point ptr... 0x304a0640, 0x304a1de0
Caught exception in looking up space point ptr... 0x421a3dc0, 0x421a40e0
Caught exception in looking up space point ptr... 0x421a40e0, 0x421a3dc0
Caught exception in looking up space point ptr... 0x357db1c0, 0x3256cf90
Caught exception in looking up space point ptr... 0x3256cf90, 0x57a8a380
Caught exception in looking up space point ptr... 0x57a8a380, 0x3256cf90
Caught exception in looking up space point ptr... 0x3256cf90, 0x2e95a360
++++>>>> total num hits: 321588, num free: 315422
04-Feb-2026 01:32:01 CET Closed output file "np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505_reco_stage0_20260204T003201_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 80.4788 110.084 133.34 117.751 16.4318 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.696e-05 0.000121954 0.000656563 9.8051e-05 0.000106647 27
produce:tpcrawdecoder:PDVDTPCReader 14.9727 16.0913 27.1993 15.3455 2.49237 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.884739 0.912179 0.96153 0.911815 0.0184099 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0293673 0.0297498 0.0301203 0.0297258 0.000190103 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.145866 0.147358 0.148436 0.147429 0.00061807 27
produce:beamevent:BeamEvent 0.00010909 0.000209413 0.00127143 0.000164211 0.000210332 27
produce:pdvddaphne:DAPHNEReaderPDVD 2.26161 2.40775 2.57816 2.40898 0.0903447 27
produce:ophit:OpHitFinder 0.0230646 0.0319518 0.038916 0.0322463 0.0036717 27
produce:wclsdatavd:WireCellToolkit 47.3959 50.1747 51.5413 50.184 0.898948 27
produce:gaushit:GausHitFinder 0.596257 1.01081 1.37106 1.06138 0.196818 27
produce:nhitsfilter:NumberOfHitsFilter 0.000234781 0.000281661 0.000579882 0.000265741 6.31742e-05 27
produce:reco3d:SpacePointSolver 7.05257 8.96766 11.5618 8.857 1.34147 27
produce:hitpdune:DisambigFromSpacePoints 0.0555893 0.161046 0.245159 0.175741 0.0496412 27
produce:cluster3d:Cluster3D 1.17169 26.5664 40.9585 34.533 14.9316 27
[art]:TriggerResults:TriggerResultInserter 3.5991e-05 5.72736e-05 0.000177151 5.307e-05 2.55343e-05 27
end_path:out1:RootOutput 9.1e-06 1.34749e-05 4.081e-05 1.182e-05 6.22467e-06 27
end_path:out1:RootOutput(write) 2.80407 3.54792 3.85438 3.69029 0.33553 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4690.32 MB
Peak resident set size usage (VmHWM): 2710.29 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505_reco_stage0_20260204T003201_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505_reco_stage0_20260204T003201_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505_reco_stage0_20260204T003201_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770162100.0,
"core.end_time": 1770165122.0,
"core.events": [
22263,
22269,
22275,
22281,
22287,
22293,
22299,
22305,
22311,
22317,
22323,
22329,
22335,
22341,
22347,
22353,
22359,
22365,
22371,
22377,
22383,
22389,
22395,
22401,
22407,
22413,
22419
],
"core.event_count": 27,
"core.first_event_number": 22263,
"core.last_event_number": 22419,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0137_df-s03-d1_dw_0_20260203T144505.hdf5"
}
]
} 