Jobsub ID 303651.125@dunegpschedd01.fnal.gov
| Jobsub ID | 303651.125@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:50:24 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce509 |
| Last heartbeat | 2026-02-03 23:56:33 |
| From worker node | Hostname | b9p34p1022.cern.ch |
| cpuinfo | AMD EPYC 9655 96-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:21:44 |
| Input files | vd-protodune:np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 32m (1956s) |
| CPU time | 31m (1880s = 96%) |
| Max RSS bytes | 2673025024 (2549 MiB) |
| Outputting started | 2026-02-03 23:54:22 |
| Output files | |
| Finished | 2026-02-03 23:56:33 |
| Saved logs | justin-logs:303651.125-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
90, 0x375868c0
Caught exception in looking up space point ptr... 0x3ad79ae0, 0x32a26290
Caught exception in looking up space point ptr... 0xbf6e760, 0xbf6b900
Caught exception in looking up space point ptr... 0xbf6b900, 0xbf6e760
Caught exception in looking up space point ptr... 0xbf6e760, 0xbf6c680
Caught exception in looking up space point ptr... 0xbf6c680, 0xbf6e760
Caught exception in looking up space point ptr... 0x2a447920, 0x4839b050
Caught exception in looking up space point ptr... 0x2cb80ec0, 0x2a447920
Caught exception in looking up space point ptr... 0x115b1d70, 0x2c088ba0
Caught exception in looking up space point ptr... 0x2c088ba0, 0x115b1d70
Caught exception in looking up space point ptr... 0x47ba2e10, 0x47ba3440
Caught exception in looking up space point ptr... 0x47ba3440, 0x47ba06f0
Caught exception in looking up space point ptr... 0x410e76e0, 0x410e6360
Caught exception in looking up space point ptr... 0x410e6360, 0x410e76e0
Caught exception in looking up space point ptr... 0xbf68ca0, 0xbf68b10
Caught exception in looking up space point ptr... 0xbf68b10, 0xbf68ca0
Caught exception in looking up space point ptr... 0x426df410, 0x426df730
Caught exception in looking up space point ptr... 0x426df730, 0x426df410
Caught exception in looking up space point ptr... 0xe4d2f60, 0x312aeb90
Caught exception in looking up space point ptr... 0x312aeb90, 0xe4d2f60
Caught exception in looking up space point ptr... 0x3dc38940, 0x3dc3e460
Caught exception in looking up space point ptr... 0x3dc3e460, 0x3dc3eb20
Caught exception in looking up space point ptr... 0x14bf58c0, 0x14bf60e0
Caught exception in looking up space point ptr... 0x14bf7930, 0x14bf58c0
Caught exception in looking up space point ptr... 0x272bc550, 0x272c00b0
Caught exception in looking up space point ptr... 0x272c00b0, 0x272ba060
Caught exception in looking up space point ptr... 0x31367820, 0x34b53e60
Caught exception in looking up space point ptr... 0x34b53e60, 0x4263fdf0
Caught exception in looking up space point ptr... 0x30bd67e0, 0x30bd68e0
Caught exception in looking up space point ptr... 0x30bd68e0, 0x30bd67e0
Caught exception in looking up space point ptr... 0xbf641d0, 0xbf64030
Caught exception in looking up space point ptr... 0xbf64030, 0xbf641d0
Caught exception in looking up space point ptr... 0x2cc8d950, 0x2702feb0
Caught exception in looking up space point ptr... 0x5992a590, 0x2cc8d950
Caught exception in looking up space point ptr... 0x483c59b0, 0x483c3c00
Caught exception in looking up space point ptr... 0x483c3c00, 0x483c3d70
Caught exception in looking up space point ptr... 0x1570ab60, 0xe44dc20
Caught exception in looking up space point ptr... 0xe44dc20, 0x554480c0
Caught exception in looking up space point ptr... 0x609db250, 0x609db000
Caught exception in looking up space point ptr... 0x609db000, 0x609daf30
Caught exception in looking up space point ptr... 0x274bad60, 0x12985490
Caught exception in looking up space point ptr... 0x12985490, 0x274bad60
Caught exception in looking up space point ptr... 0x22b80950, 0x274bae90
Caught exception in looking up space point ptr... 0x489ad270, 0x22b80950
Caught exception in looking up space point ptr... 0x2caebad0, 0x2caebb70
Caught exception in looking up space point ptr... 0x2caebb70, 0x2caebad0
Caught exception in looking up space point ptr... 0x569d8870, 0x228427c0
Caught exception in looking up space point ptr... 0x228427c0, 0x569d8870
Caught exception in looking up space point ptr... 0x2db10b40, 0x21fea680
Caught exception in looking up space point ptr... 0x21fea680, 0x2db10b40
Caught exception in looking up space point ptr... 0x272b39c0, 0x272af5e0
Caught exception in looking up space point ptr... 0x272af5e0, 0x272b39c0
Caught exception in looking up space point ptr... 0x25dbfb70, 0x25dbf880
Caught exception in looking up space point ptr... 0x25dbf880, 0x25dbf760
Caught exception in looking up space point ptr... 0x410e4700, 0x410e5050
Caught exception in looking up space point ptr... 0x410e5050, 0x410e4700
Caught exception in looking up space point ptr... 0x3fd38df0, 0x3fd38d20
Caught exception in looking up space point ptr... 0x3fd38d20, 0x42801620
Caught exception in looking up space point ptr... 0x21585350, 0x345dad20
Caught exception in looking up space point ptr... 0x345dad20, 0x21585600
Caught exception in looking up space point ptr... 0x1730f1a0, 0x30e8bf10
Caught exception in looking up space point ptr... 0x30e8bf10, 0x5edacd90
Caught exception in looking up space point ptr... 0x2146d270, 0x2146bfc0
Caught exception in looking up space point ptr... 0x2146c9b0, 0x2146d270
Caught exception in looking up space point ptr... 0x15e15b50, 0x15e14ef0
Caught exception in looking up space point ptr... 0x15e14ef0, 0x15e15b50
++++>>>> total num hits: 41003, num free: 39502
04-Feb-2026 00:54:15 CET Closed output file "np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223_reco_stage0_20260203T235415_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 57.8367 70.3342 104.315 67.8545 13.3583 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.6509e-05 9.4201e-05 0.000217045 8.4116e-05 2.94101e-05 27
produce:tpcrawdecoder:PDVDTPCReader 3.62235 4.14172 4.95902 4.07651 0.360325 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.0111843 0.0423475 0.0957584 0.0451176 0.0245909 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000390534 0.0036224 0.0267378 0.000660949 0.00618685 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.00136359 0.00246154 0.0114249 0.00192562 0.001834 27
produce:beamevent:BeamEvent 8.6529e-05 0.000159031 0.00117703 0.000118468 0.000201579 27
produce:pdvddaphne:DAPHNEReaderPDVD 1.68093 2.07109 2.80944 2.03494 0.269636 27
produce:ophit:OpHitFinder 0.0221022 0.0319291 0.0383376 0.0334628 0.00432461 27
produce:wclsdatavd:WireCellToolkit 42.7671 47.2644 54.5256 47.5292 3.06531 27
produce:gaushit:GausHitFinder 0.404353 0.688318 1.1623 0.692808 0.19019 27
produce:nhitsfilter:NumberOfHitsFilter 0.00012661 0.000210227 0.000812756 0.000189133 0.000125008 27
produce:reco3d:SpacePointSolver 3.54935 8.29484 15.5145 8.64853 2.99709 27
produce:hitpdune:DisambigFromSpacePoints 0.030044 0.0807895 0.188951 0.0704745 0.036454 27
produce:cluster3d:Cluster3D 0.394345 4.34981 28.9643 1.22961 7.95527 27
[art]:TriggerResults:TriggerResultInserter 2.4907e-05 4.03137e-05 0.000120561 3.3991e-05 1.85644e-05 27
end_path:out1:RootOutput 4.747e-06 7.33437e-06 2.5598e-05 6.33e-06 3.88343e-06 27
end_path:out1:RootOutput(write) 2.61942 3.33222 5.04141 3.13593 0.635715 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4782.1 MB
Peak resident set size usage (VmHWM): 2673.03 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223_reco_stage0_20260203T235415_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223_reco_stage0_20260203T235415_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223_reco_stage0_20260203T235415_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770160913.0,
"core.end_time": 1770162856.0,
"core.events": [
21445,
21451,
21457,
21463,
21469,
21475,
21481,
21487,
21493,
21499,
21505,
21511,
21517,
21523,
21529,
21535,
21541,
21547,
21553,
21559,
21565,
21571,
21577,
21583,
21589,
21595,
21601
],
"core.event_count": 27,
"core.first_event_number": 21445,
"core.last_event_number": 21601,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0132_df-s02-d1_dw_0_20260203T144223.hdf5"
}
]
}