Jobsub ID 303653.22@dunegpschedd01.fnal.gov
| Jobsub ID | 303653.22@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:52:24 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce509 |
| Last heartbeat | 2026-02-04 00:11:13 |
| From worker node | Hostname | b9p34p1022.cern.ch |
| cpuinfo | AMD EPYC 9655 96-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:37:16 |
| Input files | vd-protodune:np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 31m (1904s) |
| CPU time | 30m (1843s = 96%) |
| Max RSS bytes | 2552270848 (2434 MiB) |
| Outputting started | 2026-02-04 00:09:01 |
| Output files | |
| Finished | 2026-02-04 00:11:13 |
| Saved logs | justin-logs:303653.22-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
Caught exception in looking up space point ptr... 0x47d658b0, 0x47d65b50
Caught exception in looking up space point ptr... 0x44b95fd0, 0x44b95e40
Caught exception in looking up space point ptr... 0x44b95e40, 0x44b95fd0
Caught exception in looking up space point ptr... 0x561bda00, 0x561baa20
Caught exception in looking up space point ptr... 0x561baa20, 0x561b3780
Caught exception in looking up space point ptr... 0x3dd0fab0, 0x3dd12a90
Caught exception in looking up space point ptr... 0x3dd12a90, 0x3dd12bd0
Caught exception in looking up space point ptr... 0x44fe1670, 0x44fe5780
Caught exception in looking up space point ptr... 0x44fe2e40, 0x25eecbb0
Caught exception in looking up space point ptr... 0x25eecbb0, 0x44fe1840
Caught exception in looking up space point ptr... 0x44fe1740, 0x44fe1670
Caught exception in looking up space point ptr... 0x565a4300, 0x13814190
Caught exception in looking up space point ptr... 0x13814190, 0x565a4300
Caught exception in looking up space point ptr... 0x25eccbb0, 0x25eccd00
Caught exception in looking up space point ptr... 0x25eccd00, 0x25ecce50
Caught exception in looking up space point ptr... 0x37018cf0, 0x37019010
Caught exception in looking up space point ptr... 0x37016710, 0x37018cf0
Caught exception in looking up space point ptr... 0x21f99c00, 0x465c8100
Caught exception in looking up space point ptr... 0x465c8100, 0x21f99c00
Caught exception in looking up space point ptr... 0x59560ac0, 0x26a01450
Caught exception in looking up space point ptr... 0x26a01450, 0x59560ac0
Caught exception in looking up space point ptr... 0x3791d730, 0x3791da50
Caught exception in looking up space point ptr... 0x3791da50, 0x3791d730
Caught exception in looking up space point ptr... 0x3791d730, 0x3791dc30
Caught exception in looking up space point ptr... 0x3791dc30, 0x3791d730
Caught exception in looking up space point ptr... 0x336b10a0, 0x65da6ea0
Caught exception in looking up space point ptr... 0x65da6ea0, 0x336b10a0
Caught exception in looking up space point ptr... 0x561a2480, 0x3db54930
Caught exception in looking up space point ptr... 0x3db54930, 0x561a2660
Caught exception in looking up space point ptr... 0x635c9340, 0x635c9dc0
Caught exception in looking up space point ptr... 0x635c9dc0, 0x635c9340
Caught exception in looking up space point ptr... 0x3db482f0, 0x3db48160
Caught exception in looking up space point ptr... 0x3db48160, 0x3db482f0
Caught exception in looking up space point ptr... 0x30b29a50, 0x3c0adc80
Caught exception in looking up space point ptr... 0x3c0adc80, 0x30b29a50
Caught exception in looking up space point ptr... 0x25edc500, 0x25ee0290
Caught exception in looking up space point ptr... 0x25ee0290, 0x25ee1ba0
Caught exception in looking up space point ptr... 0x3db4d3c0, 0x3db4d050
Caught exception in looking up space point ptr... 0x3db4d050, 0x3db4d1e0
Caught exception in looking up space point ptr... 0x5f789cf0, 0x5f788e00
Caught exception in looking up space point ptr... 0x5f788e00, 0x5f78b200
Caught exception in looking up space point ptr... 0x8692ae0, 0x8692be0
Caught exception in looking up space point ptr... 0x8692be0, 0x8692ae0
Caught exception in looking up space point ptr... 0x214006f0, 0x213ffda0
Caught exception in looking up space point ptr... 0x213ffda0, 0x214006f0
Caught exception in looking up space point ptr... 0x341b34e0, 0x341b77c0
Caught exception in looking up space point ptr... 0x341b77c0, 0x341b74a0
Caught exception in looking up space point ptr... 0x2c198a10, 0x2c199560
Caught exception in looking up space point ptr... 0x2c199560, 0x2c198a10
Caught exception in looking up space point ptr... 0x3dceaa20, 0x3dceafd0
Caught exception in looking up space point ptr... 0x3dce7a30, 0x3dceaa20
Caught exception in looking up space point ptr... 0x4e12a480, 0x90591c0
Caught exception in looking up space point ptr... 0x90591c0, 0x90592f0
Caught exception in looking up space point ptr... 0x5d224a90, 0x5d224960
Caught exception in looking up space point ptr... 0x5d224960, 0x5d224a90
Caught exception in looking up space point ptr... 0x336b36f0, 0x336b3560
Caught exception in looking up space point ptr... 0x336b3560, 0x336b36f0
Caught exception in looking up space point ptr... 0x561ba0b0, 0x561b5430
Caught exception in looking up space point ptr... 0x561b5430, 0x561ba0b0
Caught exception in looking up space point ptr... 0x28cdf280, 0x28cdee30
Caught exception in looking up space point ptr... 0x28cdee30, 0x28cde050
Caught exception in looking up space point ptr... 0x3dd6e750, 0x242ae860
Caught exception in looking up space point ptr... 0x242ae860, 0x242ae790
Caught exception in looking up space point ptr... 0x55bd5fc0, 0x29679960
Caught exception in looking up space point ptr... 0x29679960, 0x54277e40
++++>>>> total num hits: 61852, num free: 59117
04-Feb-2026 01:08:54 CET Closed output file "np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856_reco_stage0_20260204T000854_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 62.3409 68.3166 80.8191 66.9556 4.39179 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.8908e-05 8.56218e-05 0.00022012 8.1051e-05 2.89959e-05 27
produce:tpcrawdecoder:PDVDTPCReader 2.74555 3.52391 4.82296 3.36418 0.538146 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.00924569 0.0148451 0.0350349 0.0124172 0.00734011 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000439027 0.000623054 0.00192628 0.000544835 0.000278787 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.00127333 0.00223407 0.00549386 0.00174851 0.00107822 27
produce:beamevent:BeamEvent 7.6745e-05 0.000143482 0.000867999 0.000107561 0.000145783 27
produce:pdvddaphne:DAPHNEReaderPDVD 1.75275 2.10029 2.52656 2.1089 0.164019 27
produce:ophit:OpHitFinder 0.022525 0.0334317 0.0417107 0.034444 0.00465355 27
produce:wclsdatavd:WireCellToolkit 44.1568 48.5806 54.5288 48.181 2.70538 27
produce:gaushit:GausHitFinder 0.44169 0.763694 1.10575 0.750625 0.173747 27
produce:nhitsfilter:NumberOfHitsFilter 0.000135894 0.00019268 0.000531886 0.000174772 7.24863e-05 27
produce:reco3d:SpacePointSolver 4.80191 8.26912 13.6904 7.85256 2.00952 27
produce:hitpdune:DisambigFromSpacePoints 0.0474906 0.0858212 0.186188 0.0795711 0.0322333 27
produce:cluster3d:Cluster3D 0.749051 1.57198 2.86353 1.60998 0.56897 27
[art]:TriggerResults:TriggerResultInserter 2.4237e-05 3.60998e-05 8.3185e-05 3.2378e-05 1.14545e-05 27
end_path:out1:RootOutput 4.947e-06 9.24011e-06 2.1913e-05 8.943e-06 2.91938e-06 27
end_path:out1:RootOutput(write) 2.73032 3.34071 4.72938 3.23228 0.453376 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4629.96 MB
Peak resident set size usage (VmHWM): 2552.27 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856_reco_stage0_20260204T000854_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856_reco_stage0_20260204T000854_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856_reco_stage0_20260204T000854_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770161846.0,
"core.end_time": 1770163735.0,
"core.events": [
23408,
23414,
23420,
23426,
23432,
23438,
23444,
23450,
23456,
23462,
23468,
23474,
23480,
23486,
23492,
23498,
23504,
23510,
23516,
23522,
23528,
23534,
23540,
23546,
23552,
23558,
23564
],
"core.event_count": 27,
"core.first_event_number": 23408,
"core.last_event_number": 23564,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0144_df-s03-d0_dw_0_20260203T144856.hdf5"
}
]
}