Jobsub ID 297919.3@dunegpschedd02.fnal.gov
| Jobsub ID | 297919.3@dunegpschedd02.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:54:25 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce503 |
| Last heartbeat | 2026-02-04 00:19:01 |
| From worker node | Hostname | b9p05p0564.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:28:37 |
| Input files | vd-protodune:np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 47m (2853s) |
| CPU time | 45m (2751s = 96%) |
| Max RSS bytes | 2758234112 (2630 MiB) |
| Outputting started | 2026-02-04 00:16:12 |
| Output files | |
| Finished | 2026-02-04 00:19:01 |
| Saved logs | justin-logs:297919.3-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ght exception in looking up space point ptr... 0x34eb27a0, 0x34eb2610
Caught exception in looking up space point ptr... 0x2c9ef920, 0x283eef00
Caught exception in looking up space point ptr... 0x283eef00, 0x283ef090
Caught exception in looking up space point ptr... 0x24241a40, 0x4ea0d070
Caught exception in looking up space point ptr... 0x4ea0fce0, 0x24241a40
Caught exception in looking up space point ptr... 0x490d0970, 0x490caa40
Caught exception in looking up space point ptr... 0x490caa40, 0x10afd170
Caught exception in looking up space point ptr... 0x490de560, 0x5a056f10
Caught exception in looking up space point ptr... 0x5a056f10, 0x10b0be40
Caught exception in looking up space point ptr... 0x4ea0faf0, 0x4b0c3510
Caught exception in looking up space point ptr... 0x4b0c3510, 0x4b0c3e30
Caught exception in looking up space point ptr... 0x4d5c7670, 0x39690690
Caught exception in looking up space point ptr... 0x39690690, 0x4baaea60
Caught exception in looking up space point ptr... 0x10eb1610, 0x10ea4c80
Caught exception in looking up space point ptr... 0x10ea4c80, 0x39832c00
Caught exception in looking up space point ptr... 0x1623cc00, 0x261b3130
Caught exception in looking up space point ptr... 0x261b3130, 0x1623ca70
Caught exception in looking up space point ptr... 0x160a7e90, 0x160a7c40
Caught exception in looking up space point ptr... 0x160a7c40, 0x868f9f0
Caught exception in looking up space point ptr... 0x45f8fb10, 0x2d7071d0
Caught exception in looking up space point ptr... 0x2d7071d0, 0x45f8fb10
Caught exception in looking up space point ptr... 0x45f8fb10, 0x45f90760
Caught exception in looking up space point ptr... 0x45f90760, 0x45f8fb10
Caught exception in looking up space point ptr... 0x52c50de0, 0x52c50f80
Caught exception in looking up space point ptr... 0x52c50f80, 0x52c50de0
Caught exception in looking up space point ptr... 0x52c50de0, 0x52c51290
Caught exception in looking up space point ptr... 0x52c51290, 0x52c50de0
Caught exception in looking up space point ptr... 0x55a2c6d0, 0x148fcf80
Caught exception in looking up space point ptr... 0x148fcf80, 0x55a2c6d0
Caught exception in looking up space point ptr... 0x558d9010, 0x1138aa00
Caught exception in looking up space point ptr... 0x1138aa00, 0x558d9010
Caught exception in looking up space point ptr... 0x558d9010, 0x161e5420
Caught exception in looking up space point ptr... 0x161e5420, 0x558d9010
Caught exception in looking up space point ptr... 0x12057200, 0x24f32f90
Caught exception in looking up space point ptr... 0x24f32f90, 0x12057200
Caught exception in looking up space point ptr... 0x14fa9310, 0x14fa95a0
Caught exception in looking up space point ptr... 0x14fa95a0, 0x14fa90c0
Caught exception in looking up space point ptr... 0x1063c120, 0x1063bff0
Caught exception in looking up space point ptr... 0x1063bff0, 0x1063c250
Caught exception in looking up space point ptr... 0x1063c250, 0x1063c380
Caught exception in looking up space point ptr... 0x1063c380, 0x39400830
Caught exception in looking up space point ptr... 0xf390d20, 0x36ba5800
Caught exception in looking up space point ptr... 0x36ba5800, 0xf390d20
Caught exception in looking up space point ptr... 0x15f98cc0, 0x15f98410
Caught exception in looking up space point ptr... 0x15f98410, 0x2d1aa900
Caught exception in looking up space point ptr... 0x3e9407e0, 0x2dad84a0
Caught exception in looking up space point ptr... 0x2dad84a0, 0x3e9407e0
Caught exception in looking up space point ptr... 0x3929d870, 0x3e314c60
Caught exception in looking up space point ptr... 0x3e314c60, 0x3582b3f0
Caught exception in looking up space point ptr... 0x501bcd80, 0x4e909120
Caught exception in looking up space point ptr... 0x4e909120, 0x501bcd80
Caught exception in looking up space point ptr... 0x4d71a080, 0x8241570
Caught exception in looking up space point ptr... 0x8241570, 0x46a2e680
Caught exception in looking up space point ptr... 0x160acf80, 0x29448790
Caught exception in looking up space point ptr... 0x29448790, 0x160acf80
Caught exception in looking up space point ptr... 0x2d1a9f00, 0x5344eb50
Caught exception in looking up space point ptr... 0x5344eb50, 0x2d1a9f00
Caught exception in looking up space point ptr... 0x3d343d90, 0x5b601eb0
Caught exception in looking up space point ptr... 0x5b601eb0, 0x3d343d90
Caught exception in looking up space point ptr... 0x160ad1a0, 0x29449d40
Caught exception in looking up space point ptr... 0x29449d40, 0x160ad1a0
Caught exception in looking up space point ptr... 0x160abce0, 0x2f698e50
Caught exception in looking up space point ptr... 0x2f698e50, 0x294498c0
Caught exception in looking up space point ptr... 0x160ac500, 0x160ac6d0
Caught exception in looking up space point ptr... 0x160ac6d0, 0x160ac500
++++>>>> total num hits: 327156, num free: 320440
04-Feb-2026 01:16:03 CET Closed output file "np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644_reco_stage0_20260204T001603_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 71.6393 103.469 126.704 108.882 15.4806 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 4.3591e-05 0.000102413 0.000182942 9.0721e-05 4.06741e-05 27
produce:tpcrawdecoder:PDVDTPCReader 3.98163 4.45025 5.1649 4.43895 0.288849 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.00981839 0.0143549 0.0272356 0.0127441 0.00451667 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000393235 0.000536879 0.00103666 0.000484545 0.000137681 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.00149698 0.00186227 0.00304847 0.0017803 0.000319827 27
produce:beamevent:BeamEvent 4.368e-05 0.000128171 0.000896761 9.1721e-05 0.000155986 27
produce:pdvddaphne:DAPHNEReaderPDVD 2.0892 2.72731 3.60879 2.72508 0.354151 27
produce:ophit:OpHitFinder 0.0281088 0.0364231 0.0441324 0.0359895 0.00457692 27
produce:wclsdatavd:WireCellToolkit 53.1928 60.4954 68.4099 59.7859 4.07604 27
produce:gaushit:GausHitFinder 0.567724 1.12592 2.24615 1.12272 0.400247 27
produce:nhitsfilter:NumberOfHitsFilter 0.000166432 0.00027631 0.000476866 0.000249743 8.26791e-05 27
produce:reco3d:SpacePointSolver 3.50625 7.47705 13.7247 6.71055 2.56033 27
produce:hitpdune:DisambigFromSpacePoints 0.0469053 0.149349 0.260618 0.144267 0.0634157 27
produce:cluster3d:Cluster3D 0.776613 23.0645 35.9973 28.6344 13.28 27
[art]:TriggerResults:TriggerResultInserter 2.3201e-05 3.63024e-05 9.7361e-05 3.2e-05 1.53073e-05 27
end_path:out1:RootOutput 4.33e-06 8.19522e-06 2.731e-05 6.6e-06 5.08452e-06 27
end_path:out1:RootOutput(write) 2.97824 3.9024 5.2925 3.84009 0.605438 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4781.93 MB
Peak resident set size usage (VmHWM): 2758.23 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644_reco_stage0_20260204T001603_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644_reco_stage0_20260204T001603_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644_reco_stage0_20260204T001603_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770161328.0,
"core.end_time": 1770164165.0,
"core.events": [
10755,
10761,
10767,
10773,
10779,
10785,
10791,
10797,
10803,
10809,
10815,
10821,
10827,
10833,
10839,
10845,
10851,
10857,
10863,
10869,
10875,
10881,
10887,
10893,
10899,
10905,
10911
],
"core.event_count": 27,
"core.first_event_number": 10755,
"core.last_event_number": 10911,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0066_df-s03-d1_dw_0_20260203T140644.hdf5"
}
]
}