Jobsub ID 297921.86@dunegpschedd02.fnal.gov
| Jobsub ID | 297921.86@dunegpschedd02.fnal.gov |
| Workflow ID | 12719 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:58:26 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce15-multicore |
| Last heartbeat | 2026-02-04 04:01:42 |
| From worker node | Hostname | hnode63.pic.es |
| cpuinfo | AMD EPYC 7402P 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:53:58 |
| Input files | vd-protodune:np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 4h (14682s) |
| CPU time | 1h (6593s = 44%) |
| Max RSS bytes | 9814220800 (9359 MiB) |
| Outputting started | 2026-02-04 03:58:44 |
| Output files | |
| Finished | 2026-02-04 04:01:42 |
| Saved logs | justin-logs:297921.86-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
1b6588e0
Caught exception in looking up space point ptr... 0x3c7b1050, 0x39332fc0
Caught exception in looking up space point ptr... 0x774dd0d0, 0x3c7b1050
Caught exception in looking up space point ptr... 0x57a0b670, 0x57a0b030
Caught exception in looking up space point ptr... 0x57a0b030, 0x57a0b670
Caught exception in looking up space point ptr... 0x3c551350, 0x184e1a20
Caught exception in looking up space point ptr... 0x1b0fddf0, 0x3c551350
Caught exception in looking up space point ptr... 0x393fed10, 0x19fa23c0
Caught exception in looking up space point ptr... 0x5f3730e0, 0x393fed10
Caught exception in looking up space point ptr... 0xde52320, 0x5f60cd30
Caught exception in looking up space point ptr... 0x5f60cd30, 0x5f60cba0
Caught exception in looking up space point ptr... 0xd8491b70, 0x19f03bc0
Caught exception in looking up space point ptr... 0x19f03bc0, 0xd8491b70
Caught exception in looking up space point ptr... 0x45a0c8c0, 0x11256ca10
Caught exception in looking up space point ptr... 0x11256ca10, 0x11256cb10
Caught exception in looking up space point ptr... 0x107344380, 0x107348960
Caught exception in looking up space point ptr... 0x107348960, 0x4b166570
Caught exception in looking up space point ptr... 0x1667f9f00, 0x5ebc8a70
Caught exception in looking up space point ptr... 0x19a1ec30, 0x1667f9f00
Caught exception in looking up space point ptr... 0x42a6ce00, 0x42a6caa0
Caught exception in looking up space point ptr... 0x42a6c850, 0x42a6ce00
Caught exception in looking up space point ptr... 0x134087a0, 0x57cee780
Caught exception in looking up space point ptr... 0x57cee780, 0x44647830
Caught exception in looking up space point ptr... 0xd936590, 0xd936700
Caught exception in looking up space point ptr... 0xd936700, 0xd936590
Caught exception in looking up space point ptr... 0x627ef6c0, 0x627ef3b0
Caught exception in looking up space point ptr... 0x627ef3b0, 0x627ef6c0
Caught exception in looking up space point ptr... 0x137cfa60, 0x137d0180
Caught exception in looking up space point ptr... 0x19b023c0, 0x137cfa60
Caught exception in looking up space point ptr... 0x47a475f0, 0x7f0f5670
Caught exception in looking up space point ptr... 0x7f0f5670, 0x58129fc0
Caught exception in looking up space point ptr... 0x35adaa00, 0x117d33fc0
Caught exception in looking up space point ptr... 0x117d33fc0, 0x117d33b60
Caught exception in looking up space point ptr... 0x11601430, 0x115ff690
Caught exception in looking up space point ptr... 0x115ff690, 0x16542d220
Caught exception in looking up space point ptr... 0x19fb8ca0, 0xd83da850
Caught exception in looking up space point ptr... 0x128c43d10, 0x19fb8ca0
Caught exception in looking up space point ptr... 0x550d85b0, 0x39878950
Caught exception in looking up space point ptr... 0x39878950, 0x550d85b0
Caught exception in looking up space point ptr... 0x3d2e6350, 0x19c8da80
Caught exception in looking up space point ptr... 0x19c8da80, 0x3d2e6350
Caught exception in looking up space point ptr... 0x12905d230, 0x12905f190
Caught exception in looking up space point ptr... 0x12905f190, 0xa615e6a0
Caught exception in looking up space point ptr... 0x14aeaf20, 0x14aead20
Caught exception in looking up space point ptr... 0x14aead20, 0x14aeaf20
Caught exception in looking up space point ptr... 0x160eed20, 0x19f69dd0
Caught exception in looking up space point ptr... 0x19f69dd0, 0x14e005c90
Caught exception in looking up space point ptr... 0xa581e770, 0xd8535480
Caught exception in looking up space point ptr... 0xd8535480, 0x75ee2510
Caught exception in looking up space point ptr... 0x11f0f1270, 0x11f0f1170
Caught exception in looking up space point ptr... 0x11f0f1170, 0x11f0f1270
Caught exception in looking up space point ptr... 0x8efda80, 0x39da1560
Caught exception in looking up space point ptr... 0x39da1560, 0x8efda80
Caught exception in looking up space point ptr... 0x42a6ab50, 0x42a6ae40
Caught exception in looking up space point ptr... 0x42a6ae40, 0x42a6aa50
Caught exception in looking up space point ptr... 0x365f3100, 0xf54ed30
Caught exception in looking up space point ptr... 0xf54ed30, 0x365f3100
Caught exception in looking up space point ptr... 0x42bf2660, 0x1175d57a0
Caught exception in looking up space point ptr... 0x1175d57a0, 0x19c8d5c0
Caught exception in looking up space point ptr... 0x9bcdf960, 0x9bcd2a20
Caught exception in looking up space point ptr... 0x9bcd2a20, 0x9bcdefb0
Caught exception in looking up space point ptr... 0x9bcaaa10, 0x9bcaa8e0
Caught exception in looking up space point ptr... 0x9bcaa8e0, 0x9bcaaa10
Caught exception in looking up space point ptr... 0x9bcaaa10, 0x116e8be10
Caught exception in looking up space point ptr... 0x116e8be10, 0x9bcaaa10
Caught exception in looking up space point ptr... 0x726c9ae0, 0x726c5f10
Caught exception in looking up space point ptr... 0x726c5f10, 0x5a16dc60
++++>>>> total num hits: 80837, num free: 77942
04-Feb-2026 04:58:33 CET Closed output file "np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606_reco_stage0_20260204T035833_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 424.558 768.793 1472.08 624.682 327.261 19
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.6811e-05 8.75871e-05 0.000131924 8.8052e-05 1.60336e-05 19
produce:tpcrawdecoder:PDVDTPCReader 250.105 396.696 486.066 413.131 68.349 19
produce:triggerrawdecoder:PDVDTriggerReader4 1.27893 1.77017 2.44076 1.70548 0.313046 19
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0340221 0.0389772 0.0530121 0.0374465 0.00492804 19
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0338867 0.0385637 0.113177 0.0344477 0.0175884 19
produce:beamevent:BeamEvent 6.0292e-05 0.000245699 0.00131625 0.000130224 0.00032923 19
produce:pdvddaphne:DAPHNEReaderPDVD 22.5563 30.3237 39.3354 30.352 5.36929 19
produce:ophit:OpHitFinder 0.0234124 0.034048 0.0490836 0.0322175 0.00670052 19
produce:wclsdatavd:WireCellToolkit 101.523 115.007 123.553 115.833 6.31188 19
produce:gaushit:GausHitFinder 1.03247 2.94396 9.87716 1.68049 2.52956 19
produce:nhitsfilter:NumberOfHitsFilter 0.000427552 0.000900823 0.00212467 0.000723651 0.000521026 19
produce:reco3d:SpacePointSolver 13.0802 41.1526 101.205 27.3887 28.7163 19
produce:hitpdune:DisambigFromSpacePoints 0.174177 1.38487 5.5775 0.259918 1.93779 19
produce:cluster3d:Cluster3D 1.80467 170.846 738.14 4.02324 282.205 19
[art]:TriggerResults:TriggerResultInserter 1.9851e-05 3.63779e-05 9.0862e-05 2.7971e-05 1.85313e-05 19
end_path:out1:RootOutput 3.39e-06 6.32489e-06 2.0141e-05 4.29e-06 4.16007e-06 19
end_path:out1:RootOutput(write) 4.71493 8.46525 20.9369 5.2317 5.76997 19
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 12662.2 MB
Peak resident set size usage (VmHWM): 9814.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606_reco_stage0_20260204T035833_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606_reco_stage0_20260204T035833_offline.root
Ran successfully
{
"name": "np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606_reco_stage0_20260204T035833_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770162847.0,
"core.end_time": 1770177514.0,
"core.events": [
64192,
64198,
64204,
64210,
64216,
64222,
64228,
64234,
64240,
64246,
64252,
64258,
64264,
64270,
64276,
64282,
64288,
64294,
64300
],
"core.event_count": 19,
"core.first_event_number": 64192,
"core.last_event_number": 64300,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42433
],
"core.runs_subruns": [
4243300001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042433_0563_df-s04-d0_dw_0_20260203T221606.hdf5"
}
]
} 