Jobsub ID 297919.143@dunegpschedd02.fnal.gov
| Jobsub ID | 297919.143@dunegpschedd02.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:54:25 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce15-multicore |
| Last heartbeat | 2026-02-04 02:48:26 |
| From worker node | Hostname | hnode63.pic.es |
| cpuinfo | AMD EPYC 7402P 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:36:29 |
| Input files | vd-protodune:np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 3h (11397s) |
| CPU time | 37m (2268s = 19%) |
| Max RSS bytes | 2597949440 (2477 MiB) |
| Outputting started | 2026-02-04 02:46:27 |
| Output files | |
| Finished | 2026-02-04 02:48:26 |
| Saved logs | justin-logs:297919.143-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
d50
Caught exception in looking up space point ptr... 0x445a8d50, 0x445a8a30
Caught exception in looking up space point ptr... 0x445a8a30, 0x2aabe950
Caught exception in looking up space point ptr... 0x2aabe950, 0x445a8a30
Caught exception in looking up space point ptr... 0x296b2230, 0x615d6540
Caught exception in looking up space point ptr... 0x615d6540, 0x23185710
Caught exception in looking up space point ptr... 0x23185450, 0x296b2230
Caught exception in looking up space point ptr... 0x52201d90, 0x2a00ddb0
Caught exception in looking up space point ptr... 0x4ebb9970, 0x52201d90
Caught exception in looking up space point ptr... 0x586f8820, 0x586f89e0
Caught exception in looking up space point ptr... 0x2c532d80, 0x586f8820
Caught exception in looking up space point ptr... 0x43f8d430, 0x43f89ef0
Caught exception in looking up space point ptr... 0x43f8e500, 0x43f8d430
Caught exception in looking up space point ptr... 0x3ebf0470, 0x3ebecaf0
Caught exception in looking up space point ptr... 0x3ebecaf0, 0x3ebf0050
Caught exception in looking up space point ptr... 0x7e4c900, 0x338a53b0
Caught exception in looking up space point ptr... 0x3ea07860, 0x7e4c900
Caught exception in looking up space point ptr... 0x2a6ce920, 0x2dfe14d0
Caught exception in looking up space point ptr... 0x2dfe14d0, 0x460421a0
Caught exception in looking up space point ptr... 0x460421a0, 0xc75be20
Caught exception in looking up space point ptr... 0x3e2b4ea0, 0x3e2b5da0
Caught exception in looking up space point ptr... 0x3e2b5da0, 0x40919790
Caught exception in looking up space point ptr... 0x40919790, 0x40918e30
Caught exception in looking up space point ptr... 0x126c6b80, 0x126c6530
Caught exception in looking up space point ptr... 0x126c68f0, 0x126c6b80
Caught exception in looking up space point ptr... 0x7e58c20, 0x9e85f00
Caught exception in looking up space point ptr... 0x9e85f00, 0x9e860c0
Caught exception in looking up space point ptr... 0x618670b0, 0x420518d0
Caught exception in looking up space point ptr... 0x61866d00, 0x618670b0
Caught exception in looking up space point ptr... 0x3056dee0, 0x3056ba70
Caught exception in looking up space point ptr... 0x3056ba70, 0x3056dee0
Caught exception in looking up space point ptr... 0x33be6330, 0x5c8708c0
Caught exception in looking up space point ptr... 0x4d884fc0, 0x33be6330
Caught exception in looking up space point ptr... 0x3f9636d0, 0x3f9635a0
Caught exception in looking up space point ptr... 0x3f9635a0, 0x3f9626b0
Caught exception in looking up space point ptr... 0x3e2cb930, 0x3e2c7430
Caught exception in looking up space point ptr... 0x3e2c7430, 0x3e2cb930
Caught exception in looking up space point ptr... 0xc4faa30, 0xc4fa800
Caught exception in looking up space point ptr... 0xc4fa800, 0x4901b230
Caught exception in looking up space point ptr... 0x58b81a50, 0x58b818c0
Caught exception in looking up space point ptr... 0x58b818c0, 0x58b81a50
Caught exception in looking up space point ptr... 0x4e05f160, 0x3e4152d0
Caught exception in looking up space point ptr... 0x364f6880, 0x4e05f160
Caught exception in looking up space point ptr... 0x54d55f30, 0x11ecb340
Caught exception in looking up space point ptr... 0x54d560f0, 0x54d55f30
Caught exception in looking up space point ptr... 0x3dd18790, 0x3dd184f0
Caught exception in looking up space point ptr... 0x3dd184f0, 0x3dd18790
Caught exception in looking up space point ptr... 0x4e8790c0, 0x4e8793e0
Caught exception in looking up space point ptr... 0x4e8793e0, 0x4e879570
Caught exception in looking up space point ptr... 0x4d11bd20, 0x4d11ba00
Caught exception in looking up space point ptr... 0x4d11ba00, 0x4d11bd20
Caught exception in looking up space point ptr... 0x4d11bd20, 0x3dd11f20
Caught exception in looking up space point ptr... 0x3dd11f20, 0x4d11bd20
Caught exception in looking up space point ptr... 0x4d612b10, 0x58367c20
Caught exception in looking up space point ptr... 0x58367c20, 0x4d612ca0
Caught exception in looking up space point ptr... 0x4b80b7d0, 0x4b80b330
Caught exception in looking up space point ptr... 0x4b80b330, 0x4b80b7d0
Caught exception in looking up space point ptr... 0x4b80b7d0, 0x4b80b4d0
Caught exception in looking up space point ptr... 0x4b80b4d0, 0x4b80b7d0
Caught exception in looking up space point ptr... 0x164e0850, 0x164da9b0
Caught exception in looking up space point ptr... 0x164da9b0, 0x164e0850
Caught exception in looking up space point ptr... 0x3e2b1970, 0x409198a0
Caught exception in looking up space point ptr... 0x409198a0, 0x40910460
Caught exception in looking up space point ptr... 0x40910460, 0x40921450
Caught exception in looking up space point ptr... 0x48fac3b0, 0x48faaac0
Caught exception in looking up space point ptr... 0x48faaac0, 0x48fac3b0
++++>>>> total num hits: 94295, num free: 91391
04-Feb-2026 03:46:20 CET Closed output file "np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643_reco_stage0_20260204T024620_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 299.145 418.874 563.729 426.681 61.1034 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.7472e-05 9.47816e-05 0.000260107 8.7532e-05 3.59015e-05 27
produce:tpcrawdecoder:PDVDTPCReader 204.267 305.686 436.528 304.901 55.5074 27
produce:triggerrawdecoder:PDVDTriggerReader4 1.20831 1.94246 6.53308 1.80553 0.960269 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0361886 0.0586385 0.108025 0.0452587 0.0234196 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.179052 0.193873 0.282914 0.181021 0.0245061 27
produce:beamevent:BeamEvent 6.6522e-05 0.000160472 0.00142436 9.1293e-05 0.000250112 27
produce:pdvddaphne:DAPHNEReaderPDVD 22.5561 34.8044 54.565 35.1453 7.33907 27
produce:ophit:OpHitFinder 0.0288278 0.036429 0.0469958 0.0363942 0.00517072 27
produce:wclsdatavd:WireCellToolkit 56.1179 61.4312 69.4116 60.8176 3.32521 27
produce:gaushit:GausHitFinder 0.552153 0.896165 1.92887 0.868238 0.259071 27
produce:nhitsfilter:NumberOfHitsFilter 0.000159425 0.000294705 0.000577287 0.000264887 9.94284e-05 27
produce:reco3d:SpacePointSolver 4.36224 8.51451 15.7613 7.92193 2.83793 27
produce:hitpdune:DisambigFromSpacePoints 0.0493279 0.0851629 0.157509 0.0843705 0.0258118 27
produce:cluster3d:Cluster3D 0.676337 1.70672 2.93554 1.66147 0.587504 27
[art]:TriggerResults:TriggerResultInserter 1.7411e-05 2.85773e-05 9.5703e-05 2.353e-05 1.59348e-05 27
end_path:out1:RootOutput 3.16e-06 4.862e-06 2.1001e-05 3.87e-06 3.48818e-06 27
end_path:out1:RootOutput(write) 3.06155 3.48924 4.72373 3.42066 0.398522 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4546.44 MB
Peak resident set size usage (VmHWM): 2597.95 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643_reco_stage0_20260204T024620_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643_reco_stage0_20260204T024620_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643_reco_stage0_20260204T024620_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770161799.0,
"core.end_time": 1770173181.0,
"core.events": [
16755,
16761,
16767,
16773,
16779,
16785,
16791,
16797,
16803,
16809,
16815,
16821,
16827,
16833,
16839,
16845,
16851,
16857,
16863,
16869,
16875,
16881,
16887,
16893,
16899,
16905,
16911
],
"core.event_count": 27,
"core.first_event_number": 16755,
"core.last_event_number": 16911,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0103_df-s03-d1_dw_0_20260203T142643.hdf5"
}
]
} 