Jobsub ID 303646.60@dunegpschedd01.fnal.gov
| Jobsub ID | 303646.60@dunegpschedd01.fnal.gov |
| Workflow ID | 12718 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:44:24 |
| Site | NL_NIKHEF |
| Entry | VIRGO_NL_NIKHEF_klomp_multicore |
| Last heartbeat | 2026-02-04 00:53:53 |
| From worker node | Hostname | wn-snel-029.farm.nikhef.nl |
| cpuinfo | AMD EPYC 7H12 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 22:46:25 |
| Input files | vd-protodune:np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 2h (7507s) |
| CPU time | 52m (3149s = 41%) |
| Max RSS bytes | 2606850048 (2486 MiB) |
| Outputting started | 2026-02-04 00:51:33 |
| Output files | |
| Finished | 2026-02-04 00:53:53 |
| Saved logs | justin-logs:303646.60-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
exception in looking up space point ptr... 0x5f6b26e0, 0x5c7e21e0
Caught exception in looking up space point ptr... 0x5c7e21e0, 0x735bf0a0
Caught exception in looking up space point ptr... 0x8e89d380, 0x5c2fb510
Caught exception in looking up space point ptr... 0x5c2fb510, 0x87318b20
Caught exception in looking up space point ptr... 0x59568460, 0x59565eb0
Caught exception in looking up space point ptr... 0x59565eb0, 0x59568460
Caught exception in looking up space point ptr... 0x7d204750, 0x7d205160
Caught exception in looking up space point ptr... 0x7d205160, 0x7d2114a0
Caught exception in looking up space point ptr... 0x7d2114a0, 0x7d211640
Caught exception in looking up space point ptr... 0x7d211640, 0x7d20e580
Caught exception in looking up space point ptr... 0x48d22bc0, 0x48d228a0
Caught exception in looking up space point ptr... 0x48d228a0, 0x48d22710
Caught exception in looking up space point ptr... 0x48d22710, 0x48d23940
Caught exception in looking up space point ptr... 0x77793050, 0x777911b0
Caught exception in looking up space point ptr... 0x777911b0, 0x7778ad00
Caught exception in looking up space point ptr... 0x4ba9b070, 0x6a2ef0f0
Caught exception in looking up space point ptr... 0x6a2ef0f0, 0x4ba9b070
Caught exception in looking up space point ptr... 0x43be5aa0, 0x872461a0
Caught exception in looking up space point ptr... 0x872461a0, 0x43be58c0
Caught exception in looking up space point ptr... 0x87c53840, 0x87c45ac0
Caught exception in looking up space point ptr... 0x87c45ac0, 0x74d8fb60
Caught exception in looking up space point ptr... 0x68bcc2e0, 0x68bcc0b0
Caught exception in looking up space point ptr... 0x68bcc0b0, 0x68bcc2e0
Caught exception in looking up space point ptr... 0x74eea140, 0x6750c920
Caught exception in looking up space point ptr... 0x6750c920, 0x74eea140
Caught exception in looking up space point ptr... 0x97f506b0, 0x97f50840
Caught exception in looking up space point ptr... 0x97f50840, 0x97f506b0
Caught exception in looking up space point ptr... 0x43b906e0, 0x847e2620
Caught exception in looking up space point ptr... 0x847e2620, 0x7580d530
Caught exception in looking up space point ptr... 0x76739a60, 0x76717e10
Caught exception in looking up space point ptr... 0x76717e10, 0x76739a60
Caught exception in looking up space point ptr... 0x83323720, 0x76a46970
Caught exception in looking up space point ptr... 0x76a46970, 0x4b532060
Caught exception in looking up space point ptr... 0x4a1c4480, 0x4a1b8400
Caught exception in looking up space point ptr... 0x4a1b8400, 0x4a1b4b80
Caught exception in looking up space point ptr... 0x473e8020, 0x473e7e90
Caught exception in looking up space point ptr... 0x473e7e90, 0x473e84d0
Caught exception in looking up space point ptr... 0x758376a0, 0x75837400
Caught exception in looking up space point ptr... 0x75838030, 0x758376a0
Caught exception in looking up space point ptr... 0x6f1bc3b0, 0x6f1bc020
Caught exception in looking up space point ptr... 0x6f1bc020, 0x6f1bc3b0
Caught exception in looking up space point ptr... 0x4236c860, 0x4236c360
Caught exception in looking up space point ptr... 0x4236c360, 0x4236c860
Caught exception in looking up space point ptr... 0x775538e0, 0x775537b0
Caught exception in looking up space point ptr... 0x775537b0, 0x77553500
Caught exception in looking up space point ptr... 0x63dcc160, 0x63dcbfd0
Caught exception in looking up space point ptr... 0x63dcbfd0, 0x63dcbcb0
Caught exception in looking up space point ptr... 0x4a1a79c0, 0x4a1aac70
Caught exception in looking up space point ptr... 0x4a1aac70, 0x4a1a79c0
Caught exception in looking up space point ptr... 0x62e44f70, 0x6c0d0c30
Caught exception in looking up space point ptr... 0x6c0d0c30, 0x62e44f70
Caught exception in looking up space point ptr... 0x62382750, 0x62384ad0
Caught exception in looking up space point ptr... 0x62384ad0, 0x62382750
Caught exception in looking up space point ptr... 0x611d5120, 0x4488d8f0
Caught exception in looking up space point ptr... 0x587f7770, 0x611d5120
Caught exception in looking up space point ptr... 0x846c1070, 0x406eaa60
Caught exception in looking up space point ptr... 0x4aac3940, 0x846c1070
Caught exception in looking up space point ptr... 0x43300f30, 0x589b1590
Caught exception in looking up space point ptr... 0x589b1590, 0x43300f30
Caught exception in looking up space point ptr... 0x6094e920, 0x61609320
Caught exception in looking up space point ptr... 0x61609320, 0x6094e920
Caught exception in looking up space point ptr... 0x5c9ca540, 0x5c9ecdc0
Caught exception in looking up space point ptr... 0x5c9ecdc0, 0x5c9ca540
Caught exception in looking up space point ptr... 0x3ec504a0, 0x3ec50180
Caught exception in looking up space point ptr... 0x3ec50180, 0x3ec4fff0
++++>>>> total num hits: 62477, num free: 59909
04-Feb-2026 01:51:20 CET Closed output file "np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851_reco_stage0_20260204T005120_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 203.883 273.921 363.166 259.518 36.3759 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 8.9567e-05 0.000109461 0.000284602 9.9647e-05 3.62225e-05 27
produce:tpcrawdecoder:PDVDTPCReader 94.6587 148.593 218.295 140.019 30.5893 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.608229 0.882153 1.83957 0.81993 0.270194 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0176895 0.0366066 0.273736 0.0220697 0.0485073 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0862542 0.0973505 0.127549 0.0885239 0.0128951 27
produce:beamevent:BeamEvent 9.9436e-05 0.000179873 0.00146421 0.000126787 0.000252802 27
produce:pdvddaphne:DAPHNEReaderPDVD 13.1104 19.6406 27.1675 19.5702 3.40978 27
produce:ophit:OpHitFinder 0.0322676 0.0480889 0.0629423 0.0482666 0.00759523 27
produce:wclsdatavd:WireCellToolkit 73.8313 84.8839 92.558 86.5514 5.25027 27
produce:gaushit:GausHitFinder 0.653568 1.10919 1.90876 1.01926 0.292761 27
produce:nhitsfilter:NumberOfHitsFilter 0.000238657 0.000414163 0.00102122 0.000377337 0.000158761 27
produce:reco3d:SpacePointSolver 5.91859 10.6072 15.7687 10.2461 2.65099 27
produce:hitpdune:DisambigFromSpacePoints 0.0703292 0.131547 0.261639 0.129094 0.0467754 27
produce:cluster3d:Cluster3D 1.11694 2.57684 5.82784 2.58185 0.968766 27
[art]:TriggerResults:TriggerResultInserter 2.3414e-05 3.57006e-05 0.000118251 3.1489e-05 1.70774e-05 27
end_path:out1:RootOutput 4.038e-06 1.55383e-05 0.000258905 5.229e-06 4.798e-05 27
end_path:out1:RootOutput(write) 4.39644 5.284 6.10397 5.23559 0.42337 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4519.67 MB
Peak resident set size usage (VmHWM): 2606.85 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851_reco_stage0_20260204T005120_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851_reco_stage0_20260204T005120_offline.root
Ran successfully
{
"name": "np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851_reco_stage0_20260204T005120_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770158798.0,
"core.end_time": 1770166283.0,
"core.events": [
16216,
16222,
16228,
16234,
16240,
16246,
16252,
16258,
16264,
16270,
16276,
16282,
16288,
16294,
16300,
16306,
16312,
16318,
16324,
16330,
16336,
16342,
16348,
16354,
16360,
16366,
16372
],
"core.event_count": 27,
"core.first_event_number": 16216,
"core.last_event_number": 16372,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42422
],
"core.runs_subruns": [
4242200001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042422_0100_df-s04-d0_dw_0_20260203T160851.hdf5"
}
]
} 