Jobsub ID 303653.12@dunegpschedd01.fnal.gov
| Jobsub ID | 303653.12@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:52:24 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce506 |
| Last heartbeat | 2026-02-04 00:16:19 |
| From worker node | Hostname | b9p06p2822.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:36:16 |
| Input files | vd-protodune:np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 37m (2242s) |
| CPU time | 35m (2153s = 96%) |
| Max RSS bytes | 2626048000 (2504 MiB) |
| Outputting started | 2026-02-04 00:13:40 |
| Output files | |
| Finished | 2026-02-04 00:16:09 |
| Saved logs | justin-logs:303653.12-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
ht exception in looking up space point ptr... 0x3bd5f150, 0x35e82340
Caught exception in looking up space point ptr... 0x3d8fa280, 0x3d8f9990
Caught exception in looking up space point ptr... 0x3d8f9990, 0x3d8fa280
Caught exception in looking up space point ptr... 0x3d8fa280, 0x3d8f9b20
Caught exception in looking up space point ptr... 0x3d8f9b20, 0x3d8fa280
Caught exception in looking up space point ptr... 0x52ea34d0, 0x52ea1070
Caught exception in looking up space point ptr... 0x52ea1070, 0x52ea34d0
Caught exception in looking up space point ptr... 0x52ea34d0, 0x52ea33b0
Caught exception in looking up space point ptr... 0x52ea33b0, 0x52ea34d0
Caught exception in looking up space point ptr... 0x3acbb7d0, 0x24e9f660
Caught exception in looking up space point ptr... 0x24e9f660, 0x3acbb7d0
Caught exception in looking up space point ptr... 0x310fc690, 0x310fc3f0
Caught exception in looking up space point ptr... 0x310fc3f0, 0x310fc690
Caught exception in looking up space point ptr... 0x5566e8a0, 0x31102c40
Caught exception in looking up space point ptr... 0x31102c40, 0x55688c70
Caught exception in looking up space point ptr... 0x4370d920, 0x541ada50
Caught exception in looking up space point ptr... 0x541ada50, 0x4370da20
Caught exception in looking up space point ptr... 0x42206080, 0x42208440
Caught exception in looking up space point ptr... 0x42208440, 0x42208bd0
Caught exception in looking up space point ptr... 0x23a7df30, 0x23a80940
Caught exception in looking up space point ptr... 0x23a80940, 0x23a7df30
Caught exception in looking up space point ptr... 0x125409c0, 0x264eab20
Caught exception in looking up space point ptr... 0x264eab20, 0x264ea8d0
Caught exception in looking up space point ptr... 0x36866a30, 0x368668e0
Caught exception in looking up space point ptr... 0x368668e0, 0x36866a30
Caught exception in looking up space point ptr... 0x2f9da290, 0x2f9d9f70
Caught exception in looking up space point ptr... 0x2f9d9f70, 0x336fcac0
Caught exception in looking up space point ptr... 0x55c90060, 0x375b85c0
Caught exception in looking up space point ptr... 0x375b85c0, 0x375bab60
Caught exception in looking up space point ptr... 0x4f797f30, 0x4f796e80
Caught exception in looking up space point ptr... 0x4f796e80, 0x4f797f30
Caught exception in looking up space point ptr... 0x473f29c0, 0x53153a10
Caught exception in looking up space point ptr... 0x53153a10, 0x473f29c0
Caught exception in looking up space point ptr... 0x5ef241e0, 0x5ef23ec0
Caught exception in looking up space point ptr... 0x5ef23ec0, 0x13d4a4f0
Caught exception in looking up space point ptr... 0x169a0380, 0x1699f500
Caught exception in looking up space point ptr... 0x1699f500, 0x169a6010
Caught exception in looking up space point ptr... 0x3cbe30e0, 0x4a9a5400
Caught exception in looking up space point ptr... 0x4a9a5400, 0x3cbe30e0
Caught exception in looking up space point ptr... 0x13d38ad0, 0x13d385d0
Caught exception in looking up space point ptr... 0x13d385d0, 0x13d38ad0
Caught exception in looking up space point ptr... 0x3cbdb890, 0x3cbe2d90
Caught exception in looking up space point ptr... 0x3cbe2d90, 0x3cbd7c60
Caught exception in looking up space point ptr... 0x2e4e6c50, 0x9fd7f60
Caught exception in looking up space point ptr... 0x9fd7f60, 0x2e4e6c50
Caught exception in looking up space point ptr... 0x13d378b0, 0x13d37470
Caught exception in looking up space point ptr... 0x13d37470, 0x13d37610
Caught exception in looking up space point ptr... 0x31106d30, 0x31100500
Caught exception in looking up space point ptr... 0x31100500, 0x31106d30
Caught exception in looking up space point ptr... 0x4a999410, 0x4a9a7c90
Caught exception in looking up space point ptr... 0x4a9a7c90, 0x4a999410
Caught exception in looking up space point ptr... 0x4a9a8030, 0x4a9b5b70
Caught exception in looking up space point ptr... 0x4a9b5b70, 0x4a9a8030
Caught exception in looking up space point ptr... 0x4a9a6880, 0x3cbebf00
Caught exception in looking up space point ptr... 0x3cbebf00, 0x4a9b82e0
Caught exception in looking up space point ptr... 0x4a99a650, 0x3cbe5890
Caught exception in looking up space point ptr... 0x3cbe5890, 0x3cbef570
Caught exception in looking up space point ptr... 0x1699fda0, 0x1699a560
Caught exception in looking up space point ptr... 0x1699a560, 0x1699fda0
Caught exception in looking up space point ptr... 0x4a99a7d0, 0x3cbecbd0
Caught exception in looking up space point ptr... 0x3cbecbd0, 0x4a9a13c0
Caught exception in looking up space point ptr... 0x4d8cfb70, 0x4d8ed6c0
Caught exception in looking up space point ptr... 0x4d8ed6c0, 0x4d8cfb70
Caught exception in looking up space point ptr... 0x30719460, 0x1685d000
Caught exception in looking up space point ptr... 0x1685d000, 0x30719460
++++>>>> total num hits: 95770, num free: 91365
04-Feb-2026 01:13:30 CET Closed output file "np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507_reco_stage0_20260204T001330_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 71.1863 80.8564 94.4563 81.9004 6.89988 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 4.277e-05 8.29729e-05 0.000256301 7.091e-05 4.48016e-05 27
produce:tpcrawdecoder:PDVDTPCReader 3.72124 4.39804 5.03244 4.36635 0.303824 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.00909994 0.0198502 0.053985 0.0113285 0.0134319 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000367961 0.00047448 0.000773754 0.000455693 8.80264e-05 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0012933 0.00166175 0.00213263 0.00167324 0.000187014 27
produce:beamevent:BeamEvent 4.621e-05 0.000102426 0.000960204 6.741e-05 0.000169011 27
produce:pdvddaphne:DAPHNEReaderPDVD 2.10428 2.73529 3.47197 2.74777 0.354574 27
produce:ophit:OpHitFinder 0.0242653 0.0335081 0.0411768 0.0341569 0.00522233 27
produce:wclsdatavd:WireCellToolkit 49.455 56.8032 69.8333 56.4294 5.63282 27
produce:gaushit:GausHitFinder 0.541688 0.874425 1.62952 0.815913 0.246983 27
produce:nhitsfilter:NumberOfHitsFilter 0.000183581 0.000246259 0.000375892 0.000232791 5.0309e-05 27
produce:reco3d:SpacePointSolver 6.75995 10.7652 15.4244 11.0998 2.33911 27
produce:hitpdune:DisambigFromSpacePoints 0.0568549 0.0875632 0.160359 0.0786529 0.0268459 27
produce:cluster3d:Cluster3D 0.821995 1.75498 7.49316 1.39236 1.26554 27
[art]:TriggerResults:TriggerResultInserter 1.564e-05 2.80357e-05 9.158e-05 2.336e-05 1.49977e-05 27
end_path:out1:RootOutput 3.16e-06 6.21896e-06 3.017e-05 5.26e-06 4.93116e-06 27
end_path:out1:RootOutput(write) 2.84896 3.36487 4.94883 3.14305 0.537855 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4631.29 MB
Peak resident set size usage (VmHWM): 2626.05 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507_reco_stage0_20260204T001330_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507_reco_stage0_20260204T001330_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507_reco_stage0_20260204T001330_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770161785.0,
"core.end_time": 1770164012.0,
"core.events": [
22268,
22274,
22280,
22286,
22292,
22298,
22304,
22310,
22316,
22322,
22328,
22334,
22340,
22346,
22352,
22358,
22364,
22370,
22376,
22382,
22388,
22394,
22400,
22406,
22412,
22418,
22424
],
"core.event_count": 27,
"core.first_event_number": 22268,
"core.last_event_number": 22424,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0137_df-s03-d0_dw_0_20260203T144507.hdf5"
}
]
}