Jobsub ID 303647.174@dunegpschedd01.fnal.gov
| Jobsub ID | 303647.174@dunegpschedd01.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:46:24 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce512 |
| Last heartbeat | 2026-02-03 23:31:43 |
| From worker node | Hostname | b9g24p4713.cern.ch |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 22:46:57 |
| Input files | vd-protodune:np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 42m (2545s) |
| CPU time | 40m (2435s = 95%) |
| Max RSS bytes | 2695221248 (2570 MiB) |
| Outputting started | 2026-02-03 23:29:23 |
| Output files | |
| Finished | 2026-02-03 23:31:43 |
| Saved logs | justin-logs:303647.174-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
28f330
Caught exception in looking up space point ptr... 0xe501a00, 0xe529890
Caught exception in looking up space point ptr... 0x2893a2e0, 0x4a729570
Caught exception in looking up space point ptr... 0x4a729570, 0x64f169c0
Caught exception in looking up space point ptr... 0x28b09510, 0x1475f2f0
Caught exception in looking up space point ptr... 0x1475f2f0, 0x28b09510
Caught exception in looking up space point ptr... 0x35f4d0b0, 0x2b94d720
Caught exception in looking up space point ptr... 0x2b94d720, 0x35f4d0b0
Caught exception in looking up space point ptr... 0x57acc4a0, 0x57acc350
Caught exception in looking up space point ptr... 0x57acc350, 0x57acc4a0
Caught exception in looking up space point ptr... 0x34ece120, 0x29586bb0
Caught exception in looking up space point ptr... 0x29586bb0, 0x64eecc30
Caught exception in looking up space point ptr... 0x3cc511e0, 0x46f28010
Caught exception in looking up space point ptr... 0x46f28010, 0x46f281a0
Caught exception in looking up space point ptr... 0x1006cc60, 0x1006cdf0
Caught exception in looking up space point ptr... 0x48f33430, 0x1006cc60
Caught exception in looking up space point ptr... 0x4d0f5200, 0x14d962e0
Caught exception in looking up space point ptr... 0x14d962e0, 0x1503ec80
Caught exception in looking up space point ptr... 0x28debd40, 0x5aebdf50
Caught exception in looking up space point ptr... 0x5aebdf50, 0xc5d0c20
Caught exception in looking up space point ptr... 0xc5d0c20, 0x4d555170
Caught exception in looking up space point ptr... 0x2ac768a0, 0x365c2d90
Caught exception in looking up space point ptr... 0x365c2d90, 0x36695f10
Caught exception in looking up space point ptr... 0xa3a05a0, 0xef309b0
Caught exception in looking up space point ptr... 0xef309b0, 0xef31d40
Caught exception in looking up space point ptr... 0x2bfaf950, 0x59b1e500
Caught exception in looking up space point ptr... 0x59b1e500, 0x4a392800
Caught exception in looking up space point ptr... 0x59655ba0, 0x306a1880
Caught exception in looking up space point ptr... 0x306a1880, 0x306a1ad0
Caught exception in looking up space point ptr... 0x10081540, 0x10081220
Caught exception in looking up space point ptr... 0x10081220, 0x10081540
Caught exception in looking up space point ptr... 0x128b6400, 0x3c24c680
Caught exception in looking up space point ptr... 0x3c24c680, 0x66238610
Caught exception in looking up space point ptr... 0x2bb38030, 0x2bb37d10
Caught exception in looking up space point ptr... 0x2bb37d10, 0x3bfd8cf0
Caught exception in looking up space point ptr... 0x49beb300, 0x291302a0
Caught exception in looking up space point ptr... 0x472291f0, 0x49beb300
Caught exception in looking up space point ptr... 0x4d64c170, 0x15883aa0
Caught exception in looking up space point ptr... 0x15883aa0, 0x52a2a320
Caught exception in looking up space point ptr... 0x52a2a320, 0x14762920
Caught exception in looking up space point ptr... 0x5a069420, 0x5a068f70
Caught exception in looking up space point ptr... 0x5a068f70, 0x5a069420
Caught exception in looking up space point ptr... 0x47e76c50, 0x32497a50
Caught exception in looking up space point ptr... 0x32497a50, 0x47e76c50
Caught exception in looking up space point ptr... 0x335bbac0, 0x3093e300
Caught exception in looking up space point ptr... 0x3093e300, 0xc5c15a0
Caught exception in looking up space point ptr... 0x5e8f5bc0, 0x28914750
Caught exception in looking up space point ptr... 0x28914750, 0x28912440
Caught exception in looking up space point ptr... 0x3663f940, 0x14728f80
Caught exception in looking up space point ptr... 0x14728f80, 0x266231c0
Caught exception in looking up space point ptr... 0x3c18f270, 0x2d1bdd70
Caught exception in looking up space point ptr... 0x2d1bdd70, 0x24d348a0
Caught exception in looking up space point ptr... 0x3cbe3c10, 0x3cbe3a10
Caught exception in looking up space point ptr... 0x3cbe3a10, 0x3cbe3c10
Caught exception in looking up space point ptr... 0x5be041e0, 0x2195e1e0
Caught exception in looking up space point ptr... 0x2195e1e0, 0x5d4fc310
Caught exception in looking up space point ptr... 0x5d4fc310, 0x292016d0
Caught exception in looking up space point ptr... 0x292016d0, 0x46f5b240
Caught exception in looking up space point ptr... 0x46f5b240, 0x15891b40
Caught exception in looking up space point ptr... 0x15891b40, 0x5be041e0
Caught exception in looking up space point ptr... 0x87e15a0, 0x87e1450
Caught exception in looking up space point ptr... 0x87e1450, 0x87e15a0
Caught exception in looking up space point ptr... 0x4ee6e9c0, 0x4ee6eb50
Caught exception in looking up space point ptr... 0x4ee6eb50, 0x4ee6e6a0
Caught exception in looking up space point ptr... 0x4ee6e380, 0x4ee6e510
Caught exception in looking up space point ptr... 0x4ee6e510, 0x4ee6e380
++++>>>> total num hits: 43601, num free: 41775
04-Feb-2026 00:29:17 CET Closed output file "np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838_reco_stage0_20260203T232917_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 74.3224 91.7564 108.476 93.0359 9.45517 27
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.3441e-05 0.000104681 0.000166883 9.5352e-05 3.73908e-05 27
produce:tpcrawdecoder:PDVDTPCReader 4.01161 4.97284 10.854 4.48645 1.64684 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.00871626 0.0192974 0.0609076 0.0124029 0.0144767 27
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000341847 0.000495957 0.000818326 0.000445649 0.000123927 27
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.00125565 0.00158953 0.00225242 0.00155458 0.00022111 27
produce:beamevent:BeamEvent 6.0952e-05 0.000138792 0.00110303 9.1822e-05 0.000191952 27
produce:pdvddaphne:DAPHNEReaderPDVD 2.21013 2.81051 3.3135 2.85665 0.284502 27
produce:ophit:OpHitFinder 0.0234005 0.0350619 0.0455351 0.0347208 0.00655007 27
produce:wclsdatavd:WireCellToolkit 56.3506 65.9912 73.444 67.0902 4.68066 27
produce:gaushit:GausHitFinder 0.473988 0.883444 1.49095 0.815842 0.224865 27
produce:nhitsfilter:NumberOfHitsFilter 0.000180403 0.000317815 0.000733104 0.000294806 0.000119193 27
produce:reco3d:SpacePointSolver 5.41768 11.0377 20.726 9.11549 4.15986 27
produce:hitpdune:DisambigFromSpacePoints 0.063459 0.108729 0.242392 0.0992223 0.0395489 27
produce:cluster3d:Cluster3D 0.851108 2.00973 4.28142 1.96044 0.898898 27
[art]:TriggerResults:TriggerResultInserter 2.379e-05 3.3492e-05 0.000143153 2.831e-05 2.21239e-05 27
end_path:out1:RootOutput 3.74e-06 8.52085e-06 3.79e-05 8.081e-06 6.17652e-06 27
end_path:out1:RootOutput(write) 3.19379 3.86505 4.8663 3.80281 0.506796 27
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4569.01 MB
Peak resident set size usage (VmHWM): 2695.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838_reco_stage0_20260203T232917_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838_reco_stage0_20260203T232917_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838_reco_stage0_20260203T232917_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770158828.0,
"core.end_time": 1770161358.0,
"core.events": [
14321,
14327,
14333,
14339,
14345,
14351,
14357,
14363,
14369,
14375,
14381,
14387,
14393,
14399,
14405,
14411,
14417,
14423,
14429,
14435,
14441,
14447,
14453,
14459,
14465,
14471,
14477
],
"core.event_count": 27,
"core.first_event_number": 14321,
"core.last_event_number": 14477,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0088_df-s04-d1_dw_0_20260203T141838.hdf5"
}
]
}