Jobsub ID 297923.31@dunegpschedd02.fnal.gov
| Jobsub ID | 297923.31@dunegpschedd02.fnal.gov |
| Workflow ID | 12719 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 23:00:26 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce513 |
| Last heartbeat | 2026-02-04 00:49:22 |
| From worker node | Hostname | b9p35p0008.cern.ch |
| cpuinfo | AMD EPYC 9655P 96-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:57:59 |
| Input files | vd-protodune:np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 48m (2936s) |
| CPU time | 47m (2845s = 96%) |
| Max RSS bytes | 3610222592 (3442 MiB) |
| Outputting started | 2026-02-04 00:46:57 |
| Output files | |
| Finished | 2026-02-04 00:49:22 |
| Saved logs | justin-logs:297923.31-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
t ptr... 0x167ccd10, 0x167ccb80
Caught exception in looking up space point ptr... 0x54ea7b60, 0x54ea79d0
Caught exception in looking up space point ptr... 0x54ea79d0, 0x54ed8a00
Caught exception in looking up space point ptr... 0x4f023cc0, 0x4f765f60
Caught exception in looking up space point ptr... 0x4f765f60, 0x4f6ee8e0
Caught exception in looking up space point ptr... 0x124678b0, 0x12467270
Caught exception in looking up space point ptr... 0x12467270, 0xc949b70
Caught exception in looking up space point ptr... 0x55c870f0, 0x55c86c40
Caught exception in looking up space point ptr... 0x55c87410, 0x55c870f0
Caught exception in looking up space point ptr... 0x50791e20, 0x108685b0
Caught exception in looking up space point ptr... 0x50791fb0, 0x50791e20
Caught exception in looking up space point ptr... 0x3c914de0, 0x3c914660
Caught exception in looking up space point ptr... 0x3c915420, 0x3c914de0
Caught exception in looking up space point ptr... 0x62a666f0, 0x62a661b0
Caught exception in looking up space point ptr... 0x62a661b0, 0x62a65b20
Caught exception in looking up space point ptr... 0x51728640, 0x51727e60
Caught exception in looking up space point ptr... 0x51728cd0, 0x51728640
Caught exception in looking up space point ptr... 0x516df230, 0x516dea50
Caught exception in looking up space point ptr... 0x516dea50, 0x516de120
Caught exception in looking up space point ptr... 0x54652590, 0x396ad2c0
Caught exception in looking up space point ptr... 0x54658b50, 0x54652590
Caught exception in looking up space point ptr... 0x3e386860, 0x827ad830
Caught exception in looking up space point ptr... 0x3e3869f0, 0x3e386860
Caught exception in looking up space point ptr... 0x185ff2c0, 0x185cab10
Caught exception in looking up space point ptr... 0x185ff450, 0x185ff2c0
Caught exception in looking up space point ptr... 0x3e9eb850, 0x3e9c3790
Caught exception in looking up space point ptr... 0x3e9eb9e0, 0x3e9eb850
Caught exception in looking up space point ptr... 0x776f4110, 0x776dd430
Caught exception in looking up space point ptr... 0x776f42a0, 0x776f4110
Caught exception in looking up space point ptr... 0x327cb7b0, 0x327bafa0
Caught exception in looking up space point ptr... 0x327dcdd0, 0x327cb7b0
Caught exception in looking up space point ptr... 0x4b98c990, 0x3199a150
Caught exception in looking up space point ptr... 0x3199a150, 0x3241eeb0
Caught exception in looking up space point ptr... 0x4d1977e0, 0x4d184f00
Caught exception in looking up space point ptr... 0x3c29b820, 0x4d1977e0
Caught exception in looking up space point ptr... 0x32f07a50, 0x18a84e90
Caught exception in looking up space point ptr... 0x18a84e90, 0x32f07a50
Caught exception in looking up space point ptr... 0xbeb4620, 0xbea7330
Caught exception in looking up space point ptr... 0xbec1780, 0xbeb4620
Caught exception in looking up space point ptr... 0x2bcd00f0, 0x12d57ce0
Caught exception in looking up space point ptr... 0x12d57ce0, 0x68cc1550
Caught exception in looking up space point ptr... 0x3c3220a0, 0x3c30f770
Caught exception in looking up space point ptr... 0x3c30f770, 0x3c2fd020
Caught exception in looking up space point ptr... 0x5f5749d0, 0x129ab490
Caught exception in looking up space point ptr... 0x8609ab30, 0x5f5749d0
Caught exception in looking up space point ptr... 0x790dcd00, 0x31bfea90
Caught exception in looking up space point ptr... 0x31bfea90, 0x5261a870
Caught exception in looking up space point ptr... 0x32cf1f80, 0x151aa900
Caught exception in looking up space point ptr... 0x151aa900, 0x32cf1f80
Caught exception in looking up space point ptr... 0x166f7530, 0x166f76c0
Caught exception in looking up space point ptr... 0x166f76c0, 0x166f7530
Caught exception in looking up space point ptr... 0x61657020, 0x2bb47230
Caught exception in looking up space point ptr... 0x2bb47230, 0x61649e50
Caught exception in looking up space point ptr... 0x49608d70, 0x164fea30
Caught exception in looking up space point ptr... 0x164fea30, 0x49608d70
Caught exception in looking up space point ptr... 0x4839c320, 0xffcd380
Caught exception in looking up space point ptr... 0xffcd380, 0x5006a410
Caught exception in looking up space point ptr... 0x60e78120, 0x424e95a0
Caught exception in looking up space point ptr... 0x424e95a0, 0x60e78120
Caught exception in looking up space point ptr... 0x52de24d0, 0x52de11b0
Caught exception in looking up space point ptr... 0x52de11b0, 0x52de24d0
Caught exception in looking up space point ptr... 0xcd83570, 0x61666680
Caught exception in looking up space point ptr... 0x61666680, 0xcd83570
Caught exception in looking up space point ptr... 0x52e13700, 0x54459560
Caught exception in looking up space point ptr... 0x544577b0, 0x52e13700
Caught exception in looking up space point ptr... 0x32ab49a0, 0x152431b0
Caught exception in looking up space point ptr... 0x152431b0, 0x32ab49a0
++++>>>> total num hits: 577995, num free: 567696
04-Feb-2026 01:46:50 CET Closed output file "np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138_reco_stage0_20260204T004650_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 97.9704 152.05 186.62 157.396 20.0953 19
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.3111e-05 7.69621e-05 0.00016012 7.2061e-05 2.35735e-05 19
produce:tpcrawdecoder:PDVDTPCReader 6.07081 6.76616 8.14805 6.70608 0.455727 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.00722702 0.0238481 0.0845231 0.0128464 0.022139 19
produce:timingrawdecoder:PDHDTimingRawDecoder 0.000319461 0.000504971 0.00100931 0.000476383 0.000159428 19
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.000261881 0.000405309 0.000729543 0.000365932 0.000122105 19
produce:beamevent:BeamEvent 5.654e-05 0.000112189 0.000811204 7.513e-05 0.000165064 19
produce:pdvddaphne:DAPHNEReaderPDVD 1.76259 2.11849 2.52288 2.16594 0.184216 19
produce:ophit:OpHitFinder 0.025682 0.0316333 0.037487 0.0312549 0.00307833 19
produce:wclsdatavd:WireCellToolkit 73.6978 82.0487 94.3077 81.805 5.62476 19
produce:gaushit:GausHitFinder 0.705611 1.70963 2.47496 1.70842 0.433516 19
produce:nhitsfilter:NumberOfHitsFilter 0.000175361 0.000234972 0.000443013 0.000222011 5.84608e-05 19
produce:reco3d:SpacePointSolver 7.66407 10.8353 21.4969 10.3811 2.94188 19
produce:hitpdune:DisambigFromSpacePoints 0.101852 0.243783 0.417746 0.215822 0.090769 19
produce:cluster3d:Cluster3D 1.12191 42.4995 59.6323 48.6763 17.8601 19
[art]:TriggerResults:TriggerResultInserter 2.46e-05 3.62107e-05 9.0981e-05 3.23e-05 1.40671e-05 19
end_path:out1:RootOutput 4.87e-06 6.52263e-06 2.306e-05 5.63e-06 3.92367e-06 19
end_path:out1:RootOutput(write) 4.01854 5.73782 7.47012 5.65403 0.737649 19
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5677.45 MB
Peak resident set size usage (VmHWM): 3610.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138_reco_stage0_20260204T004650_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138_reco_stage0_20260204T004650_offline.root
Ran successfully
{
"name": "np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138_reco_stage0_20260204T004650_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770163085.0,
"core.end_time": 1770166011.0,
"core.events": [
59858,
59864,
59870,
59876,
59882,
59888,
59894,
59900,
59906,
59912,
59918,
59924,
59930,
59936,
59942,
59948,
59954,
59960,
59966
],
"core.event_count": 19,
"core.first_event_number": 59858,
"core.last_event_number": 59966,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42433
],
"core.runs_subruns": [
4243300001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042433_0525_df-s03-d0_dw_0_20260203T220138.hdf5"
}
]
}