Jobsub ID 303784.3@dunegpschedd01.fnal.gov
| Jobsub ID | 303784.3@dunegpschedd01.fnal.gov |
| Workflow ID | 12719 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-04 02:30:33 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 04:26:06 |
| From worker node | Hostname | hnode68.pic.es |
| cpuinfo | AMD EPYC 7402P 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-04 02:31:10 |
| Input files | vd-protodune:np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (6708s) |
| CPU time | 1h (6248s = 93%) |
| Max RSS bytes | 9561554944 (9118 MiB) |
| Outputting started | 2026-02-04 04:23:03 |
| Output files | |
| Finished | 2026-02-04 04:26:06 |
| Saved logs | justin-logs:303784.3-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
490, 0x71854620
Caught exception in looking up space point ptr... 0x123c2bc80, 0x123c135d0
Caught exception in looking up space point ptr... 0x123c135d0, 0x123c13380
Caught exception in looking up space point ptr... 0x7181e7f0, 0x7181d570
Caught exception in looking up space point ptr... 0x7181d570, 0x5cf37d40
Caught exception in looking up space point ptr... 0x1242e65c0, 0x12430b140
Caught exception in looking up space point ptr... 0x12430b140, 0xb958e180
Caught exception in looking up space point ptr... 0x10f9ee590, 0x10f9b6590
Caught exception in looking up space point ptr... 0x10f9b6590, 0x10f9ee910
Caught exception in looking up space point ptr... 0x78756bf0, 0x15a207f0
Caught exception in looking up space point ptr... 0x15a207f0, 0x15a30400
Caught exception in looking up space point ptr... 0x97dfe480, 0x787140d0
Caught exception in looking up space point ptr... 0x787140d0, 0x97e08660
Caught exception in looking up space point ptr... 0xf34fc290, 0xf34fc040
Caught exception in looking up space point ptr... 0xf34fc040, 0xf34fd070
Caught exception in looking up space point ptr... 0x43b9e770, 0x43c18cd0
Caught exception in looking up space point ptr... 0xfaf5aeb0, 0x43b9e770
Caught exception in looking up space point ptr... 0xe3889e0, 0x19356530
Caught exception in looking up space point ptr... 0x19356530, 0x193563a0
Caught exception in looking up space point ptr... 0x120181e0, 0x1200e6f0
Caught exception in looking up space point ptr... 0x1200e6f0, 0x1200e560
Caught exception in looking up space point ptr... 0x789db600, 0x789c7d80
Caught exception in looking up space point ptr... 0x789db470, 0x789db600
Caught exception in looking up space point ptr... 0x74286740, 0x33881880
Caught exception in looking up space point ptr... 0x33881880, 0x6b0bb950
Caught exception in looking up space point ptr... 0x37b3cdc0, 0x37b3cfa0
Caught exception in looking up space point ptr... 0x37b3cfa0, 0x37b3cdc0
Caught exception in looking up space point ptr... 0x61479e10, 0x61479fa0
Caught exception in looking up space point ptr... 0x614676c0, 0x61479e10
Caught exception in looking up space point ptr... 0x45f06390, 0x3d0d0590
Caught exception in looking up space point ptr... 0x3d0d0590, 0x45f06390
Caught exception in looking up space point ptr... 0x62b9f870, 0x48f12480
Caught exception in looking up space point ptr... 0x62b9fa00, 0x62b9f870
Caught exception in looking up space point ptr... 0x6aeb2b50, 0x6aeb54f0
Caught exception in looking up space point ptr... 0xa610650, 0x6aeb2b50
Caught exception in looking up space point ptr... 0x7f495cc0, 0x7f495b30
Caught exception in looking up space point ptr... 0x7f495b30, 0x6ae2a410
Caught exception in looking up space point ptr... 0xf3519c00, 0x782a50c0
Caught exception in looking up space point ptr... 0x782a50c0, 0x3963a9e0
Caught exception in looking up space point ptr... 0x786a6a30, 0x786cfd40
Caught exception in looking up space point ptr... 0x786cfd40, 0x786e7190
Caught exception in looking up space point ptr... 0x11c004f0, 0x11bee570
Caught exception in looking up space point ptr... 0x11c0d010, 0x11c004f0
Caught exception in looking up space point ptr... 0x7426faa0, 0x43c4a6f0
Caught exception in looking up space point ptr... 0xdd95e0c0, 0x7426faa0
Caught exception in looking up space point ptr... 0x5e398070, 0x5e3c3f50
Caught exception in looking up space point ptr... 0x338e5bd0, 0x5e398070
Caught exception in looking up space point ptr... 0x938d43d0, 0x938c1e10
Caught exception in looking up space point ptr... 0x938c1e10, 0x938b0340
Caught exception in looking up space point ptr... 0x398225d0, 0xc1a7a80
Caught exception in looking up space point ptr... 0xc1a7a80, 0x398225d0
Caught exception in looking up space point ptr... 0x37197010, 0x6431bba0
Caught exception in looking up space point ptr... 0x6431bba0, 0x5c7a4310
Caught exception in looking up space point ptr... 0x6745ca20, 0x65ae2060
Caught exception in looking up space point ptr... 0x65ae2060, 0x6745ca20
Caught exception in looking up space point ptr... 0x36abbdd0, 0x38968ed0
Caught exception in looking up space point ptr... 0x38968ed0, 0x36abbdd0
Caught exception in looking up space point ptr... 0x45e0d8e0, 0x1067fbe0
Caught exception in looking up space point ptr... 0x1067fbe0, 0x45e0d8e0
Caught exception in looking up space point ptr... 0x352c1600, 0x5730dd90
Caught exception in looking up space point ptr... 0x5730dd90, 0x352c1600
Caught exception in looking up space point ptr... 0x5599bb40, 0x5c990c80
Caught exception in looking up space point ptr... 0x5c990c80, 0x5c9908f0
Caught exception in looking up space point ptr... 0x9390c3a0, 0x9390bef0
Caught exception in looking up space point ptr... 0x9390bef0, 0x9390c3a0
Caught exception in looking up space point ptr... 0x33b4da00, 0x586be250
Caught exception in looking up space point ptr... 0x586be250, 0x586be400
++++>>>> total num hits: 523395, num free: 515021
04-Feb-2026 05:22:52 CET Closed output file "np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123_reco_stage0_20260204T042252_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 194.816 348 895.138 243.281 220.293 19
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.6152e-05 0.000104209 0.000302339 8.9272e-05 4.98962e-05 19
produce:tpcrawdecoder:PDVDTPCReader 20.0138 21.7349 23.4956 21.8107 0.90301 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.994333 1.0376 1.09687 1.03266 0.0198047 19
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0342502 0.0347137 0.0353292 0.0347383 0.000210394 19
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0341174 0.034711 0.0354599 0.0347111 0.00025441 19
produce:beamevent:BeamEvent 7.9333e-05 0.00020347 0.00137276 0.000147364 0.000277229 19
produce:pdvddaphne:DAPHNEReaderPDVD 2.64214 3.451 4.27417 3.51018 0.420567 19
produce:ophit:OpHitFinder 0.0269918 0.0357331 0.048916 0.0344777 0.00516348 19
produce:wclsdatavd:WireCellToolkit 101.888 124.081 144.038 126.479 11.3695 19
produce:gaushit:GausHitFinder 1.99023 4.00873 16.3728 2.52174 3.26944 19
produce:nhitsfilter:NumberOfHitsFilter 0.000448983 0.000993269 0.00255174 0.000779792 0.000579225 19
produce:reco3d:SpacePointSolver 12.4835 33.4226 102.463 19.3923 27.5719 19
produce:hitpdune:DisambigFromSpacePoints 0.220419 1.49542 5.87781 0.301847 2.00383 19
produce:cluster3d:Cluster3D 3.92918 149.653 588.74 59.8604 182.787 19
[art]:TriggerResults:TriggerResultInserter 2.1121e-05 4.01911e-05 8.5032e-05 2.7671e-05 1.96245e-05 19
end_path:out1:RootOutput 3.6e-06 6.80705e-06 1.9991e-05 4.181e-06 4.4041e-06 19
end_path:out1:RootOutput(write) 5.40806 8.91964 19.8557 6.22897 4.51423 19
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 11788.4 MB
Peak resident set size usage (VmHWM): 9561.55 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123_reco_stage0_20260204T042252_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123_reco_stage0_20260204T042252_offline.root
Ran successfully
{
"name": "np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123_reco_stage0_20260204T042252_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770172282.0,
"core.end_time": 1770178974.0,
"core.events": [
56782,
56788,
56794,
56800,
56806,
56812,
56818,
56824,
56830,
56836,
56842,
56848,
56854,
56860,
56866,
56872,
56878,
56884,
56890
],
"core.event_count": 19,
"core.first_event_number": 56782,
"core.last_event_number": 56890,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42433
],
"core.runs_subruns": [
4243300001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042433_0498_df-s04-d0_dw_0_20260203T215123.hdf5"
}
]
} 