Jobsub ID 298060.0@dunegpschedd02.fnal.gov
| Jobsub ID | 298060.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12717 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-04 02:54:34 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce16-multicore |
| Last heartbeat | 2026-02-04 03:47:12 |
| From worker node | Hostname | td108.pic.es |
| cpuinfo | AMD EPYC 9825 144-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-04 02:55:30 |
| Input files | vd-protodune:np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 50m (3007s) |
| CPU time | 41m (2511s = 83%) |
| Max RSS bytes | 7150940160 (6819 MiB) |
| Outputting started | 2026-02-04 03:45:39 |
| Output files | |
| Finished | 2026-02-04 03:47:12 |
| Saved logs | justin-logs:298060.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
r... 0x1540d210, 0x15420de0
Caught exception in looking up space point ptr... 0x15420de0, 0x397ff790
Caught exception in looking up space point ptr... 0xae088800, 0xae088560
Caught exception in looking up space point ptr... 0xae088560, 0xae088410
Caught exception in looking up space point ptr... 0xae088410, 0xbe104850
Caught exception in looking up space point ptr... 0xae088bf0, 0xae088800
Caught exception in looking up space point ptr... 0xe2645dd0, 0xb85be40
Caught exception in looking up space point ptr... 0xb85be40, 0xe2645dd0
Caught exception in looking up space point ptr... 0x177a8e40, 0x177ac2f0
Caught exception in looking up space point ptr... 0x177ac2f0, 0x332d8ff0
Caught exception in looking up space point ptr... 0xeb96110, 0xb399e570
Caught exception in looking up space point ptr... 0xb399e8c0, 0xeb963f0
Caught exception in looking up space point ptr... 0xeb963f0, 0xeb96110
Caught exception in looking up space point ptr... 0xbdbf7090, 0xbdbf72e0
Caught exception in looking up space point ptr... 0xbdbf72e0, 0x2d8ae790
Caught exception in looking up space point ptr... 0x638aca10, 0x638af5d0
Caught exception in looking up space point ptr... 0x638b73c0, 0x638aca10
Caught exception in looking up space point ptr... 0x9e6c0b00, 0x9e6c0d50
Caught exception in looking up space point ptr... 0x9e6c0d50, 0x9e6c0fa0
Caught exception in looking up space point ptr... 0x9e6c0fa0, 0x2cafb1f0
Caught exception in looking up space point ptr... 0x2cafb1f0, 0x9e6c0b00
Caught exception in looking up space point ptr... 0xb39aad90, 0xb39a6010
Caught exception in looking up space point ptr... 0xb39a6010, 0x2ca11fe0
Caught exception in looking up space point ptr... 0x500cc010, 0x157e9e10
Caught exception in looking up space point ptr... 0x157e9e10, 0x500cc010
Caught exception in looking up space point ptr... 0x271ac600, 0x271af180
Caught exception in looking up space point ptr... 0x271af180, 0x271a7350
Caught exception in looking up space point ptr... 0x262d4160, 0x262d3f10
Caught exception in looking up space point ptr... 0x262d3f10, 0x262d3a70
Caught exception in looking up space point ptr... 0x5eff5270, 0x262d4160
Caught exception in looking up space point ptr... 0xca40de30, 0x2c9fe770
Caught exception in looking up space point ptr... 0x2c9fe770, 0xca417830
Caught exception in looking up space point ptr... 0xb1547190, 0xb1541480
Caught exception in looking up space point ptr... 0xb1541480, 0xb1547190
Caught exception in looking up space point ptr... 0x5dd50a80, 0x5dd4e810
Caught exception in looking up space point ptr... 0x5dd4e810, 0x5dd50a80
Caught exception in looking up space point ptr... 0x63639a30, 0x41dfd9a0
Caught exception in looking up space point ptr... 0x41dfd9a0, 0x63639a30
Caught exception in looking up space point ptr... 0x3a3f9490, 0x3a3f1a50
Caught exception in looking up space point ptr... 0x3a3f1a50, 0x3a3f0f70
Caught exception in looking up space point ptr... 0x3a3f0f70, 0x3a3f9290
Caught exception in looking up space point ptr... 0x3a3f9290, 0x3a3f9490
Caught exception in looking up space point ptr... 0x17b1d610, 0x2a9de580
Caught exception in looking up space point ptr... 0x2a9de580, 0x2a9de9e0
Caught exception in looking up space point ptr... 0x424a1640, 0xf4bf2af0
Caught exception in looking up space point ptr... 0xf4bf2af0, 0xca419b70
Caught exception in looking up space point ptr... 0x2d535f50, 0x2d535b60
Caught exception in looking up space point ptr... 0x2d535b60, 0x2d535f50
Caught exception in looking up space point ptr... 0xb746f20, 0x4df844b0
Caught exception in looking up space point ptr... 0x4df844b0, 0xb746f20
Caught exception in looking up space point ptr... 0xb746f20, 0xb74b800
Caught exception in looking up space point ptr... 0xb74b800, 0xb746f20
Caught exception in looking up space point ptr... 0x56493480, 0x56490310
Caught exception in looking up space point ptr... 0x56490310, 0x564901e0
Caught exception in looking up space point ptr... 0xca55e790, 0xca55e8c0
Caught exception in looking up space point ptr... 0xca55e8c0, 0xca55eb20
Caught exception in looking up space point ptr... 0x15c72450, 0x15c72620
Caught exception in looking up space point ptr... 0x15c72620, 0x15c72450
Caught exception in looking up space point ptr... 0xef4df10, 0xd9ec0700
Caught exception in looking up space point ptr... 0xd9ec0700, 0x8808cf40
Caught exception in looking up space point ptr... 0x1881ccd0, 0x1881ce60
Caught exception in looking up space point ptr... 0x1881ce60, 0x1881cb40
Caught exception in looking up space point ptr... 0xe3ef3cd0, 0xe410c960
Caught exception in looking up space point ptr... 0xe410c960, 0xe410caf0
Caught exception in looking up space point ptr... 0x5d26e3f0, 0x5d26e5d0
Caught exception in looking up space point ptr... 0x5d26e5d0, 0x5d26f750
++++>>>> total num hits: 108313, num free: 104351
04-Feb-2026 04:45:31 CET Closed output file "np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407_reco_stage0_20260204T034531_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 72.1564 105.166 501.848 78.026 95.2249 28
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.853e-05 9.80572e-05 0.000279731 9.04605e-05 3.83563e-05 28
produce:tpcrawdecoder:PDVDTPCReader 15.5618 16.6701 22.3746 16.3643 1.23761 28
produce:triggerrawdecoder:PDVDTriggerReader4 0.882621 0.947796 1.6485 0.921199 0.138438 28
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0295277 0.0344849 0.137905 0.0298052 0.0203568 28
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.146103 0.149464 0.176949 0.147396 0.0076146 28
produce:beamevent:BeamEvent 0.0001055 0.000186586 0.00110438 0.000147256 0.000180382 28
produce:pdvddaphne:DAPHNEReaderPDVD 2.21417 2.40863 2.6115 2.41504 0.10028 28
produce:ophit:OpHitFinder 0.0207476 0.0308946 0.0376756 0.0313167 0.00439677 28
produce:wclsdatavd:WireCellToolkit 43.4279 47.6414 56.7419 47.1201 3.19001 28
produce:gaushit:GausHitFinder 0.42232 1.45081 6.40704 0.620944 1.82968 28
produce:nhitsfilter:NumberOfHitsFilter 0.00014219 0.000317659 0.000937164 0.000199455 0.000238199 28
produce:reco3d:SpacePointSolver 3.77423 10.0385 49.9507 7.20041 10.1167 25
produce:hitpdune:DisambigFromSpacePoints 0.0386532 0.284197 2.97726 0.0683088 0.731839 25
produce:cluster3d:Cluster3D 0.509505 25.9791 365.209 1.14636 85.4958 25
[art]:TriggerResults:TriggerResultInserter 2.407e-05 4.28609e-05 0.00011151 3.94205e-05 1.66203e-05 28
end_path:out1:RootOutput 5.07e-06 8.45821e-06 3.079e-05 6.66e-06 5.03812e-06 28
end_path:out1:RootOutput(write) 2.55533 3.37907 10.3242 2.82046 1.65194 28
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 9533.88 MB
Peak resident set size usage (VmHWM): 7150.94 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407_reco_stage0_20260204T034531_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407_reco_stage0_20260204T034531_offline.root
Ran successfully
{
"name": "np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407_reco_stage0_20260204T034531_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770173737.0,
"core.end_time": 1770176733.0,
"core.events": [
973,
979,
985,
991,
997,
1003,
1009,
1015,
1021,
1027,
1033,
1039,
1045,
1051,
1057,
1063,
1069,
1075,
1081,
1087,
1093,
1099,
1105,
1111,
1117,
1123,
1129,
1135
],
"core.event_count": 28,
"core.first_event_number": 973,
"core.last_event_number": 1135,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42421
],
"core.runs_subruns": [
4242100001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042421_0006_df-s02-d1_dw_0_20260203T133407.hdf5"
}
]
} 