Jobsub ID 295859.0@dunegpschedd02.fnal.gov
| Jobsub ID | 295859.0@dunegpschedd02.fnal.gov |
| Workflow ID | 12428 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2026-01-27 08:49:10 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2026-01-27 09:57:41 |
| From worker node | Hostname | ccwcondor0004 |
| cpuinfo | AMD EPYC 9334 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-27 08:49:56 |
| Input files | vd-protodune:np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (3687s) |
| CPU time | 58m (3480s = 94%) |
| Max RSS bytes | 2860306432 (2727 MiB) |
| Outputting started | 2026-01-27 09:51:25 |
| Output files | |
| Finished | 2026-01-27 09:57:41 |
| Saved logs | justin-logs:295859.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
up space point ptr... 0x92695e0, 0x9268eb0
Caught exception in looking up space point ptr... 0x9269270, 0x92695e0
Caught exception in looking up space point ptr... 0x10d96190, 0x10d9bbe0
Caught exception in looking up space point ptr... 0x10d9b520, 0x10d96190
Caught exception in looking up space point ptr... 0x120d5850, 0x120d5a50
Caught exception in looking up space point ptr... 0x120d5a50, 0x120d5850
Caught exception in looking up space point ptr... 0x8fb0f70, 0x2b11e230
Caught exception in looking up space point ptr... 0x2b11e230, 0x8fb0f70
Caught exception in looking up space point ptr... 0x1b2f2a30, 0x1e4b6190
Caught exception in looking up space point ptr... 0x1e4b6190, 0x1c34e330
Caught exception in looking up space point ptr... 0x374ccf10, 0x2fb1bdf0
Caught exception in looking up space point ptr... 0x2fb1c040, 0x374ccf10
Caught exception in looking up space point ptr... 0xd040ac0, 0x26335ec0
Caught exception in looking up space point ptr... 0x26335ec0, 0xd040ac0
Caught exception in looking up space point ptr... 0xad0ab20, 0x242064f0
Caught exception in looking up space point ptr... 0x242064f0, 0xad0a740
Caught exception in looking up space point ptr... 0xda0dbf0, 0x282d5540
Caught exception in looking up space point ptr... 0x282d5540, 0xda0dbf0
Caught exception in looking up space point ptr... 0x9c38e00, 0x9c3b060
Caught exception in looking up space point ptr... 0x9c3b060, 0x9c3af60
Caught exception in looking up space point ptr... 0x9c3af60, 0x9c388b0
Caught exception in looking up space point ptr... 0x12018e10, 0x204da040
Caught exception in looking up space point ptr... 0x204da040, 0x1f92cae0
Caught exception in looking up space point ptr... 0x259ff450, 0x25bc18d0
Caught exception in looking up space point ptr... 0x25bc18d0, 0x259ff450
Caught exception in looking up space point ptr... 0x25b09560, 0x25671640
Caught exception in looking up space point ptr... 0x128d40d0, 0x25b09560
Caught exception in looking up space point ptr... 0x1c34f220, 0x256f2ed0
Caught exception in looking up space point ptr... 0x256f2ed0, 0x1f602f20
Caught exception in looking up space point ptr... 0x1c65c160, 0x8010ec0
Caught exception in looking up space point ptr... 0x8010ec0, 0x2cdbc4f0
Caught exception in looking up space point ptr... 0x2b8787a0, 0x122116a0
Caught exception in looking up space point ptr... 0x122116a0, 0x12210030
Caught exception in looking up space point ptr... 0x12210030, 0x1220ee30
Caught exception in looking up space point ptr... 0x10f63420, 0x1b1180b0
Caught exception in looking up space point ptr... 0x1b1180b0, 0x10f63420
Caught exception in looking up space point ptr... 0x80bd750, 0xe1b17c0
Caught exception in looking up space point ptr... 0xe1b17c0, 0x80bd750
Caught exception in looking up space point ptr... 0x945ba10, 0x1bd9a710
Caught exception in looking up space point ptr... 0x1bd9a710, 0x2e875c90
Caught exception in looking up space point ptr... 0xe683aa0, 0xe683ff0
Caught exception in looking up space point ptr... 0xe683ff0, 0xe684180
Caught exception in looking up space point ptr... 0x36714380, 0x2de92db0
Caught exception in looking up space point ptr... 0x2de92db0, 0x36714380
Caught exception in looking up space point ptr... 0x25598eb0, 0x2b888210
Caught exception in looking up space point ptr... 0x2b888210, 0x25598eb0
Caught exception in looking up space point ptr... 0x220e66a0, 0x13e2e000
Caught exception in looking up space point ptr... 0x13e2e000, 0x220e66a0
Caught exception in looking up space point ptr... 0x117b6e30, 0xa41b6e0
Caught exception in looking up space point ptr... 0xa41b6e0, 0x117b6e30
Caught exception in looking up space point ptr... 0x292260b0, 0x1f119ae0
Caught exception in looking up space point ptr... 0x1f119ae0, 0x292260b0
Caught exception in looking up space point ptr... 0xdbd7770, 0x3134c640
Caught exception in looking up space point ptr... 0x3134c640, 0xdbd7770
Caught exception in looking up space point ptr... 0x2de82e30, 0x29255a80
Caught exception in looking up space point ptr... 0x29255a80, 0x20c34ef0
Caught exception in looking up space point ptr... 0x1ddf6210, 0x1ddf60c0
Caught exception in looking up space point ptr... 0x1ddf60c0, 0x1ddf6210
Caught exception in looking up space point ptr... 0x21cbc870, 0x21cbc6e0
Caught exception in looking up space point ptr... 0x21cbc6e0, 0x21cbc550
Caught exception in looking up space point ptr... 0x13e204b0, 0x13e202d0
Caught exception in looking up space point ptr... 0x13e202d0, 0x13e204b0
++++>>>> total num hits: 71602, num free: 68926
27-Jan-2026 10:51:08 CET Closed output file "np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622_reco_stage0_20260127T095108_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 52.6251 74.1821 254.264 62.3313 34.7625 48
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 8.9254e-05 0.000151579 0.000463324 0.000114076 8.28777e-05 48
produce:tpcrawdecoder:PDVDTPCReader 2.7468 3.33235 5.87059 3.20768 0.523982 48
produce:triggerrawdecoder:PDVDTriggerReader4 0.0617065 0.074106 0.10176 0.0705973 0.00991725 48
produce:timingrawdecoder:PDHDTimingRawDecoder 0.00375062 0.00461894 0.0115989 0.00419089 0.00128682 48
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0179041 0.020284 0.0318278 0.0192448 0.00267226 48
produce:beamevent:BeamEvent 0.000109003 0.000227755 0.00143641 0.000177416 0.00019176 48
produce:pdvddaphne:DAPHNEReaderPDVD 0.000263214 0.000369745 0.00099541 0.000345598 0.000109744 48
produce:ophit:OpHitFinder 7.5322e-05 0.000198545 0.00154682 0.000137972 0.000235781 48
produce:wclsdatavd:WireCellToolkit 39.7267 45.662 50.3257 45.7715 2.14363 48
produce:gaushit:GausHitFinder 0.332431 0.686189 2.43144 0.549216 0.392062 48
produce:nhitsfilter:NumberOfHitsFilter 0.000108953 0.000416487 0.0109678 0.000166008 0.00154118 48
produce:reco3d:SpacePointSolver 2.10282 7.28638 47.3691 4.66475 8.38775 48
produce:hitpdune:DisambigFromSpacePoints 0.0253913 0.157548 1.53245 0.0766677 0.266501 48
produce:cluster3d:Cluster3D 0.63162 14.108 186.806 4.84017 29.1446 48
[art]:TriggerResults:TriggerResultInserter 3.2398e-05 4.91985e-05 0.000154211 4.2744e-05 2.02479e-05 48
end_path:out1:RootOutput 4.567e-06 1.06599e-05 4.0711e-05 1.01805e-05 5.49141e-06 48
end_path:out1:RootOutput(write) 2.09962 2.81875 5.73432 2.54784 0.838959 48
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4839.1 MB
Peak resident set size usage (VmHWM): 2860.31 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622_reco_stage0_20260127T095108_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622_reco_stage0_20260127T095108_offline.root
Ran successfully
{
"name": "np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622_reco_stage0_20260127T095108_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_neutrino.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1769503821.0,
"core.end_time": 1769507471.0,
"core.events": [
46832,
46836,
46840,
46844,
46848,
46852,
46856,
46860,
46864,
46868,
46872,
46876,
46880,
46884,
46888,
46892,
46896,
46900,
46904,
46908,
46912,
46916,
46920,
46924,
46928,
46932,
46936,
46940,
46944,
46948,
46952,
46956,
46960,
46964,
46968,
46972,
46976,
46980,
46984,
46988,
46992,
46996,
47000,
47004,
47008,
47012,
47016,
47020
],
"core.event_count": 48,
"core.first_event_number": 46832,
"core.last_event_number": 47020,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39636
],
"core.runs_subruns": [
3963600001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039636_0245_df-s04-d0_dw_0_20250926T004622.hdf5"
}
]
}