Jobsub ID 297920.16@dunegpschedd02.fnal.gov
| Jobsub ID | 297920.16@dunegpschedd02.fnal.gov |
| Workflow ID | 12719 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| Submitted time | 2026-02-03 22:54:27 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce15-multicore |
| Last heartbeat | 2026-02-04 00:35:44 |
| From worker node | Hostname | tdm003.pic.es |
| cpuinfo | AMD EPYC 7662 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-02-03 23:37:28 |
| Input files | vd-protodune:np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 56m (3404s) |
| CPU time | 48m (2936s = 86%) |
| Max RSS bytes | 3453042688 (3293 MiB) |
| Outputting started | 2026-02-04 00:34:13 |
| Output files | |
| Finished | 2026-02-04 00:35:44 |
| Saved logs | justin-logs:297920.16-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
tr... 0x699d62c0, 0x5bcbd000
Caught exception in looking up space point ptr... 0x3cec8720, 0x699d62c0
Caught exception in looking up space point ptr... 0x197d0f10, 0x3ddc0540
Caught exception in looking up space point ptr... 0x3ddc0540, 0x2f373bc0
Caught exception in looking up space point ptr... 0x2f373bc0, 0x192b8720
Caught exception in looking up space point ptr... 0x192b8720, 0x761dac70
Caught exception in looking up space point ptr... 0x3b8316e0, 0x3b831a00
Caught exception in looking up space point ptr... 0x3b831a00, 0x3b8316e0
Caught exception in looking up space point ptr... 0x894d000, 0x384273b0
Caught exception in looking up space point ptr... 0x5be38a80, 0x894d000
Caught exception in looking up space point ptr... 0x3908cd40, 0x3c24a1c0
Caught exception in looking up space point ptr... 0x3c24a1c0, 0x3908cd40
Caught exception in looking up space point ptr... 0x3b664ad0, 0x806b50b0
Caught exception in looking up space point ptr... 0x806b50b0, 0x3b664ad0
Caught exception in looking up space point ptr... 0x38150350, 0x2da16090
Caught exception in looking up space point ptr... 0x2da16090, 0x38150350
Caught exception in looking up space point ptr... 0x59988d80, 0x32cb0eb0
Caught exception in looking up space point ptr... 0x32cb0eb0, 0x32cb0930
Caught exception in looking up space point ptr... 0x330b8500, 0x330b6be0
Caught exception in looking up space point ptr... 0x330b6be0, 0x330b8e80
Caught exception in looking up space point ptr... 0x3b665040, 0x36666cd0
Caught exception in looking up space point ptr... 0x36666cd0, 0x3b665040
Caught exception in looking up space point ptr... 0x806b3c40, 0x806b3780
Caught exception in looking up space point ptr... 0x806b3780, 0x806b3c40
Caught exception in looking up space point ptr... 0x67f53230, 0x4d672020
Caught exception in looking up space point ptr... 0x4d672020, 0x67f53230
Caught exception in looking up space point ptr... 0x2f92e370, 0x121a4670
Caught exception in looking up space point ptr... 0x121a4670, 0x2f92e370
Caught exception in looking up space point ptr... 0x70d44b30, 0x70d475c0
Caught exception in looking up space point ptr... 0x70d475c0, 0x70d474c0
Caught exception in looking up space point ptr... 0x70d474c0, 0x70d44c30
Caught exception in looking up space point ptr... 0x18d94730, 0x18d94390
Caught exception in looking up space point ptr... 0x18d94390, 0x70d44b30
Caught exception in looking up space point ptr... 0x36b80020, 0x36b80120
Caught exception in looking up space point ptr... 0x36b80120, 0x36b80270
Caught exception in looking up space point ptr... 0x5872c190, 0x17931e40
Caught exception in looking up space point ptr... 0x17931e40, 0x5872c190
Caught exception in looking up space point ptr... 0x428340c0, 0x3aaf46e0
Caught exception in looking up space point ptr... 0x3aaf46e0, 0x3aaf4490
Caught exception in looking up space point ptr... 0x42834c80, 0x42833f50
Caught exception in looking up space point ptr... 0x42833f50, 0x428340c0
Caught exception in looking up space point ptr... 0x6645ed80, 0x6645ec30
Caught exception in looking up space point ptr... 0x6645ec30, 0x6645ed80
Caught exception in looking up space point ptr... 0x6645ed80, 0x54695620
Caught exception in looking up space point ptr... 0x54695620, 0x6645ed80
Caught exception in looking up space point ptr... 0x473a4140, 0x473a42d0
Caught exception in looking up space point ptr... 0x473a42d0, 0x473a4640
Caught exception in looking up space point ptr... 0x11e5ae30, 0x11e5b5d0
Caught exception in looking up space point ptr... 0x11e5b5d0, 0x46be8750
Caught exception in looking up space point ptr... 0x3a72b410, 0x46a66fc0
Caught exception in looking up space point ptr... 0x46a66fc0, 0x3a72b410
Caught exception in looking up space point ptr... 0x3670a9a0, 0x2f18f290
Caught exception in looking up space point ptr... 0x2f18f290, 0x3670a9a0
Caught exception in looking up space point ptr... 0x54399b30, 0x54399950
Caught exception in looking up space point ptr... 0x54399950, 0x5439aa10
Caught exception in looking up space point ptr... 0x3a6cafb0, 0xdae34f0
Caught exception in looking up space point ptr... 0xdae34f0, 0x49ed9b50
Caught exception in looking up space point ptr... 0x7e69fda0, 0x46be91b0
Caught exception in looking up space point ptr... 0x46be91b0, 0x46be8af0
Caught exception in looking up space point ptr... 0x46be8af0, 0x7e69ff30
Caught exception in looking up space point ptr... 0x7e69ff30, 0x46be8c80
Caught exception in looking up space point ptr... 0x36b803c0, 0x36b7f7b0
Caught exception in looking up space point ptr... 0x36b7f7b0, 0x36b804e0
Caught exception in looking up space point ptr... 0x63d0e410, 0x7e82d4a0
Caught exception in looking up space point ptr... 0x7e82d4a0, 0x63d0e410
Caught exception in looking up space point ptr... 0x3a44a860, 0x32161ae0
Caught exception in looking up space point ptr... 0x32161ae0, 0x32cb0ff0
++++>>>> total num hits: 50220, num free: 47858
04-Feb-2026 01:34:06 CET Closed output file "np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112_reco_stage0_20260204T003406_offline.root"
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 163.982 175.129 203.245 174.367 9.66907 19
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.4791e-05 0.000102529 0.000146542 9.4572e-05 2.21251e-05 19
produce:tpcrawdecoder:PDVDTPCReader 19.0587 24.1239 28.9886 25.7603 3.33701 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.911272 0.967155 1.05431 0.952649 0.0447384 19
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0293483 0.0361759 0.104154 0.0299549 0.0169141 19
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.0293187 0.0297346 0.0299659 0.0297799 0.000166558 19
produce:beamevent:BeamEvent 6.5831e-05 0.000149015 0.000976117 0.000102442 0.000195783 19
produce:pdvddaphne:DAPHNEReaderPDVD 3.01113 3.51577 3.89285 3.5472 0.217829 19
produce:ophit:OpHitFinder 0.0262306 0.0343437 0.0385375 0.0346843 0.00335051 19
produce:wclsdatavd:WireCellToolkit 107.528 113.568 123.357 112.949 3.73713 19
produce:gaushit:GausHitFinder 1.25668 1.66546 3.62109 1.45774 0.508743 19
produce:nhitsfilter:NumberOfHitsFilter 0.000374287 0.000515193 0.000838364 0.000478638 0.00010815 19
produce:reco3d:SpacePointSolver 14.5299 21.4133 34.0283 21.1842 5.25654 19
produce:hitpdune:DisambigFromSpacePoints 0.138627 0.205589 0.353606 0.193631 0.0513835 19
produce:cluster3d:Cluster3D 1.39452 3.98368 30.8436 2.52026 6.37375 19
[art]:TriggerResults:TriggerResultInserter 2.285e-05 2.91167e-05 0.000101811 2.47e-05 1.72183e-05 19
end_path:out1:RootOutput 3.54e-06 8.11279e-06 3.1911e-05 7.03e-06 5.86217e-06 19
end_path:out1:RootOutput(write) 5.14198 5.54542 6.32797 5.44819 0.303397 19
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5404.85 MB
Peak resident set size usage (VmHWM): 3453.04 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112_reco_stage0_20260204T003406_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112_reco_stage0_20260204T003406_offline.root
Ran successfully
{
"name": "np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112_reco_stage0_20260204T003406_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1770161857.0,
"core.end_time": 1770165247.0,
"core.events": [
32725,
32731,
32737,
32743,
32749,
32755,
32761,
32767,
32773,
32779,
32785,
32791,
32797,
32803,
32809,
32815,
32821,
32827,
32833
],
"core.event_count": 19,
"core.first_event_number": 32725,
"core.last_event_number": 32833,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
42433
],
"core.runs_subruns": [
4243300001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run042433_0287_df-s02-d1_dw_0_20260203T203112.hdf5"
}
]
} 