Jobsub ID 38205.98@dunegpschedd02.fnal.gov
| Jobsub ID | 38205.98@dunegpschedd02.fnal.gov |
| Workflow ID | 2486 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-11 10:56:33 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce506 |
| Last heartbeat | 2025-09-11 12:06:20 |
| From worker node | Hostname | b9p12p5961.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | stalled |
| Started | 2025-09-11 11:26:17 |
| Input files | vd-protodune:np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934.hdf5
|
| Outputting started | 2025-09-11 12:01:44 |
| Output files | |
| Finished | 2025-09-11 12:26:56 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
clsFrameSaver: no samples within desired window for channel 10295
wclsFrameSaver: no samples within desired window for channel 10295
wclsFrameSaver: no samples within desired window for channel 10332
wclsFrameSaver: no samples within desired window for channel 10841
wclsFrameSaver: no samples within desired window for channel 11022
wclsFrameSaver: no samples within desired window for channel 11023
wclsFrameSaver: no samples within desired window for channel 11026
wclsFrameSaver: no samples within desired window for channel 11026
wclsFrameSaver: no samples within desired window for channel 12055
wclsFrameSaver: no samples within desired window for channel 12059
wclsFrameSaver: no samples within desired window for channel 12060
wclsFrameSaver: no samples within desired window for channel 12061
wclsFrameSaver: no samples within desired window for channel 12061
wclsFrameSaver: no samples within desired window for channel 12062
wclsFrameSaver: no samples within desired window for channel 12062
wclsFrameSaver: no samples within desired window for channel 12063
wclsFrameSaver: no samples within desired window for channel 12063
wclsFrameSaver: no samples within desired window for channel 12064
wclsFrameSaver: no samples within desired window for channel 12064
wclsFrameSaver: no samples within desired window for channel 12064
wclsFrameSaver: no samples within desired window for channel 12065
wclsFrameSaver: no samples within desired window for channel 12065
wclsFrameSaver: no samples within desired window for channel 12066
wclsFrameSaver: no samples within desired window for channel 12066
wclsFrameSaver: no samples within desired window for channel 12067
wclsFrameSaver: no samples within desired window for channel 12067
FrameSaver: q=8.2701e+06 n=928527 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 575 XUs and 615 XVs -> 17 XUVs
C:0 T:1 862 XUs and 932 XVs -> 36 XUVs
C:0 T:2 661 XUs and 922 XVs -> 50 XUVs
C:0 T:3 612 XUs and 842 XVs -> 50 XUVs
C:0 T:4 1177 XUs and 1363 XVs -> 41 XUVs
C:0 T:5 896 XUs and 1134 XVs -> 59 XUVs
C:0 T:6 1701 XUs and 1501 XVs -> 48 XUVs
C:0 T:7 2924 XUs and 3540 XVs -> 127 XUVs
C:0 T:8 1691 XUs and 1809 XVs -> 86 XUVs
C:0 T:9 1878 XUs and 2127 XVs -> 101 XUVs
C:0 T:10 1397 XUs and 1876 XVs -> 91 XUVs
C:0 T:11 2907 XUs and 2840 XVs -> 180 XUVs
C:0 T:12 1244 XUs and 1880 XVs -> 74 XUVs
C:0 T:13 3106 XUs and 3429 XVs -> 237 XUVs
C:0 T:14 1708 XUs and 1899 XVs -> 84 XUVs
C:0 T:15 1493 XUs and 1494 XVs -> 85 XUVs
1366 XUVs total
1212 collection wire objects
1366 potential space points
Neighbour search...
8364 tests to find 3806 neighbours
Iterating with no regularization...
Begin: 1.2099e+09
0 1.20329e+09
1 1.20323e+09
Now with regularization...
Begin: 1.17639e+09
0 1.17632e+09
this->SetupTestBeamPfo(pNuPfo, pTestBeamPfo, testBeamStartVertex) return STATUS_CODE_NOT_FOUND
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArThreeDReco/LArEventBuilding/TestBeamParticleCreationAlgorithm.cc line#: 43
iter->second->Run() throw STATUS_CODE_NOT_FOUND
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0079, LArTestBeamParticleCreation, STATUS_CODE_NOT_FOUND
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
11-Sep-2025 14:01:16 CEST Closed output file "np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 92.7037 115.056 168.735 111.679 18.3303 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 4.413e-05 6.80432e-05 0.000132751 6.2371e-05 2.38101e-05 18
produce:tpcrawdecoder:PDVDTPCReader 5.58856 7.44393 18.9261 6.46322 3.26729 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.00569514 0.00840855 0.0151595 0.00694953 0.00320337 18
produce:pdvddaphne:DAPHNEReaderPDVD 1.68142 2.17466 2.56456 2.29125 0.287257 18
produce:ophit:OpHitFinder 0.0250232 0.0378802 0.0483593 0.037088 0.00660546 18
produce:opflash:OpFlashFinderVerticalDrift 0.00440806 0.00764149 0.0138492 0.00690573 0.00287524 18
produce:wclsdatavd:WireCellToolkit 46.5949 52.4856 61.5681 50.9133 5.20474 18
produce:gaushit:GausHitFinder 0.674807 0.980188 1.64153 0.942467 0.231327 18
produce:nhitsfilter:NumberOfHitsFilter 0.000190991 0.000248693 0.000444964 0.000237372 5.82434e-05 18
produce:reco3d:SpacePointSolver 7.36479 11.2185 15.322 11.4439 2.04161 18
produce:hitpdune:DisambigFromSpacePoints 0.0915169 0.142774 0.312408 0.1295 0.0484471 18
produce:pandora:StandardPandora 21.9041 35.7358 76.2818 34.3566 13.0855 18
produce:pandoraTrack:LArPandoraTrackCreation 0.389906 0.7186 1.08628 0.714144 0.207411 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0146118 0.0219399 0.0289787 0.0220867 0.00365059 18
[art]:TriggerResults:TriggerResultInserter 1.451e-05 2.0639e-05 4.068e-05 1.951e-05 5.34538e-06 18
end_path:out1:RootOutput 3.62e-06 7.85728e-06 2.451e-05 6.335e-06 5.57088e-06 18
end_path:out1:RootOutput(write) 3.81799 4.05103 4.46635 4.01647 0.17058 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4776.39 MB
Peak resident set size usage (VmHWM): 2815.51 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup.root
\tHists: np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757592077.0,
"core.end_time": 1757592077.0,
"core.events": [
150759,
150763,
150767,
150771,
150775,
150779,
150783,
150787,
150791,
150795,
150799,
150803,
150807,
150811,
150815,
150819,
150823,
150827
],
"core.event_count": 18,
"core.first_event_number": 150759,
"core.last_event_number": 150827,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39350
],
"core.runs_subruns": [
3935000001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934_reco_stage1_20250911T120116_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757592077.0,
"core.end_time": 1757592077.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39350
],
"core.runs_subruns": [
3935000001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039350_2064_df-s04-d3_dw_0_20250911T081934.hdf5"
}
]
} 