Jobsub ID 36377.144@dunegpschedd02.fnal.gov
| Jobsub ID | 36377.144@dunegpschedd02.fnal.gov |
| Workflow ID | 2327 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 13:33:57 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce505 |
| Last heartbeat | 2025-09-08 15:52:37 |
| From worker node | Hostname | b9p06p5944.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:57:10 |
| Input files | vd-protodune:np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 53m (3204s) |
| CPU time | 47m (2820s = 88%) |
| Max RSS bytes | 3107446784 (2963 MiB) |
| Outputting started | 2025-09-08 15:50:35 |
| Output files | |
| Finished | 2025-09-08 15:52:37 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
c : 5.36 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 4.12 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 4.02 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 3.92 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 3.87 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 3.77 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 3.35 sec
[17:45:02.548] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 3.34 sec
[17:45:02.548] I [ timer ] Timer: WireCell::Aux::Resampler : 0.23 sec
[17:45:02.548] I [ timer ] Timer: WireCell::Aux::Resampler : 0.23 sec
[17:45:02.548] I [ timer ] Timer: WireCell::Aux::Resampler : 0.22 sec
[17:45:02.549] I [ timer ] Timer: WireCell::Aux::Resampler : 0.2 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:45:02.549] I [ timer ] Timer: wcls::FrameSaver : 0.01 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::Gen::Retagger : 0 sec
[17:45:02.549] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[17:45:02.549] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[17:45:02.549] I [ timer ] Timer: Total node execution : 32.65999977104366 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 39367 traces tagged "gauss"
FrameSaver: q=6.88647e+06 n=851330 tag=gauss
wclsFrameSaver: saving 48261 traces tagged "wiener"
FrameSaver: q=7.29177e+06 n=820363 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 49 XUs and 173 XVs -> 4 XUVs
C:0 T:1 600 XUs and 526 XVs -> 18 XUVs
C:0 T:2 37 XUs and 36 XVs -> 2 XUVs
C:0 T:3 9 XUs and 13 XVs -> 0 XUVs
C:0 T:4 2637 XUs and 2598 XVs -> 61 XUVs
C:0 T:5 1856 XUs and 1585 XVs -> 51 XUVs
C:0 T:6 920 XUs and 1080 XVs -> 44 XUVs
C:0 T:7 617 XUs and 687 XVs -> 19 XUVs
C:0 T:8 17304 XUs and 44545 XVs -> 7290 XUVs
C:0 T:9 1448 XUs and 5340 XVs -> 287 XUVs
C:0 T:10 4089 XUs and 3339 XVs -> 147 XUVs
C:0 T:11 1233 XUs and 1698 XVs -> 94 XUVs
C:0 T:12 690 XUs and 710 XVs -> 48 XUVs
C:0 T:13 3502 XUs and 4305 XVs -> 285 XUVs
C:0 T:14 503 XUs and 451 XVs -> 38 XUVs
C:0 T:15 1721 XUs and 1405 XVs -> 63 XUVs
8451 XUVs total
1079 collection wire objects
8451 potential space points
Neighbour search...
1240095 tests to find 488800 neighbours
Iterating with no regularization...
Begin: 3.32227e+08
0 3.16517e+08
1 3.1574e+08
2 3.15623e+08
Now with regularization...
Begin: 3.11979e+08
0 3.119e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 17:45:55 CEST Closed output file "np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 63.0473 107.256 218.257 100.441 30.9997 27
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 4.506e-05 8.89509e-05 0.000296369 7.379e-05 5.08931e-05 27
produce:tpcrawdecoder:PDVDTPCReader 3.55666 5.24029 24.0993 4.317 3.76736 27
produce:triggerrawdecoder:PDVDTriggerReader4 0.000905836 0.00333446 0.0510416 0.00124236 0.00942065 27
produce:pdvddaphne:DAPHNEReaderPDVD 1.89028 2.41092 3.08098 2.38391 0.341885 27
produce:ophit:OpHitFinder 0.0259575 0.0381902 0.052939 0.0385159 0.00644798 27
produce:opflash:OpFlashFinderVerticalDrift 0.00460791 0.00874758 0.0150557 0.00775795 0.00315875 27
produce:wclsdatavd:WireCellToolkit 28.7357 37.8816 59.3588 36.823 6.31669 27
produce:gaushit:GausHitFinder 0.593515 1.02333 1.88172 0.99001 0.245666 27
produce:nhitsfilter:NumberOfHitsFilter 0.000163839 0.000257835 0.000411868 0.00025691 5.59076e-05 27
produce:reco3d:SpacePointSolver 6.43357 10.7776 16.2496 10.563 2.61554 27
produce:hitpdune:DisambigFromSpacePoints 0.0741296 0.168904 0.341436 0.152929 0.0665094 27
produce:pandora:StandardPandora 14.7946 45.7065 122.867 39.312 21.68 27
produce:pandoraTrack:LArPandoraTrackCreation 0.22368 0.541804 0.925832 0.492231 0.20468 27
produce:pandoraGnocalo:GnocchiCalorimetry 0.0125422 0.0218239 0.0413403 0.0216946 0.00671368 27
[art]:TriggerResults:TriggerResultInserter 1.395e-05 2.12498e-05 4.506e-05 2.082e-05 6.25221e-06 27
end_path:out1:RootOutput 3.48e-06 6.62515e-06 2.106e-05 6.38e-06 3.37307e-06 27
end_path:out1:RootOutput(write) 2.79949 3.39526 4.90793 3.18043 0.565051 27
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5085.36 MB
Peak resident set size usage (VmHWM): 3107.45 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup.root
\tHists: np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346356.0,
"core.end_time": 1757346356.0,
"core.events": [
1330511,
1330531,
1330551,
1330571,
1330591,
1330611,
1330631,
1330651,
1330671,
1330691,
1330711,
1330731,
1330751,
1330771,
1330791,
1330811,
1330831,
1330851,
1330871,
1330891,
1330911,
1330931,
1330951,
1330971,
1330991,
1331011,
1331031
],
"core.event_count": 27,
"core.first_event_number": 1330511,
"core.last_event_number": 1331031,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958_reco_stage1_20250908T154555_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346356.0,
"core.end_time": 1757346356.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_2537_df-s04-d1_dw_0_20250907T212958.hdf5"
}
]
} 