Jobsub ID 242987.2@dunegpschedd01.fnal.gov
| Jobsub ID | 242987.2@dunegpschedd01.fnal.gov |
| Workflow ID | 9584 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-11-04 14:28:11 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2025-11-04 16:05:53 |
| From worker node | Hostname | ccwcondor0397 |
| cpuinfo | AMD EPYC 9334 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-11-04 14:29:31 |
| Input files | vd-protodune:np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (5613s) |
| CPU time | 1h (5427s = 96%) |
| Max RSS bytes | 7452221440 (7106 MiB) |
| Outputting started | 2025-11-04 16:03:05 |
| Output files | |
| Finished | 2025-11-04 16:05:53 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
r ] Timer: WireCell::SigProc::OmnibusSigProc : 10.05 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.2 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.93 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.77 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.54 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.34 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.26 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.78 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Aux::Resampler : 0.25 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Aux::Resampler : 0.24 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Aux::Resampler : 0.22 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Aux::Resampler : 0.22 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:02:02.572] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:02:02.572] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[17:02:02.572] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[17:02:02.572] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[17:02:02.572] I [ timer ] Timer: Total node execution : 57.90000035241246 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 48876 traces tagged "gauss"
FrameSaver: q=1.2952e+07 n=1125407 tag=gauss
wclsFrameSaver: saving 62382 traces tagged "wiener"
FrameSaver: q=1.39534e+07 n=1079231 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 990 XUs and 1082 XVs -> 47 XUVs
C:0 T:1 485 XUs and 642 XVs -> 17 XUVs
C:0 T:2 779 XUs and 832 XVs -> 26 XUVs
C:0 T:3 93 XUs and 104 XVs -> 0 XUVs
C:0 T:4 7764 XUs and 11501 XVs -> 661 XUVs
C:0 T:5 653 XUs and 946 XVs -> 29 XUVs
C:0 T:6 804 XUs and 1019 XVs -> 47 XUVs
C:0 T:7 1197 XUs and 1456 XVs -> 57 XUVs
C:0 T:8 18884 XUs and 33410 XVs -> 6622 XUVs
C:0 T:9 6623 XUs and 8611 XVs -> 1482 XUVs
C:0 T:10 1985 XUs and 3027 XVs -> 547 XUVs
C:0 T:11 2648 XUs and 4202 XVs -> 1022 XUVs
C:0 T:12 564 XUs and 1156 XVs -> 173 XUVs
C:0 T:13 13555 XUs and 16879 XVs -> 2899 XUVs
C:0 T:14 7998 XUs and 16005 XVs -> 3247 XUVs
C:0 T:15 986 XUs and 1120 XVs -> 28 XUVs
16904 XUVs total
1951 collection wire objects
16904 potential space points
Neighbour search...
2661078 tests to find 1128686 neighbours
Iterating with no regularization...
Begin: 3.69786e+09
0 3.57039e+09
1 3.56403e+09
2 3.56325e+09
Now with regularization...
Begin: 3.51896e+09
0 3.51844e+09
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
04-Nov-2025 17:02:53 CET Closed output file "np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160253_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 88.3618 241.888 2715.33 121.137 529.076 23
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.7072e-05 0.000124588 0.000601673 0.000101815 0.000104916 23
produce:tpcrawdecoder:PDVDTPCReader 6.4166 7.42664 8.63634 7.33053 0.618537 23
produce:triggerrawdecoder:PDVDTriggerReader4 0.0852557 0.0992062 0.193225 0.0910431 0.0244755 23
produce:pdvddaphne:DAPHNEReaderPDVD 0.00031642 0.000396715 0.000753513 0.000366546 9.02569e-05 23
produce:ophit:OpHitFinder 6.632e-05 0.000146869 0.000628013 0.000128986 0.000106474 23
produce:opflash:OpFlashFinderVerticalDrift 4.61e-05 8.39701e-05 0.000325964 6.7523e-05 5.72236e-05 23
produce:wclsdatavd:WireCellToolkit 47.763 70.7298 179.484 60.7858 28.3671 23
produce:gaushit:GausHitFinder 0.5402 1.48094 4.05387 1.31344 0.703311 23
produce:nhitsfilter:NumberOfHitsFilter 0.00019139 0.000414064 0.000994759 0.000344342 0.000191535 23
produce:reco3d:SpacePointSolver 7.04301 15.3399 75.5952 12.1164 13.3772 23
produce:hitpdune:DisambigFromSpacePoints 0.0919129 0.333688 2.55243 0.218991 0.484341 23
produce:pandora:StandardPandora 9.73982 139.119 2417.64 28.7859 486.351 23
produce:pandoraTrack:LArPandoraTrackCreation 0.579024 2.39845 20.3674 1.22222 4.00412 23
produce:pandoraGnocalo:GnocchiCalorimetry 0.0217246 0.0399419 0.0734399 0.0389121 0.01311 23
[art]:TriggerResults:TriggerResultInserter 2.1223e-05 5.47313e-05 9.5074e-05 4.9916e-05 1.89733e-05 23
end_path:out1:RootOutput 5.057e-06 1.13999e-05 2.7381e-05 1.0706e-05 4.45387e-06 23
end_path:out1:RootOutput(write) 3.71382 4.80957 7.05053 4.61171 0.695988 23
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 9394.43 MB
Peak resident set size usage (VmHWM): 7452.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160253_keepup.root
\tHists: np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160254_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160253_keepup.root
Ran successfully
{
"name": "np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160253_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1762272175.0,
"core.end_time": 1762272175.0,
"core.events": [
119754,
119758,
119762,
119766,
119770,
119774,
119778,
119782,
119786,
119790,
119794,
119798,
119802,
119806,
119810,
119814,
119818,
119822,
119826,
119830,
119834,
119838,
119842
],
"core.event_count": 23,
"core.first_event_number": 119754,
"core.last_event_number": 119842,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40348
],
"core.runs_subruns": [
4034800001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935_reco_stage1_20251104T160254_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1762272175.0,
"core.end_time": 1762272175.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40348
],
"core.runs_subruns": [
4034800001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040348_1296_df-s04-d2_dw_0_20251104T130935.hdf5"
}
]
} 