Jobsub ID 36370.1@dunegpschedd02.fnal.gov
| Jobsub ID | 36370.1@dunegpschedd02.fnal.gov |
| Workflow ID | 2260 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 13:23:55 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2025-09-08 15:52:36 |
| From worker node | Hostname | cn-leo-040.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:48:21 |
| Input files | vd-protodune:np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 59m (3591s) |
| CPU time | 48m (2906s = 80%) |
| Max RSS bytes | 3067305984 (2925 MiB) |
| Outputting started | 2025-09-08 15:48:13 |
| Output files | |
| Finished | 2025-09-08 15:52:36 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
sSigProc:anode0sigproc0> EOS at call=51 anode=0
[17:43:54.303] D [ glue ] <FrameFanin:nfsp> EOS at call=51 with 8
[17:43:54.303] D [ glue ] frame sink sees EOS
[17:43:54.303] D [ pgraph ] <Pgrapher:> graph execution complete
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.3 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.67 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.62 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.44 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.33 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.32 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.27 sec
[17:43:54.303] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.19 sec
[17:43:54.303] I [ timer ] Timer: WireCell::Aux::Resampler : 0.24 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Aux::Resampler : 0.23 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Aux::Resampler : 0.22 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Aux::Resampler : 0.21 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Gen::Retagger : 0 sec
[17:43:54.304] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[17:43:54.304] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[17:43:54.304] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[17:43:54.304] I [ timer ] Timer: Total node execution : 53.08000033348799 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 33196 traces tagged "gauss"
FrameSaver: q=5.02827e+06 n=648906 tag=gauss
wclsFrameSaver: saving 41232 traces tagged "wiener"
FrameSaver: q=5.37857e+06 n=614828 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1523 XUs and 1809 XVs -> 112 XUVs
C:0 T:1 683 XUs and 394 XVs -> 23 XUVs
C:0 T:2 884 XUs and 968 XVs -> 26 XUVs
C:0 T:3 1476 XUs and 931 XVs -> 47 XUVs
C:0 T:4 246 XUs and 176 XVs -> 11 XUVs
C:0 T:5 479 XUs and 649 XVs -> 26 XUVs
C:0 T:6 143 XUs and 125 XVs -> 8 XUVs
C:0 T:7 455 XUs and 595 XVs -> 19 XUVs
C:0 T:8 12612 XUs and 18407 XVs -> 4151 XUVs
C:0 T:9 2297 XUs and 1701 XVs -> 112 XUVs
C:0 T:10 460 XUs and 601 XVs -> 38 XUVs
C:0 T:11 94 XUs and 162 XVs -> 12 XUVs
C:0 T:12 3124 XUs and 4341 XVs -> 236 XUVs
C:0 T:13 2225 XUs and 1942 XVs -> 202 XUVs
C:0 T:14 1490 XUs and 1896 XVs -> 92 XUVs
C:0 T:15 878 XUs and 776 XVs -> 52 XUVs
5167 XUVs total
1137 collection wire objects
5167 potential space points
Neighbour search...
821127 tests to find 335012 neighbours
Iterating with no regularization...
Begin: 3.47056e+08
0 3.25904e+08
1 3.24807e+08
2 3.24673e+08
Now with regularization...
Begin: 3.19315e+08
0 3.19217e+08
08-Sep-2025 17:44:25 CEST Closed output file "np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 83.1811 128.006 207.624 115.689 32.1705 26
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.4387e-05 8.10888e-05 0.000183361 7.7028e-05 2.19517e-05 26
produce:tpcrawdecoder:PDVDTPCReader 8.43896 18.2346 64.2874 14.6682 10.3976 26
produce:triggerrawdecoder:PDVDTriggerReader4 0.0207762 0.0217918 0.0224574 0.0217748 0.000336659 26
produce:pdvddaphne:DAPHNEReaderPDVD 2.15947 3.49663 17.7075 2.79575 2.91355 26
produce:ophit:OpHitFinder 0.0231907 0.0303186 0.0336598 0.0307334 0.00239249 26
produce:opflash:OpFlashFinderVerticalDrift 0.00387159 0.00858027 0.0134878 0.00852375 0.00227924 26
produce:wclsdatavd:WireCellToolkit 49.9717 56.8235 84.4551 53.9878 8.29999 26
produce:gaushit:GausHitFinder 0.469049 0.808881 1.41075 0.801461 0.229761 26
produce:nhitsfilter:NumberOfHitsFilter 0.000157571 0.00030164 0.000507177 0.000277362 8.54975e-05 26
produce:reco3d:SpacePointSolver 4.94756 9.621 20.4228 8.99253 3.60569 26
produce:hitpdune:DisambigFromSpacePoints 0.0494185 0.123238 0.262123 0.114642 0.0503751 26
produce:pandora:StandardPandora 10.676 35.3495 74.1921 30.0313 19.4845 26
produce:pandoraTrack:LArPandoraTrackCreation 0.197693 0.582594 1.65886 0.485188 0.327835 26
produce:pandoraGnocalo:GnocchiCalorimetry 0.00830269 0.0171706 0.0329856 0.0166537 0.00594214 26
[art]:TriggerResults:TriggerResultInserter 2.5403e-05 3.66788e-05 8.3182e-05 3.2731e-05 1.17315e-05 26
end_path:out1:RootOutput 5.89e-06 8.01215e-06 2.4565e-05 7.191e-06 3.36562e-06 26
end_path:out1:RootOutput(write) 2.40905 2.83865 4.28838 2.66844 0.493206 26
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4953.38 MB
Peak resident set size usage (VmHWM): 3067.31 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup.root
\tHists: np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346265.0,
"core.end_time": 1757346265.0,
"core.events": [
12403,
12423,
12443,
12463,
12483,
12503,
12523,
12543,
12563,
12583,
12603,
12623,
12643,
12663,
12683,
12703,
12723,
12743,
12763,
12783,
12803,
12823,
12843,
12863,
12883,
12903
],
"core.event_count": 26,
"core.first_event_number": 12403,
"core.last_event_number": 12903,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038_reco_stage1_20250908T154425_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346265.0,
"core.end_time": 1757346265.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39324
],
"core.runs_subruns": [
3932400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039324_0024_df-s02-d1_dw_0_20250905T151038.hdf5"
}
]
} 