Jobsub ID 238120.40@dunegpschedd01.fnal.gov
| Jobsub ID | 238120.40@dunegpschedd01.fnal.gov |
| Workflow ID | 9310 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-10-24 07:46:46 |
| Site | UK_RAL-PPD |
| Entry | CMSHTPC_T2_UK_SGrid_RALPP_hep206 |
| Last heartbeat | 2025-10-24 09:19:05 |
| From worker node | Hostname | heplnc164.pp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-10-24 07:47:22 |
| Input files | vd-protodune:np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 1h (5013s) |
| CPU time | 1h (4608s = 91%) |
| Max RSS bytes | 2983268352 (2845 MiB) |
| Outputting started | 2025-10-24 09:10:56 |
| Output files | |
| Finished | 2025-10-24 09:19:05 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
er:resmp0> EOS at call=95
[10:10:02.540] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=95 anode=0
[10:10:02.540] D [ glue ] <FrameFanin:nfsp> EOS at call=95 with 8
[10:10:02.540] D [ glue ] frame sink sees EOS
[10:10:02.540] D [ pgraph ] <Pgrapher:> graph execution complete
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.64 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.42 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.61 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.64 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Aux::Resampler : 0.29 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Aux::Resampler : 0.2 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Aux::Resampler : 0.2 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Aux::Resampler : 0.18 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[10:10:02.540] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[10:10:02.540] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[10:10:02.540] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[10:10:02.540] I [ timer ] Timer: Total node execution : 28.189999947324395 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 27139 traces tagged "gauss"
FrameSaver: q=7.69196e+06 n=658638 tag=gauss
wclsFrameSaver: saving 34587 traces tagged "wiener"
FrameSaver: q=8.23215e+06 n=638648 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 2890 XUs and 3939 XVs -> 147 XUVs
C:0 T:1 225 XUs and 320 XVs -> 9 XUVs
C:0 T:2 1515 XUs and 1745 XVs -> 49 XUVs
C:0 T:3 7199 XUs and 7803 XVs -> 614 XUVs
C:0 T:4 304 XUs and 370 XVs -> 11 XUVs
C:0 T:5 2094 XUs and 2467 XVs -> 145 XUVs
C:0 T:6 2610 XUs and 2785 XVs -> 283 XUVs
C:0 T:7 2990 XUs and 4166 XVs -> 187 XUVs
1445 XUVs total
691 collection wire objects
1445 potential space points
Neighbour search...
24701 tests to find 9104 neighbours
Iterating with no regularization...
Begin: 5.32137e+09
0 5.03822e+09
1 5.02618e+09
2 5.02601e+09
Now with regularization...
Begin: 4.88503e+09
0 4.8848e+09
24-Oct-2025 10:10:45 BST Closed output file "np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 45.2917 102.932 1021.69 69.1754 142.049 48
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.0232e-05 0.000101065 0.000259027 9.4037e-05 3.0783e-05 48
produce:tpcrawdecoder:PDVDTPCReader 6.04667 6.66682 7.63831 6.49962 0.456142 48
produce:triggerrawdecoder:PDVDTriggerReader4 0.298085 0.303479 0.334343 0.302361 0.00512375 48
produce:pdvddaphne:DAPHNEReaderPDVD 0.000329779 0.000445097 0.00126605 0.000409525 0.000145895 48
produce:ophit:OpHitFinder 6.3609e-05 0.000110944 0.000583016 9.58655e-05 7.32342e-05 48
produce:opflash:OpFlashFinderVerticalDrift 5.3461e-05 8.03646e-05 0.000390995 6.8649e-05 4.9023e-05 48
produce:wclsdatavd:WireCellToolkit 26.5921 41.0125 76.5353 35.4565 11.9204 48
produce:gaushit:GausHitFinder 0.323439 0.73557 1.63459 0.69153 0.239838 48
produce:nhitsfilter:NumberOfHitsFilter 9.7623e-05 0.000182613 0.000425921 0.000172775 6.39153e-05 48
produce:reco3d:SpacePointSolver 2.69623 6.82053 29.6252 5.41331 4.34519 48
produce:hitpdune:DisambigFromSpacePoints 0.0215599 0.09824 0.586877 0.0713508 0.0941166 48
produce:pandora:StandardPandora 4.95196 44.4779 908.179 14.8514 131.695 48
produce:pandoraTrack:LArPandoraTrackCreation 0.0999169 0.342514 1.3234 0.275141 0.256354 48
produce:pandoraGnocalo:GnocchiCalorimetry 0.00758105 0.016816 0.0398504 0.0161521 0.00635917 48
[art]:TriggerResults:TriggerResultInserter 1.5849e-05 3.13104e-05 8.3367e-05 2.7622e-05 1.46394e-05 48
end_path:out1:RootOutput 3.327e-06 7.93279e-06 2.659e-05 8.1e-06 3.44995e-06 48
end_path:out1:RootOutput(write) 1.70029 2.41864 3.57513 2.51184 0.422957 48
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4933.91 MB
Peak resident set size usage (VmHWM): 2983.27 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup.root
\tHists: np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup.root
Ran successfully
{
"name": "np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761297046.0,
"core.end_time": 1761297046.0,
"core.events": [
46617,
46621,
46625,
46629,
46633,
46637,
46641,
46645,
46649,
46653,
46657,
46661,
46665,
46669,
46673,
46677,
46681,
46685,
46689,
46693,
46697,
46701,
46705,
46709,
46713,
46717,
46721,
46725,
46729,
46733,
46737,
46741,
46745,
46749,
46753,
46757,
46761,
46765,
46769,
46773,
46777,
46781,
46785,
46789,
46793,
46797,
46801,
46805
],
"core.event_count": 48,
"core.first_event_number": 46617,
"core.last_event_number": 46805,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40222
],
"core.runs_subruns": [
4022200001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339_reco_stage1_20251024T091045_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761297046.0,
"core.end_time": 1761297046.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40222
],
"core.runs_subruns": [
4022200001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040222_0248_df-s05-d1_dw_0_20251024T045339.hdf5"
}
]
} 