Jobsub ID 38128.143@dunegpschedd02.fnal.gov
| Jobsub ID | 38128.143@dunegpschedd02.fnal.gov |
| Workflow ID | 2494 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-11 10:10:31 |
| Site | CERN |
| Entry | CMSHTPC_T2_CH_CERN_ce513 |
| Last heartbeat | 2025-09-11 12:09:59 |
| From worker node | Hostname | b9p12p6840.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 343800 (95 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | aborted |
| Started | 2025-09-11 11:11:05 |
| Input files | vd-protodune:np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320.hdf5
|
| Outputting started | 2025-09-11 12:05:16 |
| Output files | |
| Finished | 2025-09-11 12:09:59 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
EOS at call=35
[14:03:51.038] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=35 anode=0
[14:03:51.038] D [ glue ] <FrameFanin:nfsp> EOS at call=35 with 8
[14:03:51.038] D [ glue ] frame sink sees EOS
[14:03:51.038] D [ pgraph ] <Pgrapher:> graph execution complete
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.07 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.85 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.77 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.25 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.27 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.06 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.68 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 5.35 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Aux::Resampler : 0.86 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Aux::Resampler : 0.64 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Aux::Resampler : 0.6 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Aux::Resampler : 0.58 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Gen::Retagger : 0 sec
[14:03:51.038] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[14:03:51.038] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[14:03:51.038] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[14:03:51.038] I [ timer ] Timer: Total node execution : 58.99999924376607 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 51280 traces tagged "gauss"
FrameSaver: q=9.94973e+06 n=1199751 tag=gauss
wclsFrameSaver: saving 63753 traces tagged "wiener"
FrameSaver: q=1.07754e+07 n=1149065 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1029 XUs and 1220 XVs -> 54 XUVs
C:0 T:1 423 XUs and 316 XVs -> 18 XUVs
C:0 T:2 316 XUs and 303 XVs -> 15 XUVs
C:0 T:3 277 XUs and 232 XVs -> 9 XUVs
C:0 T:4 358 XUs and 421 XVs -> 24 XUVs
C:0 T:5 1926 XUs and 2084 XVs -> 60 XUVs
C:0 T:6 372 XUs and 341 XVs -> 19 XUVs
C:0 T:7 781 XUs and 1131 XVs -> 30 XUVs
C:0 T:8 19707 XUs and 63667 XVs -> 6520 XUVs
C:0 T:9 4757 XUs and 4585 XVs -> 870 XUVs
C:0 T:10 6791 XUs and 10553 XVs -> 1618 XUVs
C:0 T:11 6155 XUs and 7446 XVs -> 896 XUVs
C:0 T:12 9078 XUs and 11813 XVs -> 899 XUVs
C:0 T:13 6022 XUs and 7537 XVs -> 1311 XUVs
C:0 T:14 7371 XUs and 14526 XVs -> 2195 XUVs
C:0 T:15 4550 XUs and 13219 XVs -> 1357 XUVs
15895 XUVs total
2533 collection wire objects
15895 potential space points
Neighbour search...
1597841 tests to find 737108 neighbours
Iterating with no regularization...
Begin: 7.07166e+08
0 6.12662e+08
1 6.07054e+08
2 6.06002e+08
3 6.05637e+08
Now with regularization...
Begin: 5.9054e+08
0 5.90189e+08
11-Sep-2025 14:05:04 CEST Closed output file "np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 114.431 177.913 278.316 175.498 42.3364 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.128e-05 8.39779e-05 0.000168572 7.8821e-05 2.70162e-05 18
produce:tpcrawdecoder:PDVDTPCReader 6.31446 6.87591 9.42719 6.60867 0.685983 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.00085782 0.00298557 0.010331 0.00150228 0.00277617 18
produce:pdvddaphne:DAPHNEReaderPDVD 3.23308 4.02904 4.51817 3.93 0.406435 18
produce:ophit:OpHitFinder 0.0369566 0.0473243 0.0591625 0.0467353 0.00548461 18
produce:opflash:OpFlashFinderVerticalDrift 0.00766751 0.0123532 0.0187449 0.012207 0.0036597 18
produce:wclsdatavd:WireCellToolkit 57.1717 66.027 77.6305 65.0028 5.17896 18
produce:gaushit:GausHitFinder 0.874663 1.45704 2.24191 1.37738 0.419681 18
produce:nhitsfilter:NumberOfHitsFilter 0.000231352 0.000348437 0.000660507 0.000324949 0.000108815 18
produce:reco3d:SpacePointSolver 10.1963 17.6353 29.2356 17.6859 4.39078 18
produce:hitpdune:DisambigFromSpacePoints 0.127637 0.291719 0.520262 0.262264 0.115594 18
produce:pandora:StandardPandora 29.8124 74.8327 148.914 69.3839 34.163 18
produce:pandoraTrack:LArPandoraTrackCreation 0.606399 1.51682 2.59656 1.34436 0.574868 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0178806 0.03054 0.0567716 0.0257402 0.0103271 18
[art]:TriggerResults:TriggerResultInserter 1.491e-05 2.07968e-05 4.963e-05 1.8745e-05 7.81905e-06 18
end_path:out1:RootOutput 3.31e-06 7.33944e-06 2.829e-05 6.47e-06 5.39304e-06 18
end_path:out1:RootOutput(write) 4.20229 5.11045 7.11602 4.84565 0.7912 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5215.4 MB
Peak resident set size usage (VmHWM): 3265.19 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup.root
\tHists: np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757592304.0,
"core.end_time": 1757592304.0,
"core.events": [
673922,
673942,
673962,
673982,
674002,
674022,
674042,
674062,
674082,
674102,
674122,
674142,
674162,
674182,
674202,
674222,
674242,
674262
],
"core.event_count": 18,
"core.first_event_number": 673922,
"core.last_event_number": 674262,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39255
],
"core.runs_subruns": [
3925500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320_reco_stage1_20250911T120504_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757592304.0,
"core.end_time": 1757592304.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39255
],
"core.runs_subruns": [
3925500001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039255_1872_df-s03-d3_dw_0_20250831T075320.hdf5"
}
]
} 