Jobsub ID 240005.74@dunegpschedd01.fnal.gov
| Jobsub ID | 240005.74@dunegpschedd01.fnal.gov |
| Workflow ID | 9375 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-10-29 08:14:14 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce15-multicore |
| Last heartbeat | 2025-10-29 12:05:11 |
| From worker node | Hostname | tdm002.pic.es |
| cpuinfo | Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | aborted |
| Started | 2025-10-29 10:04:33 |
| Input files | vd-protodune:np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547.hdf5
|
| Outputting started | 2025-10-29 11:40:35 |
| Output files | |
| Finished | 2025-10-29 12:05:11 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
at call=45
[12:39:32.463] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=45 anode=0
[12:39:32.463] D [ glue ] <FrameFanin:nfsp> EOS at call=45 with 8
[12:39:32.463] D [ glue ] frame sink sees EOS
[12:39:32.464] D [ pgraph ] <Pgrapher:> graph execution complete
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 13.09 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.68 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.08 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 11.56 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 11.05 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.8 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.78 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Aux::Resampler : 0.45 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Aux::Resampler : 0.41 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Aux::Resampler : 0.31 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Aux::Resampler : 0.3 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Gen::Retagger : 0.02 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:39:32.464] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:39:32.464] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:39:32.464] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[12:39:32.464] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[12:39:32.464] I [ timer ] Timer: Total node execution : 91.6000009123236 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 36459 traces tagged "gauss"
FrameSaver: q=8.60257e+06 n=721178 tag=gauss
wclsFrameSaver: saving 44891 traces tagged "wiener"
FrameSaver: q=9.54784e+06 n=694472 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 340 XUs and 412 XVs -> 6 XUVs
C:0 T:1 939 XUs and 1097 XVs -> 46 XUVs
C:0 T:2 12168 XUs and 15313 XVs -> 966 XUVs
C:0 T:3 1069 XUs and 1253 XVs -> 50 XUVs
C:0 T:4 759 XUs and 707 XVs -> 35 XUVs
C:0 T:5 570 XUs and 571 XVs -> 32 XUVs
C:0 T:6 172 XUs and 126 XVs -> 3 XUVs
C:0 T:7 590 XUs and 778 XVs -> 17 XUVs
C:0 T:8 170 XUs and 191 XVs -> 19 XUVs
C:0 T:9 144 XUs and 295 XVs -> 8 XUVs
C:0 T:10 242 XUs and 254 XVs -> 21 XUVs
C:0 T:11 1170 XUs and 1726 XVs -> 61 XUVs
C:0 T:12 327 XUs and 264 XVs -> 15 XUVs
C:0 T:13 796 XUs and 761 XVs -> 37 XUVs
C:0 T:14 577 XUs and 951 XVs -> 28 XUVs
C:0 T:15 2439 XUs and 1985 XVs -> 81 XUVs
1425 XUVs total
837 collection wire objects
1425 potential space points
Neighbour search...
29131 tests to find 7862 neighbours
Iterating with no regularization...
Begin: 2.82878e+09
0 2.49855e+09
1 2.47981e+09
2 2.47867e+09
Now with regularization...
Begin: 2.43576e+09
0 2.43546e+09
29-Oct-2025 12:40:22 CET Closed output file "np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114022_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 163.525 247.535 451.092 228.828 79.6473 23
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 9.7541e-05 0.000137814 0.000295739 0.000121408 4.49282e-05 23
produce:tpcrawdecoder:PDVDTPCReader 20.2109 25.0437 38.0941 23.6466 4.53388 23
produce:triggerrawdecoder:PDVDTriggerReader4 0.50072 0.52876 0.861889 0.509025 0.072765 23
produce:pdvddaphne:DAPHNEReaderPDVD 0.00051642 0.000673816 0.00139256 0.00059365 0.000194072 23
produce:ophit:OpHitFinder 0.000119808 0.000207836 0.00101543 0.000157848 0.000175837 23
produce:opflash:OpFlashFinderVerticalDrift 7.2088e-05 0.000129801 0.000645138 0.000100433 0.000111548 23
produce:wclsdatavd:WireCellToolkit 93.1676 125.657 213.547 113.105 31.9526 23
produce:gaushit:GausHitFinder 1.21289 2.23998 4.1548 2.17715 0.643143 23
produce:nhitsfilter:NumberOfHitsFilter 0.000445093 0.000659582 0.00107728 0.000646799 0.00015715 23
produce:reco3d:SpacePointSolver 8.85376 17.9515 32.6818 16.734 6.16402 23
produce:hitpdune:DisambigFromSpacePoints 0.157626 0.40795 1.08547 0.345743 0.204183 23
produce:pandora:StandardPandora 23.8997 64.6476 183.327 49.0572 45.9188 23
produce:pandoraTrack:LArPandoraTrackCreation 0.812987 3.0643 9.29361 2.5794 1.91472 23
produce:pandoraGnocalo:GnocchiCalorimetry 0.0252124 0.0632665 0.118781 0.0663443 0.019269 23
[art]:TriggerResults:TriggerResultInserter 3.8435e-05 5.80229e-05 0.000160617 5.6614e-05 2.45155e-05 23
end_path:out1:RootOutput 7.338e-06 1.46492e-05 4.816e-05 1.2136e-05 8.25926e-06 23
end_path:out1:RootOutput(write) 5.01378 7.86311 10.7672 7.88316 1.21235 23
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5603.36 MB
Peak resident set size usage (VmHWM): 3674.94 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114022_keepup.root
\tHists: np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114023_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114022_keepup.root
Ran successfully
{
"name": "np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114022_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761738024.0,
"core.end_time": 1761738024.0,
"core.events": [
46567,
46575,
46583,
46591,
46599,
46607,
46615,
46623,
46631,
46639,
46647,
46655,
46663,
46671,
46679,
46687,
46695,
46703,
46711,
46719,
46727,
46735,
46743
],
"core.event_count": 23,
"core.first_event_number": 46567,
"core.last_event_number": 46743,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40267
],
"core.runs_subruns": [
4026700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547_reco_stage1_20251029T114023_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_11_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1761738024.0,
"core.end_time": 1761738024.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40267
],
"core.runs_subruns": [
4026700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040267_0251_df-s04-d3_dw_0_20251026T013547.hdf5"
}
]
} 