Jobsub ID 32023.41@dunegpschedd02.fnal.gov
| Jobsub ID | 32023.41@dunegpschedd02.fnal.gov |
| Workflow ID | 2131 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-02 07:52:28 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2025-09-02 09:30:52 |
| From worker node | Hostname | ccwcondor0564 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-02 07:54:25 |
| Input files | vd-protodune:np02vd_raw_run039275_0102_df-s02-d1_dw_0_20250901T182028.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-02 09:30:52 |
| Saved logs | justin-logs:32023.41-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
frame tags:[ "framefanin" ] 16 tagged trace sets:[ "gauss0":3091 [0] "gauss1":8237 [0] "gauss2":8348 [0] "gauss3":6322 [0] "gauss4":13710 [0] "gauss5":7453 [0] "gauss6":8332 [0] "gauss7":10801 [0] "wiener0":3751 [3751] "wiener1":10080 [10080] "wiener2":10161 [10161] "wiener3":7807 [7807] "wiener4":17473 [17473] "wiener5":9348 [9348] "wiener6":10101 [10101] "wiener7":13392 [13392] ] cmm:[ ]
Retagger: tagging trace set: wiener with 82113 traces, 82113 summary
Retagger: tagging trace set: gauss with 66294 traces, 0 summary
[11:26:48.138] D [ glue ] sink frame: #37063 @7.2e-05 with 239499 traces, frame tags:[retagger], trace tags:[gauss, wiener]
[11:26:48.138] D [ glue ] <FrameFanout:nfsp> call=35: see EOS
[11:26:48.138] D [ glue ] <ChannelSelector:chsel7> see EOS at call=17
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode7sigproc7> EOS at call=35 anode=7
[11:26:48.138] D [ glue ] <ChannelSelector:chsel6> see EOS at call=17
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode6sigproc6> EOS at call=35 anode=6
[11:26:48.138] D [ glue ] <ChannelSelector:chsel5> see EOS at call=17
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode5sigproc5> EOS at call=35 anode=5
[11:26:48.138] D [ glue ] <ChannelSelector:chsel4> see EOS at call=17
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode4sigproc4> EOS at call=35 anode=4
[11:26:48.138] D [ glue ] <ChannelSelector:chsel3> see EOS at call=17
[11:26:48.138] D [ aux ] <Resampler:resmp3> EOS at call=35
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode3sigproc3> EOS at call=35 anode=3
[11:26:48.138] D [ glue ] <ChannelSelector:chsel2> see EOS at call=17
[11:26:48.138] D [ aux ] <Resampler:resmp2> EOS at call=35
[11:26:48.138] D [sigproc ] <OmnibusSigProc:anode2sigproc2> EOS at call=35 anode=2
[11:26:48.139] D [ glue ] <ChannelSelector:chsel1> see EOS at call=17
[11:26:48.139] D [ aux ] <Resampler:resmp1> EOS at call=35
[11:26:48.139] D [sigproc ] <OmnibusSigProc:anode1sigproc1> EOS at call=35 anode=1
[11:26:48.139] D [ glue ] <ChannelSelector:chsel0> see EOS at call=17
[11:26:48.139] D [ aux ] <Resampler:resmp0> EOS at call=35
[11:26:48.139] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=35 anode=0
[11:26:48.139] D [ glue ] <FrameFanin:nfsp> EOS at call=35 with 8
[11:26:48.139] D [ glue ] frame sink sees EOS
[11:26:48.139] D [ pgraph ] <Pgrapher:> graph execution complete
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 16.94 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.35 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 12.31 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.9 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.59 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.48 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.81 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.41 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Aux::Resampler : 1.17 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Aux::Resampler : 1.15 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Aux::Resampler : 1.1 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Aux::Resampler : 1.08 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:26:48.139] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:26:48.139] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:26:48.139] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[11:26:48.139] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[11:26:48.139] I [ timer ] Timer: Total node execution : 97.39000091329217 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 66294 traces tagged "gauss"
FrameSaver: q=1.17086e+07 n=1441972 tag=gauss
wclsFrameSaver: saving 82113 traces tagged "wiener"
FrameSaver: q=1.25711e+07 n=1377447 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 156 XUs and 223 XVs -> 9 XUVs
C:0 T:1 640 XUs and 760 XVs -> 35 XUVs
C:0 T:2 1137 XUs and 1322 XVs -> 128 XUVs
C:0 T:3 1947 XUs and 2312 XVs -> 78 XUVs
C:0 T:4 1700 XUs and 2227 XVs -> 95 XUVs
C:0 T:5 1122 XUs and 918 XVs -> 56 XUVs
C:0 T:6 2452 XUs and 2073 XVs -> 63 XUVs
C:0 T:7 246 XUs and 644 XVs -> 16 XUVs
C:0 T:8 19099 XUs and 47785 XVs -> 6191 XUVs
C:0 T:9 14702 XUs and 30655 XVs -> 4147 XUVs
C:0 T:10 17845 XUs and 33135 XVs -> 6213 XUVs
C:0 T:11 1179 XUs and 2734 XVs -> 166 XUVs
C:0 T:12 8384 XUs and 11747 XVs -> 1496 XUVs
C:0 T:13 1810 XUs and 1269 XVs -> 70 XUVs
C:0 T:14 1421 XUs and 2423 XVs -> 121 XUVs
C:0 T:15 12595 XUs and 18226 XVs -> 2563 XUVs
21447 XUVs total
2539 collection wire objects
21447 potential space points
Neighbour search...
3045207 tests to find 1252884 neighbours
Iterating with no regularization...
Begin: 7.2497e+08
0 6.59975e+08
1 6.53613e+08
2 6.52306e+08
3 6.51738e+08
Now with regularization...
Begin: 6.39298e+08
0 6.38901e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
02-Sep-2025 11:30:10 CEST Closed output file "np02vd_raw_run039275_0102_df-s02-d1_dw_0_20250901T182028_reco_stage1_20250902T093010_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 182.224 314.101 444.251 324.951 77.2262 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 9.0118e-05 0.000126158 0.000273982 0.000116793 4.05701e-05 18
produce:tpcrawdecoder:PDVDTPCReader 9.26706 10.0044 11.342 9.99016 0.535984 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.00594236 0.0116851 0.0257688 0.011502 0.00546016 18
produce:pdvddaphne:DAPHNEReaderPDVD 5.373 6.04031 6.78769 6.02464 0.337759 18
produce:ophit:OpHitFinder 0.0721651 0.0838352 0.103097 0.0840405 0.00708553 18
produce:opflash:OpFlashFinderVerticalDrift 0.00980957 0.0198949 0.0341229 0.0200637 0.00571308 18
produce:wclsdatavd:WireCellToolkit 85.5321 103.777 120.179 103.951 9.0996 18
produce:gaushit:GausHitFinder 1.60019 2.27119 3.08605 2.20143 0.426564 18
produce:nhitsfilter:NumberOfHitsFilter 0.000420428 0.000659857 0.00123731 0.000596766 0.000205565 18
produce:reco3d:SpacePointSolver 12.8975 24.8866 34.8694 25.8157 6.99309 18
produce:hitpdune:DisambigFromSpacePoints 0.207984 0.473149 0.710052 0.50808 0.157987 18
produce:pandora:StandardPandora 54.8266 155.257 270.474 160.686 64.0833 18
produce:pandoraTrack:LArPandoraTrackCreation 1.87678 3.73448 6.39156 3.58373 1.375 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0350162 0.0626846 0.0871374 0.0675421 0.015385 18
[art]:TriggerResults:TriggerResultInserter 4.6366e-05 7.44958e-05 0.000183633 6.90895e-05 2.90011e-05 18
end_path:out1:RootOutput 1.129e-05 1.88185e-05 6.2257e-05 1.63305e-05 1.16917e-05 18
end_path:out1:RootOutput(write) 6.7956 7.37318 8.59186 7.32124 0.426242 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5039.51 MB
Peak resident set size usage (VmHWM): 3112.94 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0102_df-s02-d1_dw_0_20250901T182028_reco_stage1_20250902T093010_keepup.root
\tHists: np02vd_raw_run039275_0102_df-s02-d1_dw_0_20250901T182028_reco_stage1_20250902T093010_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
