Jobsub ID 32069.4@dunegpschedd02.fnal.gov
| Jobsub ID | 32069.4@dunegpschedd02.fnal.gov |
| Workflow ID | 2128 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-02 09:10:32 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2025-09-02 10:30:11 |
| From worker node | Hostname | ccwcondor0051 |
| cpuinfo | AMD EPYC 9334 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-02 09:11:12 |
| Input files | vd-protodune:np02vd_raw_run039273_1518_df-s05-d5_dw_0_20250901T064021.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-02 10:30:11 |
| Saved logs | justin-logs:32069.4-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
oc" ] 4 tagged trace sets:[ "gauss7":11714 [0] "mp2_roi7":13790 [0] "mp3_roi7":3972 [0] "wiener7":14693 [14693] ] cmm:[ ] output: frame: ident=546839 time=40 tick=500 with 199683 traces. frame tags:[ "framefanin" ] 16 tagged trace sets:[ "gauss0":2195 [0] "gauss1":2974 [0] "gauss2":6008 [0] "gauss3":3964 [0] "gauss4":15782 [0] "gauss5":6725 [0] "gauss6":5662 [0] "gauss7":11714 [0] "wiener0":2858 [2858] "wiener1":3566 [3566] "wiener2":7668 [7668] "wiener3":4749 [4749] "wiener4":20227 [20227] "wiener5":8449 [8449] "wiener6":7113 [7113] "wiener7":14693 [14693] ] cmm:[ ]
Retagger: tagging trace set: wiener with 69323 traces, 69323 summary
Retagger: tagging trace set: gauss with 55024 traces, 0 summary
[12:26:39.401] D [ glue ] sink frame: #546839 @4e-05 with 199683 traces, frame tags:[retagger], trace tags:[gauss, wiener]
[12:26:39.401] D [ glue ] <FrameFanout:nfsp> call=35: see EOS
[12:26:39.401] D [ glue ] <ChannelSelector:chsel7> see EOS at call=17
[12:26:39.401] D [sigproc ] <OmnibusSigProc:anode7sigproc7> EOS at call=35 anode=7
[12:26:39.401] D [ glue ] <ChannelSelector:chsel6> see EOS at call=17
[12:26:39.401] D [sigproc ] <OmnibusSigProc:anode6sigproc6> EOS at call=35 anode=6
[12:26:39.401] D [ glue ] <ChannelSelector:chsel5> see EOS at call=17
[12:26:39.401] D [sigproc ] <OmnibusSigProc:anode5sigproc5> EOS at call=35 anode=5
[12:26:39.401] D [ glue ] <ChannelSelector:chsel4> see EOS at call=17
[12:26:39.402] D [sigproc ] <OmnibusSigProc:anode4sigproc4> EOS at call=35 anode=4
[12:26:39.402] D [ glue ] <ChannelSelector:chsel3> see EOS at call=17
[12:26:39.402] D [ aux ] <Resampler:resmp3> EOS at call=35
[12:26:39.402] D [sigproc ] <OmnibusSigProc:anode3sigproc3> EOS at call=35 anode=3
[12:26:39.402] D [ glue ] <ChannelSelector:chsel2> see EOS at call=17
[12:26:39.402] D [ aux ] <Resampler:resmp2> EOS at call=35
[12:26:39.402] D [sigproc ] <OmnibusSigProc:anode2sigproc2> EOS at call=35 anode=2
[12:26:39.402] D [ glue ] <ChannelSelector:chsel1> see EOS at call=17
[12:26:39.402] D [ aux ] <Resampler:resmp1> EOS at call=35
[12:26:39.402] D [sigproc ] <OmnibusSigProc:anode1sigproc1> EOS at call=35 anode=1
[12:26:39.402] D [ glue ] <ChannelSelector:chsel0> see EOS at call=17
[12:26:39.402] D [ aux ] <Resampler:resmp0> EOS at call=35
[12:26:39.402] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=35 anode=0
[12:26:39.402] D [ glue ] <FrameFanin:nfsp> EOS at call=35 with 8
[12:26:39.402] D [ glue ] frame sink sees EOS
[12:26:39.402] D [ pgraph ] <Pgrapher:> graph execution complete
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 20.26 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 11.71 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.93 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.6 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.89 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.75 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.13 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.6 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Aux::Resampler : 1.31 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Aux::Resampler : 1.27 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Aux::Resampler : 1.12 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Aux::Resampler : 0.97 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Gen::Retagger : 0 sec
[12:26:39.402] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[12:26:39.402] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[12:26:39.402] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[12:26:39.402] I [ timer ] Timer: Total node execution : 90.62000127136707 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 55024 traces tagged "gauss"
FrameSaver: q=1.10949e+07 n=1251375 tag=gauss
wclsFrameSaver: saving 69323 traces tagged "wiener"
FrameSaver: q=1.20258e+07 n=1195257 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 424 XUs and 209 XVs -> 17 XUVs
C:0 T:1 387 XUs and 391 XVs -> 15 XUVs
C:0 T:2 222 XUs and 155 XVs -> 8 XUVs
C:0 T:3 331 XUs and 503 XVs -> 16 XUVs
C:0 T:4 671 XUs and 881 XVs -> 32 XUVs
C:0 T:5 1626 XUs and 1294 XVs -> 81 XUVs
C:0 T:6 290 XUs and 487 XVs -> 18 XUVs
C:0 T:7 508 XUs and 704 XVs -> 38 XUVs
C:0 T:8 20067 XUs and 50312 XVs -> 5335 XUVs
C:0 T:9 26589 XUs and 59681 XVs -> 8137 XUVs
C:0 T:10 20977 XUs and 49358 XVs -> 7777 XUVs
C:0 T:11 726 XUs and 962 XVs -> 103 XUVs
C:0 T:12 12516 XUs and 12054 XVs -> 1557 XUVs
C:0 T:13 633 XUs and 483 XVs -> 35 XUVs
C:0 T:14 3488 XUs and 4715 XVs -> 268 XUVs
C:0 T:15 7798 XUs and 11151 XVs -> 1617 XUVs
25054 XUVs total
2856 collection wire objects
25054 potential space points
Neighbour search...
3015254 tests to find 1279062 neighbours
Iterating with no regularization...
Begin: 6.9364e+08
0 5.80883e+08
1 5.74142e+08
2 5.73009e+08
3 5.72603e+08
Now with regularization...
Begin: 5.53515e+08
0 5.53097e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
02-Sep-2025 12:29:32 CEST Closed output file "np02vd_raw_run039273_1518_df-s05-d5_dw_0_20250901T064021_reco_stage1_20250902T102932_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 156.782 258.508 378.911 235.887 68.3934 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 8.3326e-05 0.000271237 0.00301206 0.000108894 0.00066498 18
produce:tpcrawdecoder:PDVDTPCReader 7.53151 8.16087 8.89431 8.19247 0.335695 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.0056136 0.00768288 0.0223072 0.00580603 0.00453374 18
produce:pdvddaphne:DAPHNEReaderPDVD 3.87184 4.68384 5.67357 4.84831 0.513988 18
produce:ophit:OpHitFinder 0.0523525 0.059146 0.0651224 0.0597451 0.0037271 18
produce:opflash:OpFlashFinderVerticalDrift 0.00815688 0.0176377 0.0236478 0.0171899 0.00351873 18
produce:wclsdatavd:WireCellToolkit 78.6671 87.3985 101.102 86.7388 5.72176 18
produce:gaushit:GausHitFinder 1.07903 2.02919 2.86281 1.99112 0.399347 18
produce:nhitsfilter:NumberOfHitsFilter 0.000371764 0.000548042 0.000865411 0.000519032 0.000144019 18
produce:reco3d:SpacePointSolver 10.2099 22.9448 36.1958 21.4059 6.25171 18
produce:hitpdune:DisambigFromSpacePoints 0.166506 0.442614 0.889301 0.416571 0.166019 18
produce:pandora:StandardPandora 36.4485 123.066 212.495 102.498 58.0277 18
produce:pandoraTrack:LArPandoraTrackCreation 1.24088 2.74083 6.37235 2.37919 1.22651 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0312558 0.0532821 0.0881347 0.0511288 0.0133245 18
[art]:TriggerResults:TriggerResultInserter 4.4688e-05 6.51809e-05 0.000110107 5.94e-05 1.93586e-05 18
end_path:out1:RootOutput 9.054e-06 1.49466e-05 3.3291e-05 1.28845e-05 5.76045e-06 18
end_path:out1:RootOutput(write) 6.2892 6.80729 7.62801 6.73571 0.342146 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5197.97 MB
Peak resident set size usage (VmHWM): 3277.48 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039273_1518_df-s05-d5_dw_0_20250901T064021_reco_stage1_20250902T102932_keepup.root
\tHists: np02vd_raw_run039273_1518_df-s05-d5_dw_0_20250901T064021_reco_stage1_20250902T102932_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
