Jobsub ID 32245.5@dunegpschedd02.fnal.gov
| Jobsub ID | 32245.5@dunegpschedd02.fnal.gov |
| Workflow ID | 2183 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-02 15:22:50 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2025-09-02 16:00:03 |
| From worker node | Hostname | ccwcondor0564 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-02 15:23:22 |
| Input files | vd-protodune:np02vd_raw_run038563_0060_df-s05-d5_dw_0_20250804T192828.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-02 16:00:03 |
| Saved logs | justin-logs:32245.5-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss2":5907 [0] "mp2_roi2":7996 [0] "mp3_roi2":1355 [0] "wiener2":7499 [7499] ] cmm:[ ] input 3: frame: ident=12077 time=95 tick=500 with 20881 traces. frame tags:[ "sigproc" ] 4 tagged trace sets:[ "gauss3":5871 [0] "mp2_roi3":6294 [0] "mp3_roi3":1486 [0] "wiener3":7230 [7230] ] cmm:[ ] input 4: frame: ident=12077 time=95 tick=512 with 0 traces. frame tags:[ ] 0 tagged trace sets:[ ] cmm:[ ] input 5: frame: ident=12077 time=95 tick=512 with 0 traces. frame tags:[ ] 0 tagged trace sets:[ ] cmm:[ ] input 6: frame: ident=12077 time=95 tick=512 with 0 traces. frame tags:[ ] 0 tagged trace sets:[ ] cmm:[ ] input 7: frame: ident=12077 time=95 tick=512 with 0 traces. frame tags:[ ] 0 tagged trace sets:[ ] cmm:[ ] output: frame: ident=12077 time=95 tick=500 with 91019 traces. frame tags:[ "framefanin" ] 8 tagged trace sets:[ "gauss0":5018 [0] "gauss1":7708 [0] "gauss2":5907 [0] "gauss3":5871 [0] "wiener0":6583 [6583] "wiener1":9395 [9395] "wiener2":7499 [7499] "wiener3":7230 [7230] ] cmm:[ ]
Retagger: tagging trace set: wiener with 30707 traces, 30707 summary
Retagger: tagging trace set: gauss with 24504 traces, 0 summary
[17:59:11.658] D [ glue ] sink frame: #12077 @9.5e-05 with 91019 traces, frame tags:[retagger], trace tags:[gauss, wiener]
[17:59:11.659] D [ glue ] <FrameFanout:nfsp> call=65: see EOS
[17:59:11.659] D [ glue ] <ChannelSelector:chsel7> see EOS at call=32
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode7sigproc7> EOS at call=65 anode=7
[17:59:11.659] D [ glue ] <ChannelSelector:chsel6> see EOS at call=32
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode6sigproc6> EOS at call=65 anode=6
[17:59:11.659] D [ glue ] <ChannelSelector:chsel5> see EOS at call=32
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode5sigproc5> EOS at call=65 anode=5
[17:59:11.659] D [ glue ] <ChannelSelector:chsel4> see EOS at call=32
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode4sigproc4> EOS at call=65 anode=4
[17:59:11.659] D [ glue ] <ChannelSelector:chsel3> see EOS at call=32
[17:59:11.659] D [ aux ] <Resampler:resmp3> EOS at call=65
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode3sigproc3> EOS at call=65 anode=3
[17:59:11.659] D [ glue ] <ChannelSelector:chsel2> see EOS at call=32
[17:59:11.659] D [ aux ] <Resampler:resmp2> EOS at call=65
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode2sigproc2> EOS at call=65 anode=2
[17:59:11.659] D [ glue ] <ChannelSelector:chsel1> see EOS at call=32
[17:59:11.659] D [ aux ] <Resampler:resmp1> EOS at call=65
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode1sigproc1> EOS at call=65 anode=1
[17:59:11.659] D [ glue ] <ChannelSelector:chsel0> see EOS at call=32
[17:59:11.659] D [ aux ] <Resampler:resmp0> EOS at call=65
[17:59:11.659] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=65 anode=0
[17:59:11.659] D [ glue ] <FrameFanin:nfsp> EOS at call=65 with 8
[17:59:11.659] D [ glue ] frame sink sees EOS
[17:59:11.659] D [ pgraph ] <Pgrapher:> graph execution complete
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.28 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 8.21 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.63 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.47 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Aux::Resampler : 1.03 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Aux::Resampler : 1.03 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Aux::Resampler : 1.01 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Aux::Resampler : 0.62 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.02 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Gen::Retagger : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[17:59:11.659] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[17:59:11.659] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[17:59:11.659] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[17:59:11.659] I [ timer ] Timer: Total node execution : 34.3399996124208 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 24504 traces tagged "gauss"
FrameSaver: q=3.20814e+06 n=552632 tag=gauss
wclsFrameSaver: saving 30707 traces tagged "wiener"
FrameSaver: q=3.35541e+06 n=533993 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 2352 XUs and 1780 XVs -> 89 XUVs
C:0 T:1 1599 XUs and 1533 XVs -> 66 XUVs
C:0 T:2 2565 XUs and 2191 XVs -> 118 XUVs
C:0 T:3 1306 XUs and 1550 XVs -> 47 XUVs
C:0 T:4 944 XUs and 1164 XVs -> 40 XUVs
C:0 T:5 1322 XUs and 1668 XVs -> 77 XUVs
C:0 T:6 2096 XUs and 1828 XVs -> 66 XUVs
C:0 T:7 346 XUs and 488 XVs -> 23 XUVs
526 XUVs total
432 collection wire objects
526 potential space points
Neighbour search...
4030 tests to find 2450 neighbours
Iterating with no regularization...
Begin: 7.71804e+07
0 7.51023e+07
1 7.50937e+07
Now with regularization...
Begin: 7.31758e+07
0 7.31742e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
02-Sep-2025 17:59:40 CEST Closed output file "np02vd_raw_run038563_0060_df-s05-d5_dw_0_20250804T192828_reco_stage1_20250902T155940_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 52.8705 64.6969 87.7854 62.1858 7.7465 33
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.0993e-05 0.000102864 0.000276426 8.6602e-05 4.1783e-05 33
produce:tpcrawdecoder:PDVDTPCReader 3.7904 4.09606 4.48313 4.04824 0.160146 33
produce:triggerrawdecoder:PDVDTriggerReader4 0.00596056 0.00638108 0.00860097 0.00628397 0.000466866 33
produce:pdvddaphne:DAPHNEReaderPDVD 4.23189 4.88248 5.24906 4.95455 0.282242 33
produce:ophit:OpHitFinder 0.0478055 0.0677047 0.0829869 0.0688441 0.00810158 33
produce:opflash:OpFlashFinderVerticalDrift 0.00851227 0.0160885 0.0270012 0.0160951 0.00399532 33
produce:wclsdatavd:WireCellToolkit 30.3283 35.2833 39.1634 35.4903 2.03733 33
produce:gaushit:GausHitFinder 0.258225 0.573149 0.873389 0.567408 0.133186 33
produce:nhitsfilter:NumberOfHitsFilter 9.2613e-05 0.000152974 0.000298408 0.000146033 3.83246e-05 33
produce:reco3d:SpacePointSolver 2.29854 4.66251 10.4861 4.55533 1.84136 33
produce:hitpdune:DisambigFromSpacePoints 0.0174513 0.0424493 0.0950572 0.0358729 0.0181892 33
produce:pandora:StandardPandora 4.40411 11.4008 25.4304 9.57778 5.29084 33
produce:pandoraTrack:LArPandoraTrackCreation 0.0719034 0.206705 0.44421 0.173126 0.0944854 33
produce:pandoraGnocalo:GnocchiCalorimetry 0.00665599 0.0171756 0.0306981 0.0169709 0.00583083 33
[art]:TriggerResults:TriggerResultInserter 1.6511e-05 3.57326e-05 0.000108823 3.3252e-05 1.55014e-05 33
end_path:out1:RootOutput 3.276e-06 8.24003e-06 3.5046e-05 8.026e-06 5.17521e-06 33
end_path:out1:RootOutput(write) 2.49483 3.42135 5.5714 3.38574 0.516291 33
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4080.14 MB
Peak resident set size usage (VmHWM): 2162.16 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run038563_0060_df-s05-d5_dw_0_20250804T192828_reco_stage1_20250902T155940_keepup.root
\tHists: np02vd_raw_run038563_0060_df-s05-d5_dw_0_20250804T192828_reco_stage1_20250902T155940_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
