Jobsub ID 38909.5@dunegpschedd01.fnal.gov
| Jobsub ID | 38909.5@dunegpschedd01.fnal.gov |
| Workflow ID | 2131 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-02 12:14:40 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2025-09-02 13:16:02 |
| From worker node | Hostname | ccwcondor0051 |
| cpuinfo | AMD EPYC 9334 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-09-02 12:15:53 |
| Input files | vd-protodune:np02vd_raw_run039275_0112_df-s01-d2_dw_0_20250901T183227.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-09-02 13:16:02 |
| Saved logs | justin-logs:38909.5-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
call=35: see EOS
[15:12:00.340] D [ glue ] <ChannelSelector:chsel7> see EOS at call=17
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode7sigproc7> EOS at call=35 anode=7
[15:12:00.340] D [ glue ] <ChannelSelector:chsel6> see EOS at call=17
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode6sigproc6> EOS at call=35 anode=6
[15:12:00.340] D [ glue ] <ChannelSelector:chsel5> see EOS at call=17
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode5sigproc5> EOS at call=35 anode=5
[15:12:00.340] D [ glue ] <ChannelSelector:chsel4> see EOS at call=17
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode4sigproc4> EOS at call=35 anode=4
[15:12:00.340] D [ glue ] <ChannelSelector:chsel3> see EOS at call=17
[15:12:00.340] D [ aux ] <Resampler:resmp3> EOS at call=35
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode3sigproc3> EOS at call=35 anode=3
[15:12:00.340] D [ glue ] <ChannelSelector:chsel2> see EOS at call=17
[15:12:00.340] D [ aux ] <Resampler:resmp2> EOS at call=35
[15:12:00.340] D [sigproc ] <OmnibusSigProc:anode2sigproc2> EOS at call=35 anode=2
[15:12:00.340] D [ glue ] <ChannelSelector:chsel1> see EOS at call=17
[15:12:00.341] D [ aux ] <Resampler:resmp1> EOS at call=35
[15:12:00.341] D [sigproc ] <OmnibusSigProc:anode1sigproc1> EOS at call=35 anode=1
[15:12:00.341] D [ glue ] <ChannelSelector:chsel0> see EOS at call=17
[15:12:00.341] D [ aux ] <Resampler:resmp0> EOS at call=35
[15:12:00.341] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=35 anode=0
[15:12:00.341] D [ glue ] <FrameFanin:nfsp> EOS at call=35 with 8
[15:12:00.341] D [ glue ] frame sink sees EOS
[15:12:00.341] D [ pgraph ] <Pgrapher:> graph execution complete
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.63 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 10.15 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 9.03 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.92 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.89 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.71 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 7.52 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::OmnibusSigProc : 6.74 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Aux::Resampler : 0.89 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Aux::Resampler : 0.88 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Aux::Resampler : 0.86 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Aux::Resampler : 0.83 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Gen::Retagger : 0.01 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Gen::DumpFrames : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[15:12:00.341] I [ timer ] Timer: WireCell::Gen::FrameFanout : 0 sec
[15:12:00.341] I [ timer ] Timer: wcls::RawFrameSource : 0 sec
[15:12:00.341] I [ timer ] Timer: wcls::FrameSaver : 0 sec
[15:12:00.341] I [ timer ] Timer: Total node execution : 71.1099991761148 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 62844 traces tagged "gauss"
FrameSaver: q=1.09675e+07 n=1365861 tag=gauss
wclsFrameSaver: saving 76685 traces tagged "wiener"
FrameSaver: q=1.16384e+07 n=1316798 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1320 XUs and 1535 XVs -> 51 XUVs
C:0 T:1 358 XUs and 280 XVs -> 6 XUVs
C:0 T:2 685 XUs and 746 XVs -> 33 XUVs
C:0 T:3 1307 XUs and 1432 XVs -> 69 XUVs
C:0 T:4 569 XUs and 645 XVs -> 19 XUVs
C:0 T:5 301 XUs and 464 XVs -> 19 XUVs
C:0 T:6 711 XUs and 470 XVs -> 43 XUVs
C:0 T:7 668 XUs and 816 XVs -> 12 XUVs
C:0 T:8 13387 XUs and 31841 XVs -> 4222 XUVs
C:0 T:9 3374 XUs and 3482 XVs -> 305 XUVs
C:0 T:10 4590 XUs and 10117 XVs -> 682 XUVs
C:0 T:11 2285 XUs and 2605 XVs -> 98 XUVs
C:0 T:12 8527 XUs and 10914 XVs -> 930 XUVs
C:0 T:13 13896 XUs and 27811 XVs -> 3203 XUVs
C:0 T:14 1925 XUs and 2292 XVs -> 95 XUVs
C:0 T:15 4682 XUs and 4261 XVs -> 558 XUVs
10345 XUVs total
1995 collection wire objects
10345 potential space points
Neighbour search...
1073785 tests to find 461202 neighbours
Iterating with no regularization...
Begin: 4.78339e+08
0 4.33881e+08
1 4.30772e+08
2 4.30362e+08
Now with regularization...
Begin: 4.21138e+08
0 4.20899e+08
this->SetupTestBeamPfo(pNuPfo, pTestBeamPfo, testBeamStartVertex) return STATUS_CODE_NOT_FOUND
in function: Run
in file: /scratch/workspace/build-larsoft/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/ALMA9/build/larpandoracontent/v04_16_00-buildFW/src/larpandoracontent/LArThreeDReco/LArEventBuilding/TestBeamParticleCreationAlgorithm.cc line#: 43
iter->second->Run() throw STATUS_CODE_NOT_FOUND
in function: RunAlgorithm
in file: /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0079, LArTestBeamParticleCreation, STATUS_CODE_NOT_FOUND
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
02-Sep-2025 15:15:29 CEST Closed output file "np02vd_raw_run039275_0112_df-s01-d2_dw_0_20250901T183227_reco_stage1_20250902T131529_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 143.439 195.327 295.481 188.631 37.1803 18
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 5.7837e-05 0.000116589 0.000260695 9.04625e-05 5.77126e-05 18
produce:tpcrawdecoder:PDVDTPCReader 8.13848 8.9558 9.53433 9.09261 0.412198 18
produce:triggerrawdecoder:PDVDTriggerReader4 0.00744026 0.0106256 0.0185133 0.00813477 0.00387211 18
produce:pdvddaphne:DAPHNEReaderPDVD 3.73402 4.18968 4.82444 4.20805 0.309959 18
produce:ophit:OpHitFinder 0.0470599 0.0599996 0.0700442 0.0607517 0.0054754 18
produce:opflash:OpFlashFinderVerticalDrift 0.0078879 0.0153249 0.0240273 0.0142683 0.00452186 18
produce:wclsdatavd:WireCellToolkit 66.5364 76.2234 86.7635 76.5553 5.2268 18
produce:gaushit:GausHitFinder 1.18914 1.66015 2.24933 1.64342 0.304491 18
produce:nhitsfilter:NumberOfHitsFilter 0.000247505 0.000371759 0.000724619 0.000343115 0.000112813 18
produce:reco3d:SpacePointSolver 10.1048 17.7957 26.4921 17.2995 4.50704 18
produce:hitpdune:DisambigFromSpacePoints 0.143605 0.321292 0.553766 0.301973 0.0954843 18
produce:pandora:StandardPandora 34.2174 78.3262 171.296 72.3263 31.2509 18
produce:pandoraTrack:LArPandoraTrackCreation 0.70288 1.49813 3.00148 1.49438 0.558989 18
produce:pandoraGnocalo:GnocchiCalorimetry 0.0284597 0.041233 0.0526077 0.0413057 0.00671681 18
[art]:TriggerResults:TriggerResultInserter 3.5464e-05 5.36524e-05 0.000121775 4.98855e-05 1.78593e-05 18
end_path:out1:RootOutput 5.678e-06 1.27599e-05 3.8669e-05 1.20985e-05 6.90115e-06 18
end_path:out1:RootOutput(write) 5.58082 6.13404 6.47079 6.17145 0.249423 18
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5039.71 MB
Peak resident set size usage (VmHWM): 3110.79 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0112_df-s01-d2_dw_0_20250901T183227_reco_stage1_20250902T131529_keepup.root
\tHists: np02vd_raw_run039275_0112_df-s01-d2_dw_0_20250901T183227_reco_stage1_20250902T131530_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata 