justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242889.128@dunegpschedd01.fnal.gov

Jobsub ID242889.128@dunegpschedd01.fnal.gov
Workflow ID9584
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-04 11:46:03
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_brug_multicore
Last heartbeat2025-11-04 12:59:47
From worker nodeHostnamewn-snel-017.farm.nikhef.nl
cpuinfoAMD EPYC 7H12 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 11:54:08
Input filesvd-protodune:np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918.hdf5
JobscriptExit code0
Real time1h (3803s)
CPU time59m (3558s = 93%)
Max RSS bytes4707713024 (4489 MiB)
Outputting started2025-11-04 12:57:33
Output filesvd-protodune-det-reco:np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125720_keepup.root
Finished2025-11-04 12:59:47
Saved logsjustin-logs:242889.128-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

call=45 
[13:56:35.832] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=45 anode=0 
[13:56:35.832] D [  glue  ] <FrameFanin:nfsp> EOS at call=45 with 8 
[13:56:35.833] D [  glue  ] frame sink sees EOS
[13:56:35.833] D [ pgraph ] <Pgrapher:> graph execution complete 
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 10.54 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.22 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.55 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.18 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.12 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.12 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.03 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.84 sec
[13:56:35.833] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.31 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.3 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.22 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.21 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[13:56:35.834] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[13:56:35.834] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[13:56:35.834] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[13:56:35.834] I [ timer  ] Timer: Total node execution : 69.67000038735569 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 37681 traces tagged "gauss"
FrameSaver: q=1.11859e+07 n=824441 tag=gauss
wclsFrameSaver: saving 47963 traces tagged "wiener"
FrameSaver: q=1.19403e+07 n=779495 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 488 XUs and 446 XVs -> 24 XUVs
C:0 T:1 10485 XUs and 14788 XVs -> 809 XUVs
C:0 T:2 102 XUs and 94 XVs -> 2 XUVs
C:0 T:3 2592 XUs and 5271 XVs -> 169 XUVs
C:0 T:4 411 XUs and 302 XVs -> 27 XUVs
C:0 T:5 578 XUs and 728 XVs -> 24 XUVs
C:0 T:6 738 XUs and 712 XVs -> 33 XUVs
C:0 T:7 122 XUs and 217 XVs -> 5 XUVs
C:0 T:8 1105 XUs and 1391 XVs -> 40 XUVs
C:0 T:9 215 XUs and 299 XVs -> 24 XUVs
C:0 T:10 514 XUs and 562 XVs -> 40 XUVs
C:0 T:11 121 XUs and 151 XVs -> 13 XUVs
C:0 T:12 834 XUs and 848 XVs -> 65 XUVs
C:0 T:13 890 XUs and 9172 XVs -> 112 XUVs
C:0 T:14 10973 XUs and 20819 XVs -> 6056 XUVs
C:0 T:15 8180 XUs and 20811 XVs -> 3508 XUVs
10951 XUVs total
1370 collection wire objects
10951 potential space points
Neighbour search...
1957835 tests to find 745506 neighbours
Iterating with no regularization...
Begin: 1.21462e+11
0 1.21383e+11
Now with regularization...
Begin: 1.21323e+11
0 1.2132e+11
04-Nov-2025 13:57:20 CET  Closed output file "np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125720_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        107.464       162.495       362.175       143.267       63.4233        23     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      7.4038e-05    9.91226e-05   0.000300583   8.9788e-05    4.41475e-05      23     
produce:tpcrawdecoder:PDVDTPCReader               12.6534       14.2214       25.5823       13.6142       2.4666         23     
produce:triggerrawdecoder:PDVDTriggerReader4     0.504735      0.528661      0.550879      0.526862      0.0113056       23     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000323225   0.000509147   0.000890467   0.000460352   0.000137377      23     
produce:ophit:OpHitFinder                       5.3089e-05    0.000100925   0.000652952   7.4249e-05    0.000118375      23     
produce:opflash:OpFlashFinderVerticalDrift      3.8993e-05    7.56613e-05   0.000404858    5.866e-05    7.13136e-05      23     
produce:wclsdatavd:WireCellToolkit                55.198        81.9376       153.47        71.1439       22.0249        23     
produce:gaushit:GausHitFinder                    0.968403       1.34551       1.97462       1.31294      0.276093        23     
produce:nhitsfilter:NumberOfHitsFilter          0.000250929   0.000432912   0.000931183   0.000370042   0.000174673      23     
produce:reco3d:SpacePointSolver                   8.98995       13.2694       26.6027       11.7997       4.30065        23     
produce:hitpdune:DisambigFromSpacePoints         0.100899      0.219264      0.476234      0.196318      0.0876154       23     
produce:pandora:StandardPandora                   14.0518       44.2512       156.68        26.1743       41.0764        23     
produce:pandoraTrack:LArPandoraTrackCreation     0.531821       1.36444       4.74726       1.03592       0.95019        23     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0241915     0.0364572     0.0649293     0.0334935     0.0116268       23     
[art]:TriggerResults:TriggerResultInserter      1.8084e-05    3.14768e-05   7.8167e-05    2.3705e-05    1.58069e-05      23     
end_path:out1:RootOutput                         3.757e-06    6.07665e-06   2.0509e-05     5.01e-06     3.57187e-06      23     
end_path:out1:RootOutput(write)                   4.25484       5.2552        6.53553       5.07178      0.706941        23     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6645.58 MB
  Peak resident set size usage (VmHWM): 4707.71 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125720_keepup.root
\tHists: np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125721_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125720_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125720_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762261042.0,
    "core.end_time": 1762261042.0,
    "core.events": [
      105372,
      105376,
      105380,
      105384,
      105388,
      105392,
      105396,
      105400,
      105404,
      105408,
      105412,
      105416,
      105420,
      105424,
      105428,
      105432,
      105436,
      105440,
      105444,
      105448,
      105452,
      105456,
      105460
    ],
    "core.event_count": 23,
    "core.first_event_number": 105372,
    "core.last_event_number": 105460,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40348
    ],
    "core.runs_subruns": [
      4034800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918_reco_stage1_20251104T125721_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762261042.0,
    "core.end_time": 1762261042.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40348
    ],
    "core.runs_subruns": [
      4034800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040348_1140_df-s04-d0_dw_0_20251104T105918.hdf5"
    }
  ]
}
justIN time: 2025-11-05 08:41:12 UTC       justIN version: 01.05.01