justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242267.30@dunegpschedd01.fnal.gov

Jobsub ID242267.30@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 09:38:41
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-11-03 11:01:34
From worker nodeHostnamewn-s04-05-11-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 10:13:28
Input filesvd-protodune:np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101.hdf5
JobscriptExit code0
Real time41m (2511s)
CPU time33m (2031s = 80%)
Max RSS bytes3354542080 (3199 MiB)
Outputting started2025-11-03 10:55:20
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup.root
Finished2025-11-03 11:01:34
Saved logsjustin-logs:242267.30-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

[sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=45 anode=0 
[11:54:47.680] D [  glue  ] <FrameFanin:nfsp> EOS at call=45 with 8 
[11:54:47.680] D [  glue  ] frame sink sees EOS
[11:54:47.680] D [ pgraph ] <Pgrapher:> graph execution complete 
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.67 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.64 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.63 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.52 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.46 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.06 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 4.06 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 3.83 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.14 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.14 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.14 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.14 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[11:54:47.680] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[11:54:47.680] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[11:54:47.680] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[11:54:47.680] I [ timer  ] Timer: Total node execution : 35.47999988682568 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 41464 traces tagged "gauss"
FrameSaver: q=1.1695e+07 n=944978 tag=gauss
wclsFrameSaver: saving 51571 traces tagged "wiener"
FrameSaver: q=1.25537e+07 n=907888 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 834 XUs and 661 XVs -> 21 XUVs
C:0 T:1 7493 XUs and 11779 XVs -> 540 XUVs
C:0 T:2 1100 XUs and 1177 XVs -> 30 XUVs
C:0 T:3 2462 XUs and 3042 XVs -> 121 XUVs
C:0 T:4 1357 XUs and 1251 XVs -> 49 XUVs
C:0 T:5 887 XUs and 1171 XVs -> 24 XUVs
C:0 T:6 260 XUs and 313 XVs -> 5 XUVs
C:0 T:7 264 XUs and 259 XVs -> 25 XUVs
C:0 T:8 482 XUs and 663 XVs -> 47 XUVs
C:0 T:9 1037 XUs and 824 XVs -> 99 XUVs
C:0 T:10 1018 XUs and 767 XVs -> 84 XUVs
C:0 T:11 145 XUs and 174 XVs -> 18 XUVs
C:0 T:12 1199 XUs and 2130 XVs -> 94 XUVs
C:0 T:13 385 XUs and 652 XVs -> 44 XUVs
C:0 T:14 2071 XUs and 1946 XVs -> 171 XUVs
C:0 T:15 549 XUs and 857 XVs -> 62 XUVs
1434 XUVs total
985 collection wire objects
1434 potential space points
Neighbour search...
20260 tests to find 6916 neighbours
Iterating with no regularization...
Begin: 8.89936e+09
0 8.24574e+09
1 8.23531e+09
2 8.2352e+09
Now with regularization...
Begin: 8.12833e+09
0 8.12827e+09
03-Nov-2025 11:55:12 CET  Closed output file "np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        55.0933       95.8216       254.048       85.2003       40.1157        23     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      4.9173e-05    7.77593e-05   0.000146866   6.6496e-05    2.63841e-05      23     
produce:tpcrawdecoder:PDVDTPCReader               8.89215       10.2944       13.4942       9.93655       1.16195        23     
produce:triggerrawdecoder:PDVDTriggerReader4     0.298592      0.324218      0.578572      0.312928      0.055233        23     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000192513   0.00033358    0.000997987   0.000317608   0.000160545      23     
produce:ophit:OpHitFinder                       4.2019e-05    0.000132564   0.000902397   7.8357e-05    0.000171425      23     
produce:opflash:OpFlashFinderVerticalDrift      3.1409e-05    7.44676e-05   0.000528614   4.7178e-05    9.82139e-05      23     
produce:wclsdatavd:WireCellToolkit                32.0259       49.6743       83.1611       49.3248       13.4743        23     
produce:gaushit:GausHitFinder                    0.433383      0.840822       1.68813       0.66327      0.332115        23     
produce:nhitsfilter:NumberOfHitsFilter          0.000130516   0.000228942   0.000819722   0.000171082   0.000139162      23     
produce:reco3d:SpacePointSolver                   4.01364       8.10307       19.0594       7.08189       3.4763         23     
produce:hitpdune:DisambigFromSpacePoints         0.0442378     0.127644      0.405167      0.0959269     0.0820306       23     
produce:pandora:StandardPandora                   5.71801       22.0559       141.598       16.107        26.9993        23     
produce:pandoraTrack:LArPandoraTrackCreation     0.243633      0.669299       2.2285       0.592782      0.423063        23     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0115618     0.0238135     0.0594037     0.0171571     0.013265        23     
[art]:TriggerResults:TriggerResultInserter      1.2153e-05    2.63457e-05   0.00011705    1.9627e-05    2.11844e-05      23     
end_path:out1:RootOutput                         6.472e-06    1.06354e-05   4.3753e-05     7.694e-06    7.69749e-06      23     
end_path:out1:RootOutput(write)                   2.70968       3.67418       5.95753       3.18667      0.995458        23     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5394.18 MB
  Peak resident set size usage (VmHWM): 3354.54 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup.root
\tHists: np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762167313.0,
    "core.end_time": 1762167313.0,
    "core.events": [
      297713,
      297717,
      297721,
      297725,
      297729,
      297733,
      297737,
      297741,
      297745,
      297749,
      297753,
      297757,
      297761,
      297765,
      297769,
      297773,
      297777,
      297781,
      297785,
      297789,
      297793,
      297797,
      297801
    ],
    "core.event_count": 23,
    "core.first_event_number": 297713,
    "core.last_event_number": 297801,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101_reco_stage1_20251103T105512_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762167313.0,
    "core.end_time": 1762167313.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3224_df-s04-d1_dw_0_20251103T091101.hdf5"
    }
  ]
}
justIN time: 2025-11-03 18:32:40 UTC       justIN version: 01.05.01