justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 48054.5@dunegpschedd01.fnal.gov

Jobsub ID48054.5@dunegpschedd01.fnal.gov
Workflow ID2862
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 03:53:59
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-09-18 04:59:41
From worker nodeHostnamemilan25.farm.particle.cz
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 03:55:50
Input filesvd-protodune:np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503.hdf5
JobscriptExit code0
Real time1h (3742s)
CPU time55m (3319s = 88%)
Max RSS bytes4336349184 (4135 MiB)
Outputting started2025-09-18 04:58:13
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045753_keepup.root
Finished2025-09-18 04:59:41
Saved logsjustin-logs:48054.5-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er:resmp0> EOS at call=97 
[06:57:29.286] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=97 anode=0 
[06:57:29.286] D [  glue  ] <FrameFanin:nfsp> EOS at call=97 with 8 
[06:57:29.286] D [  glue  ] frame sink sees EOS
[06:57:29.286] D [ pgraph ] <Pgrapher:> graph execution complete 
[06:57:29.286] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 7.2 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 6.36 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.5 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 5.1 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.4 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.4 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.39 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.39 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[06:57:29.287] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[06:57:29.287] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[06:57:29.287] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[06:57:29.287] I [ timer  ] Timer: Total node execution : 25.74999983049929 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 16646 traces tagged "gauss"
FrameSaver: q=3.18086e+06 n=393972 tag=gauss
wclsFrameSaver: saving 21508 traces tagged "wiener"
FrameSaver: q=3.38432e+06 n=378383 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 372 XUs and 296 XVs -> 10 XUVs
C:0 T:1 3780 XUs and 2516 XVs -> 179 XUVs
C:0 T:2 17117 XUs and 14950 XVs -> 873 XUVs
C:0 T:3 1165 XUs and 1370 XVs -> 57 XUVs
C:0 T:4 216 XUs and 198 XVs -> 9 XUVs
C:0 T:5 183 XUs and 332 XVs -> 18 XUVs
C:0 T:6 16 XUs and 14 XVs -> 0 XUVs
C:0 T:7 356 XUs and 524 XVs -> 29 XUVs
1175 XUVs total
752 collection wire objects
1175 potential space points
Neighbour search...
32963 tests to find 8492 neighbours
Iterating with no regularization...
Begin: 2.23321e+09
0 2.05334e+09
1 2.03894e+09
2 2.03888e+09
Now with regularization...
Begin: 2.02826e+09
0 2.02825e+09
18-Sep-2025 06:57:53 CEST  Closed output file "np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045753_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        42.7956       74.6007       424.79        58.469        57.4679        49     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.92e-05     0.000120066   0.00020964    0.00012171    3.15408e-05      49     
produce:tpcrawdecoder:PDVDTPCReader               5.52066       6.06455       8.27089       5.97767      0.471136        49     
produce:triggerrawdecoder:PDVDTriggerReader4     0.266875       0.27313      0.337921      0.270208      0.0121188       49     
produce:pdvddaphne:DAPHNEReaderPDVD               2.76253       3.20277       3.67203       3.22755      0.210653        49     
produce:ophit:OpHitFinder                        0.0394805     0.0522695     0.0724606     0.0511274    0.00599691       49     
produce:opflash:OpFlashFinderVerticalDrift       0.0019566     0.0106364     0.0235923     0.0106987    0.00386037       49     
produce:wclsdatavd:WireCellToolkit                23.2571       32.5193       91.5489       27.199        11.4632        49     
produce:gaushit:GausHitFinder                    0.305961      0.647473       1.88506      0.599512      0.254934        49     
produce:nhitsfilter:NumberOfHitsFilter           8.151e-05    0.000144998   0.000294381   0.00013125    4.18268e-05      49     
produce:reco3d:SpacePointSolver                   1.34142       4.88152       17.4422       4.2587        2.93258        49     
produce:hitpdune:DisambigFromSpacePoints         0.0116431     0.087654      0.429625      0.0635186     0.0787792       49     
produce:pandora:StandardPandora                   2.56966       23.9438       299.258       12.2799       44.2831        49     
produce:pandoraTrack:LArPandoraTrackCreation     0.0899797     0.296474       1.25958      0.199906      0.239847        49     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00756185     0.0151563     0.0310661     0.0146646    0.00466492       49     
[art]:TriggerResults:TriggerResultInserter       2.042e-05    2.9167e-05     8.628e-05     2.679e-05    1.02572e-05      49     
end_path:out1:RootOutput                         6.98e-06     8.65676e-06    2.459e-05     8.27e-06     2.43185e-06      49     
end_path:out1:RootOutput(write)                   2.24524       2.58921       3.68266       2.50423       0.2885         49     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6234.1 MB
  Peak resident set size usage (VmHWM): 4336.35 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045753_keepup.root
\tHists: np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045754_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045753_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045753_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758171475.0,
    "core.end_time": 1758171475.0,
    "core.events": [
      80367,
      80373,
      80379,
      80385,
      80391,
      80397,
      80403,
      80409,
      80415,
      80421,
      80427,
      80433,
      80439,
      80445,
      80451,
      80457,
      80463,
      80469,
      80475,
      80481,
      80487,
      80493,
      80499,
      80505,
      80511,
      80517,
      80523,
      80529,
      80535,
      80541,
      80547,
      80553,
      80559,
      80565,
      80571,
      80577,
      80583,
      80589,
      80595,
      80601,
      80607,
      80613,
      80619,
      80625,
      80631,
      80637,
      80643,
      80649,
      80655
    ],
    "core.event_count": 49,
    "core.first_event_number": 80367,
    "core.last_event_number": 80655,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503_reco_stage1_20250918T045754_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758171475.0,
    "core.end_time": 1758171475.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0280_df-s05-d3_dw_0_20250918T025503.hdf5"
    }
  ]
}
justIN time: 2025-09-18 19:19:09 UTC       justIN version: 01.05.00