justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 41395.150@dunegpschedd02.fnal.gov

Jobsub ID41395.150@dunegpschedd02.fnal.gov
Workflow ID2862
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-18 00:43:50
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-09-18 01:44:19
From worker nodeHostnamemilan25.farm.particle.cz
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-18 00:44:54
Input filesvd-protodune:np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624.hdf5
JobscriptExit code0
Real time58m (3486s)
CPU time51m (3106s = 89%)
Max RSS bytes3213295616 (3064 MiB)
Outputting started2025-09-18 01:43:01
Output filesvd-protodune-det-reco:np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014248_keepup.root
Finished2025-09-18 01:44:19
Saved logsjustin-logs:41395.150-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

] <Resampler:resmp0> EOS at call=95 
[03:42:26.127] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=95 anode=0 
[03:42:26.127] D [  glue  ] <FrameFanin:nfsp> EOS at call=95 with 8 
[03:42:26.127] D [  glue  ] frame sink sees EOS
[03:42:26.127] D [ pgraph ] <Pgrapher:> graph execution complete 
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.86 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.56 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.93 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.21 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.24 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.22 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.22 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.21 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.02 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0.01 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::Gen::Retagger : 0 sec
[03:42:26.127] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[03:42:26.127] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 0 sec
[03:42:26.127] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[03:42:26.127] I [ timer  ] Timer: Total node execution : 37.490000404417515 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 17451 traces tagged "gauss"
FrameSaver: q=3.98313e+06 n=456830 tag=gauss
wclsFrameSaver: saving 21426 traces tagged "wiener"
FrameSaver: q=4.20351e+06 n=444097 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 587 XUs and 817 XVs -> 34 XUVs
C:0 T:1 345 XUs and 454 XVs -> 11 XUVs
C:0 T:2 2324 XUs and 2362 XVs -> 68 XUVs
C:0 T:3 1606 XUs and 1834 XVs -> 99 XUVs
C:0 T:4 27 XUs and 43 XVs -> 3 XUVs
C:0 T:5 9554 XUs and 10290 XVs -> 837 XUVs
C:0 T:6 176 XUs and 201 XVs -> 2 XUVs
C:0 T:7 53 XUs and 47 XVs -> 3 XUVs
1057 XUVs total
680 collection wire objects
1057 potential space points
Neighbour search...
24991 tests to find 6502 neighbours
Iterating with no regularization...
Begin: 2.11e+10
0 2.03598e+10
1 2.03295e+10
2 2.0329e+10
Now with regularization...
Begin: 1.97076e+10
0 1.97065e+10
18-Sep-2025 03:42:48 CEST  Closed output file "np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014248_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        38.4975       71.0655       191.246       64.7728       25.7627        48     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.497e-05    0.000110854   0.00031211    9.2785e-05    5.13535e-05      48     
produce:tpcrawdecoder:PDVDTPCReader               4.93765       5.72431       7.57895       5.30956      0.760367        48     
produce:triggerrawdecoder:PDVDTriggerReader4     0.265438      0.270039      0.288367      0.267869     0.00540656       48     
produce:pdvddaphne:DAPHNEReaderPDVD               2.76505       3.21707       3.53767       3.23313      0.196777        48     
produce:ophit:OpHitFinder                        0.0374464     0.0518543     0.0748369     0.0499009     0.0074831       48     
produce:opflash:OpFlashFinderVerticalDrift      0.00053766     0.0106265     0.026637      0.010714     0.00441712       48     
produce:wclsdatavd:WireCellToolkit                22.5076       32.7024       60.7954       30.9789       7.64221        48     
produce:gaushit:GausHitFinder                    0.276219      0.637626      0.985898      0.615831      0.143817        48     
produce:nhitsfilter:NumberOfHitsFilter           8.003e-05    0.000165161   0.000425701   0.000154495   5.44655e-05      48     
produce:reco3d:SpacePointSolver                   1.61486        5.055        14.1746       4.53495       2.11114        48     
produce:hitpdune:DisambigFromSpacePoints         0.0250245     0.0887363     0.312681      0.0706177     0.0576598       48     
produce:pandora:StandardPandora                   2.41707       20.2721       102.207       14.2189       17.9663        48     
produce:pandoraTrack:LArPandoraTrackCreation     0.0815131     0.305727       1.33185      0.261426      0.204659        48     
produce:pandoraGnocalo:GnocchiCalorimetry       0.00733233     0.0160715     0.0298847     0.0158897    0.00408845       48     
[art]:TriggerResults:TriggerResultInserter       2.192e-05    3.1344e-05     6.059e-05    3.0315e-05    7.65148e-06      48     
end_path:out1:RootOutput                           4e-06      8.5494e-06     3.023e-05     7.77e-06     4.51281e-06      48     
end_path:out1:RootOutput(write)                   1.85752       2.69265       3.89839       2.63467      0.356446        48     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5116.6 MB
  Peak resident set size usage (VmHWM): 3213.3 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014248_keepup.root
\tHists: np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014249_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014248_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014248_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758159770.0,
    "core.end_time": 1758159770.0,
    "core.events": [
      59181,
      59187,
      59193,
      59199,
      59205,
      59211,
      59217,
      59223,
      59229,
      59235,
      59241,
      59247,
      59253,
      59259,
      59265,
      59271,
      59277,
      59283,
      59289,
      59295,
      59301,
      59307,
      59313,
      59319,
      59325,
      59331,
      59337,
      59343,
      59349,
      59355,
      59361,
      59367,
      59373,
      59379,
      59385,
      59391,
      59397,
      59403,
      59409,
      59415,
      59421,
      59427,
      59433,
      59439,
      59445,
      59451,
      59457,
      59463
    ],
    "core.event_count": 48,
    "core.first_event_number": 59181,
    "core.last_event_number": 59463,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624_reco_stage1_20250918T014249_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758159770.0,
    "core.end_time": 1758159770.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39433
    ],
    "core.runs_subruns": [
      3943300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039433_0206_df-s05-d3_dw_0_20250917T235624.hdf5"
    }
  ]
}
justIN time: 2025-09-18 19:19:09 UTC       justIN version: 01.05.00