justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242278.109@dunegpschedd01.fnal.gov

Jobsub ID242278.109@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 10:40:44
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2025-11-03 11:23:11
From worker nodeHostnameb9p22p8819.cern.ch
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 10:42:39
Input filesvd-protodune:np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032.hdf5
JobscriptExit code0
Real time38m (2299s)
CPU time36m (2202s = 95%)
Max RSS bytes3420803072 (3262 MiB)
Outputting started2025-11-03 11:20:59
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup.root
Finished2025-11-03 11:23:11
Saved logsjustin-logs:242278.109-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0:18.892] D [  glue  ] <FrameFanin:nfsp> EOS at call=45 with 8 
[12:20:18.892] D [  glue  ] frame sink sees EOS
[12:20:18.892] D [ pgraph ] <Pgrapher:> graph execution complete 
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.32 sec
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.29 sec
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.25 sec
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 9.08 sec
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.82 sec
[12:20:18.892] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.74 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.62 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 8.45 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.37 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.37 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.36 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.36 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.01 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[12:20:18.893] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[12:20:18.893] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[12:20:18.893] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[12:20:18.893] I [ timer  ] Timer: Total node execution : 73.05999877862632 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 44843 traces tagged "gauss"
FrameSaver: q=1.13408e+07 n=897132 tag=gauss
wclsFrameSaver: saving 55698 traces tagged "wiener"
FrameSaver: q=1.22353e+07 n=857917 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 650 XUs and 369 XVs -> 30 XUVs
C:0 T:1 316 XUs and 226 XVs -> 17 XUVs
C:0 T:2 450 XUs and 367 XVs -> 24 XUVs
C:0 T:3 1106 XUs and 974 XVs -> 29 XUVs
C:0 T:4 1007 XUs and 930 XVs -> 50 XUVs
C:0 T:5 575 XUs and 962 XVs -> 22 XUVs
C:0 T:6 11658 XUs and 15164 XVs -> 1165 XUVs
C:0 T:7 1441 XUs and 1485 XVs -> 69 XUVs
C:0 T:8 565 XUs and 682 XVs -> 34 XUVs
C:0 T:9 533 XUs and 811 XVs -> 37 XUVs
C:0 T:10 540 XUs and 857 XVs -> 40 XUVs
C:0 T:11 492 XUs and 647 XVs -> 31 XUVs
C:0 T:12 1583 XUs and 2159 XVs -> 120 XUVs
C:0 T:13 777 XUs and 943 XVs -> 78 XUVs
C:0 T:14 918 XUs and 1329 XVs -> 56 XUVs
C:0 T:15 314 XUs and 387 XVs -> 26 XUVs
1828 XUVs total
1105 collection wire objects
1828 potential space points
Neighbour search...
68312 tests to find 15428 neighbours
Iterating with no regularization...
Begin: 1.57088e+10
0 1.51829e+10
1 1.51364e+10
2 1.51327e+10
Now with regularization...
Begin: 1.49799e+10
0 1.49788e+10
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
03-Nov-2025 12:20:49 CET  Closed output file "np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        69.2404       97.9598       172.632       93.0277       21.7404        23     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       5.045e-05    7.96337e-05    0.0002151     6.702e-05    3.48397e-05      23     
produce:tpcrawdecoder:PDVDTPCReader               4.32057       5.59059       6.70791       5.54089      0.627007        23     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00983467     0.0130882     0.0305668     0.0113779    0.00479772       23     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000231951   0.000304967   0.000530832   0.000287561   6.80903e-05      23     
produce:ophit:OpHitFinder                        5.492e-05    9.1863e-05    0.000537772   6.4751e-05    9.64367e-05      23     
produce:opflash:OpFlashFinderVerticalDrift       3.359e-05    5.4478e-05    0.000281751    4.07e-05     4.93764e-05      23     
produce:wclsdatavd:WireCellToolkit                43.9636       59.0682       90.1704        60.87        12.6514        23     
produce:gaushit:GausHitFinder                    0.555767      0.802401       1.33249      0.786594      0.177696        23     
produce:nhitsfilter:NumberOfHitsFilter           0.0001562    0.000253694   0.000518202   0.00026073    7.06138e-05      23     
produce:reco3d:SpacePointSolver                   4.27349       8.85398       14.2202       9.04407       2.37569        23     
produce:hitpdune:DisambigFromSpacePoints         0.0757382     0.129337      0.237538      0.123641      0.0397021       23     
produce:pandora:StandardPandora                   9.05885       18.9896       56.852        16.4863       9.69178        23     
produce:pandoraTrack:LArPandoraTrackCreation     0.310179      0.630565       1.49213      0.639234      0.256873        23     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0155757     0.025399      0.0354119     0.025713     0.00489607       23     
[art]:TriggerResults:TriggerResultInserter       1.474e-05    2.38987e-05    7.553e-05     1.902e-05     1.303e-05       23     
end_path:out1:RootOutput                         3.62e-06     6.33091e-06    2.497e-05     4.67e-06     4.27909e-06      23     
end_path:out1:RootOutput(write)                   3.40021       3.81454       4.51623       3.76549      0.305222        23     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5493.35 MB
  Peak resident set size usage (VmHWM): 3420.8 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup.root
\tHists: np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762168849.0,
    "core.end_time": 1762168849.0,
    "core.events": [
      301779,
      301783,
      301787,
      301791,
      301795,
      301799,
      301803,
      301807,
      301811,
      301815,
      301819,
      301823,
      301827,
      301831,
      301835,
      301839,
      301843,
      301847,
      301851,
      301855,
      301859,
      301863,
      301867
    ],
    "core.event_count": 23,
    "core.first_event_number": 301779,
    "core.last_event_number": 301867,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032_reco_stage1_20251103T112049_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762168849.0,
    "core.end_time": 1762168849.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3270_df-s04-d3_dw_0_20251103T095032.hdf5"
    }
  ]
}
justIN time: 2025-11-03 18:32:45 UTC       justIN version: 01.05.01