justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242250.105@dunegpschedd01.fnal.gov

Jobsub ID242250.105@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 08:36:38
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-11-03 10:37:19
From worker nodeHostnamecn007.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 08:38:01
Input filesvd-protodune:np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014.hdf5
JobscriptExit code0
Real time1h (7023s)
CPU time1h (6711s = 95%)
Max RSS bytes6994075648 (6670 MiB)
Outputting started2025-11-03 10:35:05
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103452_keepup.root
Finished2025-11-03 10:37:19
Saved logsjustin-logs:242250.105-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

usSigProc:anode0sigproc0> EOS at call=47 anode=0 
[10:33:49.540] D [  glue  ] <FrameFanin:nfsp> EOS at call=47 with 8 
[10:33:49.540] D [  glue  ] frame sink sees EOS
[10:33:49.541] D [ pgraph ] <Pgrapher:> graph execution complete 
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 13.98 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 13.88 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.86 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.69 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.48 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.35 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 10.49 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 10.3 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.45 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.37 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.35 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Aux::Resampler : 0.34 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.03 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.02 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[10:33:49.541] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[10:33:49.541] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[10:33:49.541] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[10:33:49.541] I [ timer  ] Timer: Total node execution : 99.67999876663089 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 36424 traces tagged "gauss"
FrameSaver: q=9.17846e+06 n=801582 tag=gauss
wclsFrameSaver: saving 45003 traces tagged "wiener"
FrameSaver: q=9.88293e+06 n=768336 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1217 XUs and 900 XVs -> 75 XUVs
C:0 T:1 1519 XUs and 990 XVs -> 40 XUVs
C:0 T:2 196 XUs and 172 XVs -> 11 XUVs
C:0 T:3 989 XUs and 803 XVs -> 27 XUVs
C:0 T:4 403 XUs and 544 XVs -> 36 XUVs
C:0 T:5 385 XUs and 465 XVs -> 10 XUVs
C:0 T:6 11208 XUs and 11997 XVs -> 1024 XUVs
C:0 T:7 517 XUs and 635 XVs -> 18 XUVs
C:0 T:8 821 XUs and 1117 XVs -> 39 XUVs
C:0 T:9 1807 XUs and 2207 XVs -> 119 XUVs
C:0 T:10 274 XUs and 331 XVs -> 20 XUVs
C:0 T:11 235 XUs and 258 XVs -> 18 XUVs
C:0 T:12 576 XUs and 868 XVs -> 45 XUVs
C:0 T:13 209 XUs and 544 XVs -> 14 XUVs
C:0 T:14 404 XUs and 437 XVs -> 23 XUVs
C:0 T:15 980 XUs and 1034 XVs -> 67 XUVs
1586 XUVs total
1051 collection wire objects
1586 potential space points
Neighbour search...
56348 tests to find 12792 neighbours
Iterating with no regularization...
Begin: 6.99562e+09
0 6.41623e+09
1 6.41317e+09
Now with regularization...
Begin: 6.2535e+09
0 6.25323e+09
03-Nov-2025 10:34:52 GMT  Closed output file "np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103452_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        149.327       289.622       1842.07       195.676       334.533        24     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       7.791e-05    0.000107983   0.000229348   9.55475e-05   3.26854e-05      24     
produce:tpcrawdecoder:PDVDTPCReader               12.596        15.0041       20.4324       14.4238       2.10914        24     
produce:triggerrawdecoder:PDVDTriggerReader4     0.496907      0.503565      0.522152      0.500641     0.00724516       24     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000459799   0.000612498   0.00103765    0.000569339   0.000117039      24     
produce:ophit:OpHitFinder                       8.1206e-05    0.000152654   0.00070976    0.000127739   0.000120005      24     
produce:opflash:OpFlashFinderVerticalDrift      7.2584e-05    0.000114796   0.000448688   9.12305e-05   7.43809e-05      24     
produce:wclsdatavd:WireCellToolkit                78.9724       117.212       189.524       104.18        29.2549        24     
produce:gaushit:GausHitFinder                     1.19286       2.11941       3.66955       1.96773      0.590494        24     
produce:nhitsfilter:NumberOfHitsFilter          0.000469741   0.000656141   0.000988958   0.00061779    0.000144728      24     
produce:reco3d:SpacePointSolver                   9.10453       16.5522       31.8636       15.3156       5.91441        24     
produce:hitpdune:DisambigFromSpacePoints          0.19144      0.475903       2.08336      0.338865      0.400143        24     
produce:pandora:StandardPandora                   20.0292       127.116       1604.38       42.7572       313.005        24     
produce:pandoraTrack:LArPandoraTrackCreation      1.27297       4.10255       10.5373       3.69489       2.31949        24     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0339009     0.0600806      0.10315      0.0577269     0.0177266       24     
[art]:TriggerResults:TriggerResultInserter      4.1611e-05    6.47463e-05   0.000122301   6.47575e-05   1.52323e-05      24     
end_path:out1:RootOutput                        1.0279e-05    1.69339e-05    3.684e-05    1.61785e-05   5.41709e-06      24     
end_path:out1:RootOutput(write)                   5.48106       6.39064       8.65761       6.1645       0.683508        24     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9018.09 MB
  Peak resident set size usage (VmHWM): 6994.08 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103452_keepup.root
\tHists: np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103453_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103452_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103452_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762166094.0,
    "core.end_time": 1762166094.0,
    "core.events": [
      278938,
      278942,
      278946,
      278950,
      278954,
      278958,
      278962,
      278966,
      278970,
      278974,
      278978,
      278982,
      278986,
      278990,
      278994,
      278998,
      279002,
      279006,
      279010,
      279014,
      279018,
      279022,
      279026,
      279030
    ],
    "core.event_count": 24,
    "core.first_event_number": 278938,
    "core.last_event_number": 279030,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014_reco_stage1_20251103T103453_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762166094.0,
    "core.end_time": 1762166094.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3021_df-s04-d2_dw_0_20251103T062014.hdf5"
    }
  ]
}
justIN time: 2025-11-03 18:44:22 UTC       justIN version: 01.05.01