justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242471.141@dunegpschedd01.fnal.gov

Jobsub ID242471.141@dunegpschedd01.fnal.gov
Workflow ID9538
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors2
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-11-03 17:53:07
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-03 19:11:14
From worker nodeHostnamewn1207090.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors2
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 17:54:29
Input filesvd-protodune:np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416.hdf5
JobscriptExit code0
Real time1h (4436s)
CPU time1h (4037s = 91%)
Max RSS bytes3084275712 (2941 MiB)
Outputting started2025-11-03 19:08:26
Output filesvd-protodune-det-reco:np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup.root
Finished2025-11-03 19:11:14
Saved logsjustin-logs:242471.141-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

call=45 
[19:06:54.527] D [sigproc ] <OmnibusSigProc:anode0sigproc0> EOS at call=45 anode=0 
[19:06:54.527] D [  glue  ] <FrameFanin:nfsp> EOS at call=45 with 8 
[19:06:54.527] D [  glue  ] frame sink sees EOS
[19:06:54.527] D [ pgraph ] <Pgrapher:> graph execution complete 
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.77 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.74 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 15.63 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.66 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 14.1 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.78 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 12.73 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::OmnibusSigProc : 11.76 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.66 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.41 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.28 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Aux::Resampler : 1.23 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Gen::FrameFanin : 0.03 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0.01 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::Gen::Retagger : 0.01 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:06:54.527] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:06:54.528] I [ timer  ] Timer: WireCell::Gen::DumpFrames : 0 sec
[19:06:54.528] I [ timer  ] Timer: WireCell::SigProc::ChannelSelector : 0 sec
[19:06:54.528] I [ timer  ] Timer: WireCell::Gen::FrameFanout : 0 sec
[19:06:54.528] I [ timer  ] Timer: wcls::RawFrameSource : 0 sec
[19:06:54.528] I [ timer  ] Timer: wcls::FrameSaver : 0 sec
[19:06:54.528] I [ timer  ] Timer: Total node execution : 118.81999999843538 sec
wclsFrameSaver saving cooked to 10000 ticks
wclsFrameSaver: saving 50738 traces tagged "gauss"
FrameSaver: q=1.20315e+07 n=1100966 tag=gauss
wclsFrameSaver: saving 61885 traces tagged "wiener"
FrameSaver: q=1.26328e+07 n=1060629 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 736 XUs and 914 XVs -> 33 XUVs
C:0 T:1 4170 XUs and 4412 XVs -> 179 XUVs
C:0 T:2 640 XUs and 620 XVs -> 16 XUVs
C:0 T:3 746 XUs and 687 XVs -> 28 XUVs
C:0 T:4 337 XUs and 666 XVs -> 22 XUVs
C:0 T:5 99 XUs and 114 XVs -> 0 XUVs
C:0 T:6 311 XUs and 418 XVs -> 6 XUVs
C:0 T:7 424 XUs and 393 XVs -> 21 XUVs
C:0 T:8 1641 XUs and 2180 XVs -> 76 XUVs
C:0 T:9 1509 XUs and 1513 XVs -> 65 XUVs
C:0 T:10 1964 XUs and 3580 XVs -> 91 XUVs
C:0 T:11 2063 XUs and 2319 XVs -> 93 XUVs
C:0 T:12 279 XUs and 335 XVs -> 22 XUVs
C:0 T:13 572 XUs and 919 XVs -> 73 XUVs
C:0 T:14 870 XUs and 1134 XVs -> 36 XUVs
C:0 T:15 1200 XUs and 1472 XVs -> 59 XUVs
820 XUVs total
706 collection wire objects
820 potential space points
Neighbour search...
5100 tests to find 2516 neighbours
Iterating with no regularization...
Begin: 1.22884e+11
0 1.22877e+11
Now with regularization...
Begin: 1.22872e+11
0 1.22872e+11
03-Nov-2025 19:08:11 GMT  Closed output file "np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        132.269       188.022       249.259       184.744       36.5977        23     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      9.3907e-05    0.000114224   0.000334991   0.00010049    4.8473e-05       23     
produce:tpcrawdecoder:PDVDTPCReader               15.6788       16.7673       19.3823       16.2304       1.08294        23     
produce:triggerrawdecoder:PDVDTriggerReader4     0.673625      0.695577      0.722953      0.700095      0.0132925       23     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000435051   0.000545155   0.00119728    0.000504832   0.000145478      23     
produce:ophit:OpHitFinder                       7.6585e-05    0.00013278    0.000933881   8.7225e-05    0.000172846      23     
produce:opflash:OpFlashFinderVerticalDrift      5.6987e-05    0.000102048   0.000547062   6.8239e-05    0.000102909      23     
produce:wclsdatavd:WireCellToolkit                78.8498       102.311       155.672       96.5672       19.9466        23     
produce:gaushit:GausHitFinder                    0.892995       1.66114       2.35508       1.69635      0.393371        23     
produce:nhitsfilter:NumberOfHitsFilter          0.000320374   0.000663479   0.00122051    0.000619158   0.000242246      23     
produce:reco3d:SpacePointSolver                   8.56928       15.2224       20.2596       15.9799       3.54089        23     
produce:hitpdune:DisambigFromSpacePoints         0.109033      0.253438      0.423894      0.251011      0.0940656       23     
produce:pandora:StandardPandora                   16.1088       42.555        102.033       35.2198       23.4082        23     
produce:pandoraTrack:LArPandoraTrackCreation     0.467506       1.68728       3.02635       1.60689      0.710013        23     
produce:pandoraGnocalo:GnocchiCalorimetry         0.02954      0.0515541     0.0773876     0.0501663     0.0130439       23     
[art]:TriggerResults:TriggerResultInserter      2.5628e-05    5.67698e-05   0.000132029   3.9685e-05    3.07164e-05      23     
end_path:out1:RootOutput                         4.658e-06    8.18065e-06   2.7793e-05     5.921e-06    5.61242e-06      23     
end_path:out1:RootOutput(write)                   5.36508       6.73314       9.17334       6.92306       0.99733        23     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5160.8 MB
  Peak resident set size usage (VmHWM): 3084.28 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup.root
\tHists: np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762196892.0,
    "core.end_time": 1762196892.0,
    "core.events": [
      345873,
      345877,
      345881,
      345885,
      345889,
      345893,
      345897,
      345901,
      345905,
      345909,
      345913,
      345917,
      345921,
      345925,
      345929,
      345933,
      345937,
      345941,
      345945,
      345949,
      345953,
      345957,
      345961
    ],
    "core.event_count": 23,
    "core.first_event_number": 345873,
    "core.last_event_number": 345961,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416_reco_stage1_20251103T190811_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_11_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1762196892.0,
    "core.end_time": 1762196892.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      40346
    ],
    "core.runs_subruns": [
      4034600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run040346_3745_df-s04-d1_dw_0_20251103T165416.hdf5"
    }
  ]
}
justIN time: 2025-11-05 07:11:58 UTC       justIN version: 01.05.01