justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40870.92@dunegpschedd02.fnal.gov

Jobsub ID40870.92@dunegpschedd02.fnal.gov
Workflow ID2740
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-17 08:51:00
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce02
Last heartbeat2025-09-17 10:11:02
From worker nodeHostnamedune001-8170246.0-lcg2713.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 09:17:00
Input filesvd-protodune:np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237.hdf5
JobscriptExit code0
Real time48m (2925s)
CPU time42m (2564s = 87%)
Max RSS bytes3021086720 (2881 MiB)
Outputting started2025-09-17 10:05:45
Output filesvd-protodune-det-reco:np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup.root
Finished2025-09-17 10:11:02
Saved logsjustin-logs:40870.92-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

within desired window for channel 9770
wclsFrameSaver: no samples within desired window for channel 9772
wclsFrameSaver: no samples within desired window for channel 9773
wclsFrameSaver: no samples within desired window for channel 10044
wclsFrameSaver: no samples within desired window for channel 10045
wclsFrameSaver: no samples within desired window for channel 10045
wclsFrameSaver: no samples within desired window for channel 10047
wclsFrameSaver: no samples within desired window for channel 10047
wclsFrameSaver: no samples within desired window for channel 10048
wclsFrameSaver: no samples within desired window for channel 10184
wclsFrameSaver: no samples within desired window for channel 10207
wclsFrameSaver: no samples within desired window for channel 10441
wclsFrameSaver: no samples within desired window for channel 10482
wclsFrameSaver: no samples within desired window for channel 10508
wclsFrameSaver: no samples within desired window for channel 10573
wclsFrameSaver: no samples within desired window for channel 10586
wclsFrameSaver: no samples within desired window for channel 10775
wclsFrameSaver: no samples within desired window for channel 10776
wclsFrameSaver: no samples within desired window for channel 10791
wclsFrameSaver: no samples within desired window for channel 11010
wclsFrameSaver: no samples within desired window for channel 11011
wclsFrameSaver: no samples within desired window for channel 11011
wclsFrameSaver: no samples within desired window for channel 11072
wclsFrameSaver: no samples within desired window for channel 11108
wclsFrameSaver: no samples within desired window for channel 11368
wclsFrameSaver: no samples within desired window for channel 11368
wclsFrameSaver: no samples within desired window for channel 11504
wclsFrameSaver: no samples within desired window for channel 11505
wclsFrameSaver: no samples within desired window for channel 11553
wclsFrameSaver: no samples within desired window for channel 11771
wclsFrameSaver: no samples within desired window for channel 11772
wclsFrameSaver: no samples within desired window for channel 11774
wclsFrameSaver: no samples within desired window for channel 11775
wclsFrameSaver: no samples within desired window for channel 11775
wclsFrameSaver: no samples within desired window for channel 12119
wclsFrameSaver: no samples within desired window for channel 12120
wclsFrameSaver: no samples within desired window for channel 12124
wclsFrameSaver: no samples within desired window for channel 12125
FrameSaver: q=1.12118e+07 n=1193654 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 685 XUs and 778 XVs -> 28 XUVs
C:0 T:1 240 XUs and 305 XVs -> 24 XUVs
C:0 T:2 525 XUs and 442 XVs -> 43 XUVs
C:0 T:3 970 XUs and 1178 XVs -> 31 XUVs
C:0 T:4 554 XUs and 643 XVs -> 16 XUVs
C:0 T:5 928 XUs and 885 XVs -> 31 XUVs
C:0 T:6 552 XUs and 642 XVs -> 10 XUVs
C:0 T:7 999 XUs and 1502 XVs -> 41 XUVs
C:0 T:8 18779 XUs and 36980 XVs -> 4427 XUVs
C:0 T:9 11373 XUs and 19625 XVs -> 3492 XUVs
C:0 T:10 11521 XUs and 25858 XVs -> 5412 XUVs
C:0 T:11 2892 XUs and 3993 XVs -> 814 XUVs
C:0 T:12 7638 XUs and 12016 XVs -> 1513 XUVs
C:0 T:13 9975 XUs and 11285 XVs -> 1570 XUVs
C:0 T:14 1342 XUs and 1567 XVs -> 285 XUVs
C:0 T:15 1065 XUs and 1336 XVs -> 94 XUVs
17831 XUVs total
2148 collection wire objects
17831 potential space points
Neighbour search...
2884027 tests to find 1178186 neighbours
Iterating with no regularization...
Begin: 1.03678e+09
0 8.79133e+08
1 8.63209e+08
2 8.61805e+08
3 8.61325e+08
Now with regularization...
Begin: 8.45988e+08
0 8.45591e+08
17-Sep-2025 10:05:35 UTC  Closed output file "np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                          103         150.025       222.402       145.921        33.3          19     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.4882e-05    9.98668e-05   0.000291248    8.606e-05    4.94918e-05      19     
produce:tpcrawdecoder:PDVDTPCReader               16.2001       17.5767       33.3198       16.7531       3.72588        19     
produce:triggerrawdecoder:PDVDTriggerReader4     0.388493      0.398791      0.433002      0.394668      0.0100863       19     
produce:pdvddaphne:DAPHNEReaderPDVD             0.000196896   0.000335626   0.000562065   0.000330778   6.74465e-05      19     
produce:ophit:OpHitFinder                       0.000136405   0.000238846   0.000747524   0.000178278   0.000146192      19     
produce:opflash:OpFlashFinderVerticalDrift       4.665e-05    7.82323e-05   0.000269946   6.8383e-05    4.82289e-05      19     
produce:wclsdatavd:WireCellToolkit                50.4803       61.7522       79.6581       58.2238       8.81094        19     
produce:gaushit:GausHitFinder                    0.792763       1.26892       2.07477       1.15787      0.350351        19     
produce:nhitsfilter:NumberOfHitsFilter          0.000210276   0.000396346   0.000776237   0.000334533   0.000165389      19     
produce:reco3d:SpacePointSolver                   8.76826       13.5407       21.8067       12.7558       3.95225        19     
produce:hitpdune:DisambigFromSpacePoints         0.114749      0.221077      0.419252      0.194921      0.0826403       19     
produce:pandora:StandardPandora                   19.415        49.6112       99.7604       44.1521       22.6684        19     
produce:pandoraTrack:LArPandoraTrackCreation     0.560779       1.02921       1.6603       0.956418      0.333042        19     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0207488     0.0296333     0.0494618     0.0267111    0.00711012       19     
[art]:TriggerResults:TriggerResultInserter       1.944e-05    4.08008e-05   0.000115464   3.2629e-05    2.32575e-05      19     
end_path:out1:RootOutput                         5.509e-06    1.09503e-05   3.8448e-05     9.555e-06    7.05739e-06      19     
end_path:out1:RootOutput(write)                   3.72987       4.53249       5.30897       4.56063      0.428922        19     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5096.46 MB
  Peak resident set size usage (VmHWM): 3021.09 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup.root
\tHists: np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758103536.0,
    "core.end_time": 1758103536.0,
    "core.events": [
      40287,
      40291,
      40295,
      40299,
      40303,
      40307,
      40311,
      40315,
      40319,
      40323,
      40327,
      40331,
      40335,
      40339,
      40343,
      40347,
      40351,
      40355,
      40359
    ],
    "core.event_count": 19,
    "core.first_event_number": 40287,
    "core.last_event_number": 40359,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39388
    ],
    "core.runs_subruns": [
      3938800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237_reco_stage1_20250917T100535_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1758103536.0,
    "core.end_time": 1758103536.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39388
    ],
    "core.runs_subruns": [
      3938800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039388_0523_df-s05-d3_dw_0_20250917T001237.hdf5"
    }
  ]
}
justIN time: 2025-09-19 07:38:38 UTC       justIN version: 01.05.00