justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 37302.158@dunegpschedd02.fnal.gov

Jobsub ID37302.158@dunegpschedd02.fnal.gov
Workflow ID2380
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-10 02:39:48
SiteUK_Glasgow
EntryCLAS12_T3_UK_ScotGrid_GLA_ce04_scitok
Last heartbeat2025-09-10 04:38:24
From worker nodeHostnamewn-d22-033.beowulf.cluster
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-10 02:40:55
Input filesvd-protodune:np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118.hdf5
JobscriptExit code0
Real time1h (6607s)
CPU time33m (1996s = 30%)
Max RSS bytes2848989184 (2717 MiB)
Outputting started2025-09-10 04:31:02
Output filesvd-protodune-det-reco:np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup.root
Finished2025-09-10 04:38:24
Saved logsjustin-logs:37302.158-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nnel 7022
wclsFrameSaver: no samples within desired window for channel 7022
wclsFrameSaver: no samples within desired window for channel 7023
wclsFrameSaver: no samples within desired window for channel 7023
wclsFrameSaver: no samples within desired window for channel 7097
wclsFrameSaver: no samples within desired window for channel 7121
wclsFrameSaver: no samples within desired window for channel 7265
wclsFrameSaver: no samples within desired window for channel 7411
wclsFrameSaver: no samples within desired window for channel 7533
wclsFrameSaver: no samples within desired window for channel 7701
wclsFrameSaver: no samples within desired window for channel 7704
wclsFrameSaver: no samples within desired window for channel 7705
wclsFrameSaver: no samples within desired window for channel 7706
wclsFrameSaver: no samples within desired window for channel 7707
wclsFrameSaver: no samples within desired window for channel 7709
wclsFrameSaver: no samples within desired window for channel 7757
wclsFrameSaver: no samples within desired window for channel 7884
wclsFrameSaver: no samples within desired window for channel 7935
wclsFrameSaver: no samples within desired window for channel 7935
wclsFrameSaver: no samples within desired window for channel 7995
wclsFrameSaver: no samples within desired window for channel 8017
wclsFrameSaver: no samples within desired window for channel 8792
wclsFrameSaver: no samples within desired window for channel 8792
wclsFrameSaver: no samples within desired window for channel 8793
wclsFrameSaver: no samples within desired window for channel 9068
wclsFrameSaver: no samples within desired window for channel 9199
wclsFrameSaver: no samples within desired window for channel 9199
wclsFrameSaver: no samples within desired window for channel 9199
wclsFrameSaver: no samples within desired window for channel 9622
wclsFrameSaver: no samples within desired window for channel 9931
wclsFrameSaver: no samples within desired window for channel 10037
wclsFrameSaver: no samples within desired window for channel 10204
wclsFrameSaver: no samples within desired window for channel 10263
wclsFrameSaver: no samples within desired window for channel 10542
wclsFrameSaver: no samples within desired window for channel 10561
wclsFrameSaver: no samples within desired window for channel 10595
wclsFrameSaver: no samples within desired window for channel 10664
wclsFrameSaver: no samples within desired window for channel 10842
wclsFrameSaver: no samples within desired window for channel 11069
wclsFrameSaver: no samples within desired window for channel 12020
FrameSaver: q=6.42368e+06 n=820374 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 725 XUs and 609 XVs -> 15 XUVs
C:0 T:1 696 XUs and 630 XVs -> 42 XUVs
C:0 T:2 1996 XUs and 2252 XVs -> 88 XUVs
C:0 T:3 271 XUs and 350 XVs -> 13 XUVs
C:0 T:4 124 XUs and 178 XVs -> 7 XUVs
C:0 T:5 937 XUs and 1344 XVs -> 47 XUVs
C:0 T:6 482 XUs and 542 XVs -> 17 XUVs
C:0 T:7 614 XUs and 631 XVs -> 20 XUVs
C:0 T:8 608 XUs and 621 XVs -> 35 XUVs
C:0 T:9 353 XUs and 537 XVs -> 29 XUVs
C:0 T:10 1218 XUs and 1401 XVs -> 67 XUVs
C:0 T:11 1471 XUs and 2046 XVs -> 92 XUVs
C:0 T:12 1175 XUs and 983 XVs -> 65 XUVs
C:0 T:13 388 XUs and 332 XVs -> 29 XUVs
C:0 T:14 1773 XUs and 2100 XVs -> 112 XUVs
C:0 T:15 2403 XUs and 2425 XVs -> 135 XUVs
813 XUVs total
721 collection wire objects
813 potential space points
Neighbour search...
3539 tests to find 1678 neighbours
Iterating with no regularization...
Begin: 2.13509e+08
0 2.10269e+08
1 2.10231e+08
Now with regularization...
Begin: 2.07602e+08
0 2.07598e+08
10-Sep-2025 05:30:54 BST  Closed output file "np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        102.142       363.966       4381.53       127.08        974.598        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.0053e-05    8.90417e-05   0.000267342   6.90395e-05   4.77087e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               17.0966       255.554       4278.13       18.834        975.617        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.478481      0.483339      0.493475      0.482325     0.00333867       18     
produce:pdvddaphne:DAPHNEReaderPDVD               2.15153       2.39196       2.73824       2.37509      0.160842        18     
produce:ophit:OpHitFinder                        0.0261732     0.0336919     0.041774      0.033343     0.00425376       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0041055     0.0067463     0.0098456    0.00623521    0.00191022       18     
produce:wclsdatavd:WireCellToolkit                48.0631       51.2088       57.1072       50.7283       2.29226        18     
produce:gaushit:GausHitFinder                    0.700708       0.97621       1.35864      0.985039      0.201396        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000189716   0.000315033   0.000620125   0.000283628   0.000109107      18     
produce:reco3d:SpacePointSolver                   7.17427       11.3338       16.8092       10.2607       3.12555        18     
produce:hitpdune:DisambigFromSpacePoints         0.0848986     0.137807      0.212352       0.12987      0.0386868       18     
produce:pandora:StandardPandora                   17.9599       36.7581       61.7277       34.742        15.8561        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.471973      0.833873       1.97622      0.643991      0.415951        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0175123     0.0237979     0.0371853     0.0226643    0.00540207       18     
[art]:TriggerResults:TriggerResultInserter      1.6712e-05    3.48103e-05    9.625e-05    3.3613e-05    1.65986e-05      18     
end_path:out1:RootOutput                         4.649e-06    8.0635e-06    3.5487e-05     6.231e-06    6.76638e-06      18     
end_path:out1:RootOutput(write)                   3.85004       4.17929       4.50949       4.17647      0.182422        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4839.22 MB
  Peak resident set size usage (VmHWM): 2848.99 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup.root
\tHists: np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757478655.0,
    "core.end_time": 1757478655.0,
    "core.events": [
      40526,
      40530,
      40534,
      40538,
      40542,
      40546,
      40550,
      40554,
      40558,
      40562,
      40566,
      40570,
      40574,
      40578,
      40582,
      40586,
      40590,
      40594
    ],
    "core.event_count": 18,
    "core.first_event_number": 40526,
    "core.last_event_number": 40594,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39350
    ],
    "core.runs_subruns": [
      3935000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118_reco_stage1_20250910T043054_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757478655.0,
    "core.end_time": 1757478655.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39350
    ],
    "core.runs_subruns": [
      3935000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039350_0554_df-s04-d2_dw_0_20250910T021118.hdf5"
    }
  ]
}
justIN time: 2025-09-19 02:18:43 UTC       justIN version: 01.05.00