justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 37938.77@dunegpschedd02.fnal.gov

Jobsub ID37938.77@dunegpschedd02.fnal.gov
Workflow ID2482
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-11 00:42:01
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2025-09-11 03:08:26
From worker nodeHostnameb9p27p8513.cern.ch
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-11 01:48:57
Input filesvd-protodune:np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905.hdf5
Outputting started2025-09-11 03:01:30
Output filesvd-protodune-det-reco:np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup.root
Finished2025-09-11 03:08:26
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ired window for channel 10730
wclsFrameSaver: no samples within desired window for channel 10731
wclsFrameSaver: no samples within desired window for channel 10731
wclsFrameSaver: no samples within desired window for channel 10732
wclsFrameSaver: no samples within desired window for channel 10732
wclsFrameSaver: no samples within desired window for channel 10733
wclsFrameSaver: no samples within desired window for channel 10734
wclsFrameSaver: no samples within desired window for channel 10852
wclsFrameSaver: no samples within desired window for channel 11125
wclsFrameSaver: no samples within desired window for channel 11149
wclsFrameSaver: no samples within desired window for channel 11212
wclsFrameSaver: no samples within desired window for channel 11213
wclsFrameSaver: no samples within desired window for channel 11219
wclsFrameSaver: no samples within desired window for channel 11220
wclsFrameSaver: no samples within desired window for channel 11220
wclsFrameSaver: no samples within desired window for channel 11221
wclsFrameSaver: no samples within desired window for channel 11221
wclsFrameSaver: no samples within desired window for channel 11222
wclsFrameSaver: no samples within desired window for channel 11223
wclsFrameSaver: no samples within desired window for channel 11223
wclsFrameSaver: no samples within desired window for channel 11224
wclsFrameSaver: no samples within desired window for channel 11507
wclsFrameSaver: no samples within desired window for channel 11508
wclsFrameSaver: no samples within desired window for channel 11509
wclsFrameSaver: no samples within desired window for channel 11510
wclsFrameSaver: no samples within desired window for channel 11511
wclsFrameSaver: no samples within desired window for channel 11781
wclsFrameSaver: no samples within desired window for channel 11781
wclsFrameSaver: no samples within desired window for channel 11782
wclsFrameSaver: no samples within desired window for channel 11782
wclsFrameSaver: no samples within desired window for channel 11782
wclsFrameSaver: no samples within desired window for channel 11783
wclsFrameSaver: no samples within desired window for channel 11783
wclsFrameSaver: no samples within desired window for channel 11783
wclsFrameSaver: no samples within desired window for channel 11784
wclsFrameSaver: no samples within desired window for channel 11785
wclsFrameSaver: no samples within desired window for channel 11787
wclsFrameSaver: no samples within desired window for channel 11788
FrameSaver: q=1.12356e+07 n=1134498 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 638 XUs and 676 XVs -> 38 XUVs
C:0 T:1 529 XUs and 451 XVs -> 36 XUVs
C:0 T:2 552 XUs and 520 XVs -> 30 XUVs
C:0 T:3 105 XUs and 152 XVs -> 9 XUVs
C:0 T:4 487 XUs and 655 XVs -> 36 XUVs
C:0 T:5 313 XUs and 233 XVs -> 8 XUVs
C:0 T:6 340 XUs and 258 XVs -> 16 XUVs
C:0 T:7 584 XUs and 713 XVs -> 17 XUVs
C:0 T:8 12891 XUs and 38129 XVs -> 4941 XUVs
C:0 T:9 19215 XUs and 50518 XVs -> 5359 XUVs
C:0 T:10 16039 XUs and 25204 XVs -> 4672 XUVs
C:0 T:11 11822 XUs and 20656 XVs -> 2915 XUVs
C:0 T:12 11257 XUs and 16379 XVs -> 2436 XUVs
C:0 T:13 4680 XUs and 6242 XVs -> 907 XUVs
C:0 T:14 3329 XUs and 13815 XVs -> 997 XUVs
C:0 T:15 5495 XUs and 6470 XVs -> 571 XUVs
22988 XUVs total
3033 collection wire objects
22988 potential space points
Neighbour search...
2795676 tests to find 1212758 neighbours
Iterating with no regularization...
Begin: 7.50263e+08
0 6.31282e+08
1 6.23633e+08
2 6.22162e+08
3 6.21612e+08
Now with regularization...
Begin: 5.98507e+08
0 5.98017e+08
11-Sep-2025 05:01:18 CEST  Closed output file "np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        171.904       238.553       377.264       213.328       58.0798        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.6791e-05    0.000138106   0.000522328   8.46265e-05   0.000115134      18     
produce:tpcrawdecoder:PDVDTPCReader               6.80659       8.28003       19.807        7.44863       2.84162        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00129156    0.00420171     0.0232032    0.00170658    0.00579644       18     
produce:pdvddaphne:DAPHNEReaderPDVD               4.66493       5.1409        6.07931       5.0756       0.378826        18     
produce:ophit:OpHitFinder                        0.0374298     0.0513781     0.0586558     0.0518322    0.00624344       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0110673     0.0164055     0.0236618     0.0158274    0.00367528       18     
produce:wclsdatavd:WireCellToolkit                72.1074       81.7051       97.7604       79.1221       7.71416        18     
produce:gaushit:GausHitFinder                     1.12713       1.68422       2.67498       1.6373       0.460033        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000378036   0.000512581   0.000842312   0.000491778   0.000113973      18     
produce:reco3d:SpacePointSolver                   15.7409       21.7302       36.2314       19.2056       5.74566        18     
produce:hitpdune:DisambigFromSpacePoints         0.214449      0.390871      0.684269      0.360681      0.138026        18     
produce:pandora:StandardPandora                   60.331        111.475       202.596       89.7532       47.0932        18     
produce:pandoraTrack:LArPandoraTrackCreation      1.27128       2.03636        4.343        1.85143      0.700139        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.028592      0.0504529     0.0833007     0.0501549     0.014737        18     
[art]:TriggerResults:TriggerResultInserter      1.9281e-05    3.3266e-05    7.8151e-05    2.7965e-05    1.56918e-05      18     
end_path:out1:RootOutput                         6.23e-06     9.97567e-06   2.9781e-05      7.7e-06     6.94395e-06      18     
end_path:out1:RootOutput(write)                   5.1859        5.92188       7.78074       5.87574      0.625784        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5106.16 MB
  Peak resident set size usage (VmHWM): 3202.4 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup.root
\tHists: np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757559678.0,
    "core.end_time": 1757559678.0,
    "core.events": [
      370802,
      370822,
      370842,
      370862,
      370882,
      370902,
      370922,
      370942,
      370962,
      370982,
      371002,
      371022,
      371042,
      371062,
      371082,
      371102,
      371122,
      371142
    ],
    "core.event_count": 18,
    "core.first_event_number": 370802,
    "core.last_event_number": 371142,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905_reco_stage1_20250911T030118_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757559678.0,
    "core.end_time": 1757559678.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1030_df-s03-d3_dw_0_20250830T212905.hdf5"
    }
  ]
}
justIN time: 2025-09-19 13:38:21 UTC       justIN version: 01.05.00