justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 44603.191@dunegpschedd01.fnal.gov

Jobsub ID44603.191@dunegpschedd01.fnal.gov
Workflow ID2484
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-10 22:08:55
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2025-09-11 03:06:43
From worker nodeHostnameb9p12p9960.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-11 01:33:11
Input filesvd-protodune:np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837.hdf5
Outputting started2025-09-11 02:48:28
Output filesvd-protodune-det-reco:np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024818_keepup.root
Finished2025-09-11 03:06:43
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ired window for channel 8056
wclsFrameSaver: no samples within desired window for channel 8056
wclsFrameSaver: no samples within desired window for channel 8057
wclsFrameSaver: no samples within desired window for channel 8057
wclsFrameSaver: no samples within desired window for channel 8433
wclsFrameSaver: no samples within desired window for channel 8833
wclsFrameSaver: no samples within desired window for channel 8834
wclsFrameSaver: no samples within desired window for channel 8834
wclsFrameSaver: no samples within desired window for channel 8834
wclsFrameSaver: no samples within desired window for channel 8835
wclsFrameSaver: no samples within desired window for channel 8836
wclsFrameSaver: no samples within desired window for channel 9336
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9687
wclsFrameSaver: no samples within desired window for channel 9790
wclsFrameSaver: no samples within desired window for channel 9809
wclsFrameSaver: no samples within desired window for channel 9848
wclsFrameSaver: no samples within desired window for channel 9848
wclsFrameSaver: no samples within desired window for channel 9961
wclsFrameSaver: no samples within desired window for channel 10183
wclsFrameSaver: no samples within desired window for channel 10225
wclsFrameSaver: no samples within desired window for channel 10225
wclsFrameSaver: no samples within desired window for channel 10225
wclsFrameSaver: no samples within desired window for channel 10321
wclsFrameSaver: no samples within desired window for channel 10749
wclsFrameSaver: no samples within desired window for channel 10787
wclsFrameSaver: no samples within desired window for channel 10787
wclsFrameSaver: no samples within desired window for channel 11359
wclsFrameSaver: no samples within desired window for channel 11359
wclsFrameSaver: no samples within desired window for channel 11360
wclsFrameSaver: no samples within desired window for channel 11360
wclsFrameSaver: no samples within desired window for channel 11867
wclsFrameSaver: no samples within desired window for channel 11867
wclsFrameSaver: no samples within desired window for channel 11869
wclsFrameSaver: no samples within desired window for channel 11869
FrameSaver: q=1.19118e+07 n=1305826 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1201 XUs and 1251 XVs -> 50 XUVs
C:0 T:1 1467 XUs and 2170 XVs -> 89 XUVs
C:0 T:2 351 XUs and 279 XVs -> 21 XUVs
C:0 T:3 607 XUs and 486 XVs -> 19 XUVs
C:0 T:4 2382 XUs and 3950 XVs -> 135 XUVs
C:0 T:5 555 XUs and 648 XVs -> 29 XUVs
C:0 T:6 815 XUs and 1106 XVs -> 64 XUVs
C:0 T:7 1015 XUs and 1499 XVs -> 79 XUVs
C:0 T:8 4442 XUs and 4776 XVs -> 1033 XUVs
C:0 T:9 4362 XUs and 5980 XVs -> 816 XUVs
C:0 T:10 4407 XUs and 3899 XVs -> 594 XUVs
C:0 T:11 8566 XUs and 14395 XVs -> 1579 XUVs
C:0 T:12 23283 XUs and 26653 XVs -> 2247 XUVs
C:0 T:13 3778 XUs and 4459 XVs -> 297 XUVs
C:0 T:14 2550 XUs and 2521 XVs -> 150 XUVs
C:0 T:15 3288 XUs and 3674 XVs -> 275 XUVs
7477 XUVs total
2755 collection wire objects
7477 potential space points
Neighbour search...
525359 tests to find 252960 neighbours
Iterating with no regularization...
Begin: 9.62252e+08
0 8.80215e+08
1 8.75736e+08
2 8.75152e+08
Now with regularization...
Begin: 8.56409e+08
0 8.5616e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
11-Sep-2025 04:48:18 CEST  Closed output file "np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024818_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        123.073       248.523       1307.43       172.665       261.91         18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      5.4761e-05    7.90962e-05   0.000170132   7.2201e-05    2.67057e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               6.25524       7.37929       12.467        7.1066        1.30574        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.000785592   0.00623339     0.0863552    0.00121217     0.0194673       18     
produce:pdvddaphne:DAPHNEReaderPDVD               2.83018       3.86299       4.93963       3.71346      0.580541        18     
produce:ophit:OpHitFinder                        0.0362289     0.0448443     0.053156      0.0453097    0.00521458       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00727546     0.0107901     0.0164939     0.0100787    0.00269816       18     
produce:wclsdatavd:WireCellToolkit                52.5359       62.1306       74.5798       59.858        6.80448        18     
produce:gaushit:GausHitFinder                    0.981652       1.31135       1.98554       1.25304      0.259176        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000261304   0.000348316   0.000479487   0.000341155   6.71064e-05      18     
produce:reco3d:SpacePointSolver                   14.2183       19.844        38.5211       18.2374       5.63125        18     
produce:hitpdune:DisambigFromSpacePoints         0.176605      0.336717      0.687527      0.319828      0.131084        18     
produce:pandora:StandardPandora                   31.6982       147.447       1170.33       66.4677       252.131        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.788721       1.47719       3.58392       1.21124      0.666454        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0217704     0.0283458     0.0376051     0.0277806    0.00387789       18     
[art]:TriggerResults:TriggerResultInserter       1.852e-05    2.83526e-05   8.3891e-05    2.35105e-05   1.50453e-05      18     
end_path:out1:RootOutput                         3.63e-06     7.76444e-06    3.306e-05     5.715e-06    7.00869e-06      18     
end_path:out1:RootOutput(write)                   4.0974        4.60098       6.72127       4.41972      0.580411        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5478.8 MB
  Peak resident set size usage (VmHWM): 3512.18 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024818_keepup.root
\tHists: np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024819_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024818_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024818_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757558899.0,
    "core.end_time": 1757558899.0,
    "core.events": [
      579247,
      579267,
      579287,
      579307,
      579327,
      579347,
      579367,
      579387,
      579407,
      579427,
      579447,
      579467,
      579487,
      579507,
      579527,
      579547,
      579567,
      579587
    ],
    "core.event_count": 18,
    "core.first_event_number": 579247,
    "core.last_event_number": 579587,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837_reco_stage1_20250911T024819_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757558899.0,
    "core.end_time": 1757558899.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1609_df-s05-d0_dw_0_20250831T041837.hdf5"
    }
  ]
}
justIN time: 2025-09-19 19:08:18 UTC       justIN version: 01.05.00