justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 44835.55@dunegpschedd01.fnal.gov

Jobsub ID44835.55@dunegpschedd01.fnal.gov
Workflow ID2495
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-11 09:24:30
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2025-09-11 12:05:49
From worker nodeHostnameb9p10p3352.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-11 11:10:40
Input filesvd-protodune:np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905.hdf5
Outputting started2025-09-11 12:02:09
Output filesvd-protodune-det-reco:np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup.root
Finished2025-09-11 12:05:49
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

window for channel 7474
wclsFrameSaver: no samples within desired window for channel 7635
wclsFrameSaver: no samples within desired window for channel 7663
wclsFrameSaver: no samples within desired window for channel 7665
wclsFrameSaver: no samples within desired window for channel 8055
wclsFrameSaver: no samples within desired window for channel 8055
wclsFrameSaver: no samples within desired window for channel 8056
wclsFrameSaver: no samples within desired window for channel 8057
wclsFrameSaver: no samples within desired window for channel 8057
wclsFrameSaver: no samples within desired window for channel 8821
wclsFrameSaver: no samples within desired window for channel 8824
wclsFrameSaver: no samples within desired window for channel 8825
wclsFrameSaver: no samples within desired window for channel 8825
wclsFrameSaver: no samples within desired window for channel 8826
wclsFrameSaver: no samples within desired window for channel 8826
wclsFrameSaver: no samples within desired window for channel 8827
wclsFrameSaver: no samples within desired window for channel 9377
wclsFrameSaver: no samples within desired window for channel 9760
wclsFrameSaver: no samples within desired window for channel 9909
wclsFrameSaver: no samples within desired window for channel 9920
wclsFrameSaver: no samples within desired window for channel 9921
wclsFrameSaver: no samples within desired window for channel 9922
wclsFrameSaver: no samples within desired window for channel 10217
wclsFrameSaver: no samples within desired window for channel 10374
wclsFrameSaver: no samples within desired window for channel 10389
wclsFrameSaver: no samples within desired window for channel 10482
wclsFrameSaver: no samples within desired window for channel 10556
wclsFrameSaver: no samples within desired window for channel 10695
wclsFrameSaver: no samples within desired window for channel 10759
wclsFrameSaver: no samples within desired window for channel 10784
wclsFrameSaver: no samples within desired window for channel 10785
wclsFrameSaver: no samples within desired window for channel 10810
wclsFrameSaver: no samples within desired window for channel 10893
wclsFrameSaver: no samples within desired window for channel 10945
wclsFrameSaver: no samples within desired window for channel 11198
wclsFrameSaver: no samples within desired window for channel 11982
wclsFrameSaver: no samples within desired window for channel 11982
wclsFrameSaver: no samples within desired window for channel 11982
wclsFrameSaver: no samples within desired window for channel 11982
FrameSaver: q=8.43816e+06 n=998575 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 633 XUs and 861 XVs -> 40 XUVs
C:0 T:1 5345 XUs and 13193 XVs -> 706 XUVs
C:0 T:2 494 XUs and 503 XVs -> 14 XUVs
C:0 T:3 1736 XUs and 3852 XVs -> 164 XUVs
C:0 T:4 1054 XUs and 1326 XVs -> 102 XUVs
C:0 T:5 2490 XUs and 2666 XVs -> 113 XUVs
C:0 T:6 191 XUs and 260 XVs -> 5 XUVs
C:0 T:7 199 XUs and 377 XVs -> 12 XUVs
C:0 T:8 16094 XUs and 29161 XVs -> 5915 XUVs
C:0 T:9 2566 XUs and 2760 XVs -> 274 XUVs
C:0 T:10 3307 XUs and 2871 XVs -> 144 XUVs
C:0 T:11 1825 XUs and 2165 XVs -> 105 XUVs
C:0 T:12 1625 XUs and 1824 XVs -> 125 XUVs
C:0 T:13 439 XUs and 589 XVs -> 31 XUVs
C:0 T:14 993 XUs and 1584 XVs -> 225 XUVs
C:0 T:15 1439 XUs and 1935 XVs -> 89 XUVs
8064 XUVs total
1812 collection wire objects
8064 potential space points
Neighbour search...
1230004 tests to find 492164 neighbours
Iterating with no regularization...
Begin: 6.78414e+08
0 6.05775e+08
1 6.00869e+08
2 6.00537e+08
Now with regularization...
Begin: 5.91373e+08
0 5.91219e+08
11-Sep-2025 14:01:43 CEST  Closed output file "np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        116.329       168.495       252.229       163.405       36.6617        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       3.794e-05    7.80008e-05   0.000162171   6.60055e-05   3.26097e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               4.84381       5.85379       6.70934       5.85277      0.568693        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.000808657   0.00232315     0.0104211     0.0012153    0.00258101       18     
produce:pdvddaphne:DAPHNEReaderPDVD               2.84701       3.63551       4.40057       3.69186      0.513606        18     
produce:ophit:OpHitFinder                        0.0338342     0.0445799     0.0615671     0.0435479    0.00760234       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00746719     0.0125634     0.0204778     0.0121863    0.00389546       18     
produce:wclsdatavd:WireCellToolkit                51.5977       61.3974       71.8092       61.5995       6.44167        18     
produce:gaushit:GausHitFinder                    0.993305       1.25765       1.88801       1.24338       0.21265        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000222922   0.000317523   0.000491214   0.000306348   6.28671e-05      18     
produce:reco3d:SpacePointSolver                   12.9648       16.8054       25.5773       15.0485       3.75795        18     
produce:hitpdune:DisambigFromSpacePoints         0.178546      0.279489      0.469534      0.274591      0.0730601       18     
produce:pandora:StandardPandora                   34.9715       73.331        137.023       68.2655       28.4148        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.590446       1.20163       2.40437       1.19142      0.416473        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0180373     0.0293903     0.0652821     0.0275011    0.00971752       18     
[art]:TriggerResults:TriggerResultInserter       1.666e-05    2.39123e-05    6.358e-05     1.987e-05    1.06351e-05      18     
end_path:out1:RootOutput                         4.36e-06     8.44283e-06   2.9071e-05     7.605e-06    5.21614e-06      18     
end_path:out1:RootOutput(write)                   4.05992       4.61399       5.32712       4.55993      0.394813        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5226.42 MB
  Peak resident set size usage (VmHWM): 3265.25 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup.root
\tHists: np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757592104.0,
    "core.end_time": 1757592104.0,
    "core.events": [
      30601,
      30621,
      30641,
      30661,
      30681,
      30701,
      30721,
      30741,
      30761,
      30781,
      30801,
      30821,
      30841,
      30861,
      30881,
      30901,
      30921,
      30941
    ],
    "core.event_count": 18,
    "core.first_event_number": 30601,
    "core.last_event_number": 30941,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39273
    ],
    "core.runs_subruns": [
      3927300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905_reco_stage1_20250911T120143_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757592104.0,
    "core.end_time": 1757592104.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39273
    ],
    "core.runs_subruns": [
      3927300001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039273_0085_df-s01-d3_dw_0_20250831T104905.hdf5"
    }
  ]
}
justIN time: 2025-09-19 07:33:20 UTC       justIN version: 01.05.00