justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 44833.34@dunegpschedd01.fnal.gov

Jobsub ID44833.34@dunegpschedd01.fnal.gov
Workflow ID2494
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-11 09:14:29
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce512
Last heartbeat2025-09-11 12:05:21
From worker nodeHostnameb9p04p2638.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-11 11:01:49
Input filesvd-protodune:np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043.hdf5
Outputting started2025-09-11 12:01:07
Output filesvd-protodune-det-reco:np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup.root
Finished2025-09-11 12:05:21
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

s within desired window for channel 11162
wclsFrameSaver: no samples within desired window for channel 11162
wclsFrameSaver: no samples within desired window for channel 11163
wclsFrameSaver: no samples within desired window for channel 11163
wclsFrameSaver: no samples within desired window for channel 11164
wclsFrameSaver: no samples within desired window for channel 11164
wclsFrameSaver: no samples within desired window for channel 11165
wclsFrameSaver: no samples within desired window for channel 11165
wclsFrameSaver: no samples within desired window for channel 11166
wclsFrameSaver: no samples within desired window for channel 11166
wclsFrameSaver: no samples within desired window for channel 11167
wclsFrameSaver: no samples within desired window for channel 11167
wclsFrameSaver: no samples within desired window for channel 11168
wclsFrameSaver: no samples within desired window for channel 11168
wclsFrameSaver: no samples within desired window for channel 11169
wclsFrameSaver: no samples within desired window for channel 11169
wclsFrameSaver: no samples within desired window for channel 11170
wclsFrameSaver: no samples within desired window for channel 11170
wclsFrameSaver: no samples within desired window for channel 11171
wclsFrameSaver: no samples within desired window for channel 11171
wclsFrameSaver: no samples within desired window for channel 11172
wclsFrameSaver: no samples within desired window for channel 11423
wclsFrameSaver: no samples within desired window for channel 11432
wclsFrameSaver: no samples within desired window for channel 11435
wclsFrameSaver: no samples within desired window for channel 12076
wclsFrameSaver: no samples within desired window for channel 12078
wclsFrameSaver: no samples within desired window for channel 12079
wclsFrameSaver: no samples within desired window for channel 12079
wclsFrameSaver: no samples within desired window for channel 12080
wclsFrameSaver: no samples within desired window for channel 12080
wclsFrameSaver: no samples within desired window for channel 12081
wclsFrameSaver: no samples within desired window for channel 12116
wclsFrameSaver: no samples within desired window for channel 12116
wclsFrameSaver: no samples within desired window for channel 12116
wclsFrameSaver: no samples within desired window for channel 12117
wclsFrameSaver: no samples within desired window for channel 12117
wclsFrameSaver: no samples within desired window for channel 12117
wclsFrameSaver: no samples within desired window for channel 12118
FrameSaver: q=8.99876e+06 n=892689 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 139 XUs and 173 XVs -> 4 XUVs
C:0 T:1 27 XUs and 30 XVs -> 0 XUVs
C:0 T:2 631 XUs and 561 XVs -> 37 XUVs
C:0 T:3 1317 XUs and 1437 XVs -> 57 XUVs
C:0 T:4 240 XUs and 275 XVs -> 15 XUVs
C:0 T:5 1000 XUs and 1083 XVs -> 50 XUVs
C:0 T:6 56 XUs and 80 XVs -> 1 XUVs
C:0 T:7 489 XUs and 614 XVs -> 18 XUVs
C:0 T:8 1489 XUs and 2142 XVs -> 462 XUVs
C:0 T:9 4694 XUs and 7404 XVs -> 1075 XUVs
C:0 T:10 5330 XUs and 8427 XVs -> 1240 XUVs
C:0 T:11 3624 XUs and 5063 XVs -> 610 XUVs
C:0 T:12 3832 XUs and 4832 XVs -> 546 XUVs
C:0 T:13 8350 XUs and 11072 XVs -> 1715 XUVs
C:0 T:14 12627 XUs and 30682 XVs -> 3930 XUVs
C:0 T:15 2894 XUs and 4316 XVs -> 674 XUVs
10434 XUVs total
2651 collection wire objects
10434 potential space points
Neighbour search...
812118 tests to find 376182 neighbours
Iterating with no regularization...
Begin: 5.44782e+08
0 4.79847e+08
1 4.74958e+08
2 4.74236e+08
3 4.74005e+08
Now with regularization...
Begin: 4.54008e+08
0 4.53836e+08
11-Sep-2025 14:00:41 CEST  Closed output file "np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        111.844       193.731       324.346       190.743       45.4904        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.6501e-05    9.27843e-05   0.000206482   8.0271e-05    3.40402e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               6.27871       6.97537       7.37501       6.98513      0.266074        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00121255    0.00448563     0.0149525    0.00150579    0.00429983       18     
produce:pdvddaphne:DAPHNEReaderPDVD               3.49503       4.34942       5.08429       4.27913      0.368275        18     
produce:ophit:OpHitFinder                        0.0460151     0.0526413     0.057945      0.0526004    0.00321705       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0108188     0.0184339     0.0320655     0.0174921    0.00481912       18     
produce:wclsdatavd:WireCellToolkit                58.6671       74.9779       89.4592       75.4103       8.6848         18     
produce:gaushit:GausHitFinder                    0.743696       1.64453       2.60878       1.65824      0.384153        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000246972   0.000376637   0.000667247   0.000354584   9.71426e-05      18     
produce:reco3d:SpacePointSolver                   10.8752       19.3857       36.1566       18.4036       5.54213        18     
produce:hitpdune:DisambigFromSpacePoints         0.145475      0.299089      0.529029       0.28191      0.105729        18     
produce:pandora:StandardPandora                   25.1602       78.9887       174.877       69.6306       34.0046        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.515753       1.32734       2.75508       1.26173      0.586745        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0211705     0.0368114     0.0712494     0.0324204     0.0120822       18     
[art]:TriggerResults:TriggerResultInserter       1.675e-05    3.05959e-05   0.000102031   2.66305e-05   1.80969e-05      18     
end_path:out1:RootOutput                         5.05e-06     1.04307e-05   3.7641e-05    9.1805e-06    6.72488e-06      18     
end_path:out1:RootOutput(write)                   4.35642       5.63596       6.95504       5.74554      0.809535        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5208.13 MB
  Peak resident set size usage (VmHWM): 3271.91 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup.root
\tHists: np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757592041.0,
    "core.end_time": 1757592041.0,
    "core.events": [
      667074,
      667094,
      667114,
      667134,
      667154,
      667174,
      667194,
      667214,
      667234,
      667254,
      667274,
      667294,
      667314,
      667334,
      667354,
      667374,
      667394,
      667414
    ],
    "core.event_count": 18,
    "core.first_event_number": 667074,
    "core.last_event_number": 667414,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043_reco_stage1_20250911T120041_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757592041.0,
    "core.end_time": 1757592041.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39255
    ],
    "core.runs_subruns": [
      3925500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039255_1853_df-s01-d3_dw_0_20250831T074043.hdf5"
    }
  ]
}
justIN time: 2025-09-19 16:09:10 UTC       justIN version: 01.05.00