justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 39746.184@dunegpschedd02.fnal.gov

Jobsub ID39746.184@dunegpschedd02.fnal.gov
Workflow ID2586
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-14 21:51:54
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce505
Last heartbeat2025-09-14 22:48:48
From worker nodeHostnameb9p02p8097.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-14 21:53:34
Input filesvd-protodune:np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434.hdf5
Outputting started2025-09-14 22:45:14
Output files
Finished2025-09-14 22:48:48
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

0550
wclsFrameSaver: no samples within desired window for channel 10551
wclsFrameSaver: no samples within desired window for channel 10551
wclsFrameSaver: no samples within desired window for channel 10552
wclsFrameSaver: no samples within desired window for channel 10552
wclsFrameSaver: no samples within desired window for channel 10552
wclsFrameSaver: no samples within desired window for channel 10574
wclsFrameSaver: no samples within desired window for channel 10576
wclsFrameSaver: no samples within desired window for channel 10576
wclsFrameSaver: no samples within desired window for channel 10608
wclsFrameSaver: no samples within desired window for channel 10729
wclsFrameSaver: no samples within desired window for channel 10924
wclsFrameSaver: no samples within desired window for channel 10925
wclsFrameSaver: no samples within desired window for channel 10927
wclsFrameSaver: no samples within desired window for channel 10942
wclsFrameSaver: no samples within desired window for channel 10943
wclsFrameSaver: no samples within desired window for channel 10995
wclsFrameSaver: no samples within desired window for channel 11421
wclsFrameSaver: no samples within desired window for channel 11421
wclsFrameSaver: no samples within desired window for channel 11421
wclsFrameSaver: no samples within desired window for channel 11657
wclsFrameSaver: no samples within desired window for channel 11657
wclsFrameSaver: no samples within desired window for channel 11657
wclsFrameSaver: no samples within desired window for channel 11658
wclsFrameSaver: no samples within desired window for channel 12078
wclsFrameSaver: no samples within desired window for channel 12078
wclsFrameSaver: no samples within desired window for channel 12079
wclsFrameSaver: no samples within desired window for channel 12079
wclsFrameSaver: no samples within desired window for channel 12079
wclsFrameSaver: no samples within desired window for channel 12080
wclsFrameSaver: no samples within desired window for channel 12081
wclsFrameSaver: no samples within desired window for channel 12120
wclsFrameSaver: no samples within desired window for channel 12120
wclsFrameSaver: no samples within desired window for channel 12120
wclsFrameSaver: no samples within desired window for channel 12122
wclsFrameSaver: no samples within desired window for channel 12122
wclsFrameSaver: no samples within desired window for channel 12123
wclsFrameSaver: no samples within desired window for channel 12123
wclsFrameSaver: no samples within desired window for channel 12123
FrameSaver: q=8.98184e+06 n=932438 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 324 XUs and 300 XVs -> 11 XUVs
C:0 T:1 141 XUs and 158 XVs -> 6 XUVs
C:0 T:2 224 XUs and 242 XVs -> 8 XUVs
C:0 T:3 142 XUs and 179 XVs -> 8 XUVs
C:0 T:4 218 XUs and 282 XVs -> 7 XUVs
C:0 T:5 578 XUs and 732 XVs -> 29 XUVs
C:0 T:6 285 XUs and 312 XVs -> 17 XUVs
C:0 T:7 666 XUs and 1026 XVs -> 62 XUVs
C:0 T:8 4931 XUs and 5220 XVs -> 542 XUVs
C:0 T:9 1686 XUs and 1407 XVs -> 89 XUVs
C:0 T:10 2650 XUs and 3501 XVs -> 454 XUVs
C:0 T:11 305 XUs and 433 XVs -> 42 XUVs
C:0 T:12 1226 XUs and 1086 XVs -> 82 XUVs
C:0 T:13 10158 XUs and 9782 XVs -> 612 XUVs
C:0 T:14 3461 XUs and 3162 XVs -> 116 XUVs
C:0 T:15 3968 XUs and 3561 XVs -> 195 XUVs
2280 XUVs total
1560 collection wire objects
2280 potential space points
Neighbour search...
51748 tests to find 25420 neighbours
Iterating with no regularization...
Begin: 1.13611e+09
0 1.0919e+09
1 1.08829e+09
2 1.08807e+09
Now with regularization...
Begin: 1.07193e+09
0 1.07188e+09
15-Sep-2025 00:45:02 CEST  Closed output file "np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        123.97        169.377       214.147       169.907       30.8326        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      7.1862e-05    0.000134113   0.00043997    0.000111043   7.84868e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               6.73967       7.22855       7.85497       7.30091      0.275507        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00637214    0.00971382     0.035569     0.00776999    0.00640116       18     
produce:pdvddaphne:DAPHNEReaderPDVD               2.32901       2.77169       3.03891       2.78567      0.163316        18     
produce:ophit:OpHitFinder                        0.0382222     0.0466214     0.0539233     0.0465095    0.00463312       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0078467     0.0122432     0.0175324     0.0123963    0.00246572       18     
produce:wclsdatavd:WireCellToolkit                61.382        73.6232       91.5353       71.9695       9.00784        18     
produce:gaushit:GausHitFinder                    0.923945       1.56441       2.16728       1.70766      0.420167        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000189555   0.000355541   0.000481141   0.000385988   8.62421e-05      18     
produce:reco3d:SpacePointSolver                   8.11094       15.7403        22.99        15.728        4.08134        18     
produce:hitpdune:DisambigFromSpacePoints         0.112504      0.239837      0.366172      0.239531      0.0803166       18     
produce:pandora:StandardPandora                   26.8211       60.7011       103.192       62.3443       22.0191        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.413514       1.0496        1.67491       1.11003      0.324778        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0221574     0.0363721     0.0580979     0.0344051     0.0105467       18     
[art]:TriggerResults:TriggerResultInserter      1.7081e-05    2.78006e-05   4.7091e-05     2.736e-05    7.12717e-06      18     
end_path:out1:RootOutput                          5.5e-06     1.00507e-05   2.5201e-05     8.875e-06    4.05165e-06      18     
end_path:out1:RootOutput(write)                   4.69721       6.31976       7.25186       6.44784      0.690426        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4849.31 MB
  Peak resident set size usage (VmHWM): 2911.38 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup.root
\tHists: np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757889903.0,
    "core.end_time": 1757889903.0,
    "core.events": [
      462800,
      462804,
      462808,
      462812,
      462816,
      462820,
      462824,
      462828,
      462832,
      462836,
      462840,
      462844,
      462848,
      462852,
      462856,
      462860,
      462864,
      462868
    ],
    "core.event_count": 18,
    "core.first_event_number": 462800,
    "core.last_event_number": 462868,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39350
    ],
    "core.runs_subruns": [
      3935000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434_reco_stage1_20250914T224502_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757889903.0,
    "core.end_time": 1757889903.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39350
    ],
    "core.runs_subruns": [
      3935000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039350_6340_df-s04-d0_dw_0_20250914T214434.hdf5"
    }
  ]
}
justIN time: 2025-09-19 07:38:20 UTC       justIN version: 01.05.00