justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 39336.100@dunegpschedd02.fnal.gov

Jobsub ID39336.100@dunegpschedd02.fnal.gov
Workflow ID2576
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-13 15:36:17
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_klomp
Last heartbeat2025-09-13 18:14:42
From worker nodeHostnamewn-snel-030.farm.nikhef.nl
cpuinfoAMD EPYC 7H12 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job stateaborted
Started2025-09-13 16:51:51
Input filesvd-protodune:np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135.hdf5
Outputting started2025-09-13 18:11:52
Output filesvd-protodune-det-reco:np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181144_keepup.root
Finished2025-09-13 18:14:42
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

les within desired window for channel 8822
wclsFrameSaver: no samples within desired window for channel 9150
wclsFrameSaver: no samples within desired window for channel 9150
wclsFrameSaver: no samples within desired window for channel 9151
wclsFrameSaver: no samples within desired window for channel 9151
wclsFrameSaver: no samples within desired window for channel 9152
wclsFrameSaver: no samples within desired window for channel 9153
wclsFrameSaver: no samples within desired window for channel 9154
wclsFrameSaver: no samples within desired window for channel 9794
wclsFrameSaver: no samples within desired window for channel 9846
wclsFrameSaver: no samples within desired window for channel 9846
wclsFrameSaver: no samples within desired window for channel 9847
wclsFrameSaver: no samples within desired window for channel 9848
wclsFrameSaver: no samples within desired window for channel 9849
wclsFrameSaver: no samples within desired window for channel 9980
wclsFrameSaver: no samples within desired window for channel 10189
wclsFrameSaver: no samples within desired window for channel 10211
wclsFrameSaver: no samples within desired window for channel 10283
wclsFrameSaver: no samples within desired window for channel 10470
wclsFrameSaver: no samples within desired window for channel 10760
wclsFrameSaver: no samples within desired window for channel 10834
wclsFrameSaver: no samples within desired window for channel 10835
wclsFrameSaver: no samples within desired window for channel 10838
wclsFrameSaver: no samples within desired window for channel 10904
wclsFrameSaver: no samples within desired window for channel 10907
wclsFrameSaver: no samples within desired window for channel 11794
wclsFrameSaver: no samples within desired window for channel 11794
wclsFrameSaver: no samples within desired window for channel 11794
wclsFrameSaver: no samples within desired window for channel 11794
wclsFrameSaver: no samples within desired window for channel 11796
wclsFrameSaver: no samples within desired window for channel 11796
wclsFrameSaver: no samples within desired window for channel 11797
wclsFrameSaver: no samples within desired window for channel 12179
wclsFrameSaver: no samples within desired window for channel 12179
wclsFrameSaver: no samples within desired window for channel 12181
wclsFrameSaver: no samples within desired window for channel 12182
wclsFrameSaver: no samples within desired window for channel 12183
FrameSaver: q=6.50886e+06 n=836037 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 928 XUs and 1073 XVs -> 38 XUVs
C:0 T:1 1004 XUs and 1303 XVs -> 52 XUVs
C:0 T:2 401 XUs and 300 XVs -> 10 XUVs
C:0 T:3 319 XUs and 217 XVs -> 15 XUVs
C:0 T:4 651 XUs and 606 XVs -> 28 XUVs
C:0 T:5 282 XUs and 268 XVs -> 8 XUVs
C:0 T:6 155 XUs and 125 XVs -> 11 XUVs
C:0 T:7 781 XUs and 802 XVs -> 31 XUVs
C:0 T:8 90 XUs and 89 XVs -> 7 XUVs
C:0 T:9 4352 XUs and 3372 XVs -> 179 XUVs
C:0 T:10 2483 XUs and 2888 XVs -> 139 XUVs
C:0 T:11 1054 XUs and 1497 XVs -> 60 XUVs
C:0 T:12 879 XUs and 1063 XVs -> 75 XUVs
C:0 T:13 897 XUs and 955 XVs -> 71 XUVs
C:0 T:14 970 XUs and 858 XVs -> 43 XUVs
C:0 T:15 448 XUs and 646 XVs -> 38 XUVs
805 XUVs total
723 collection wire objects
805 potential space points
Neighbour search...
3263 tests to find 1732 neighbours
Iterating with no regularization...
Begin: 3.19381e+08
0 3.1703e+08
1 3.16937e+08
Now with regularization...
Begin: 3.14044e+08
0 3.14041e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
13-Sep-2025 20:11:44 CEST  Closed output file "np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181144_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        217.005       263.931       326.507        266.8        31.7045        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.0513e-05    7.73691e-05   0.000172634   7.2176e-05    2.35662e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               81.2172       107.34        145.15        104.633       19.2324        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0344874     0.035577      0.0492125     0.0347466    0.00331605       18     
produce:pdvddaphne:DAPHNEReaderPDVD               12.6579       14.9456       18.2523       14.5639       1.64165        18     
produce:ophit:OpHitFinder                        0.0656115     0.0797747     0.100843      0.0805293    0.00793587       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0105183     0.0194374     0.0298776     0.019257      0.0042345       18     
produce:wclsdatavd:WireCellToolkit                56.3902       70.752        85.4286       71.1138       6.56968        18     
produce:gaushit:GausHitFinder                    0.935373       1.20556       1.49595       1.20524      0.181103        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000251291   0.000343495   0.000557736   0.00033541    6.92117e-05      18     
produce:reco3d:SpacePointSolver                   8.95835       13.8458       20.4171       13.4863       3.26409        18     
produce:hitpdune:DisambigFromSpacePoints         0.123311      0.183665       0.2455       0.175092      0.0381752       18     
produce:pandora:StandardPandora                   25.6017        49.94        73.9405       51.2822       14.0481        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.545836      0.956585       1.64735      0.823491      0.335261        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0195474     0.0274353     0.0407073     0.0264757    0.00516649       18     
[art]:TriggerResults:TriggerResultInserter      1.3645e-05    1.68248e-05   4.2029e-05    1.52585e-05   6.36316e-06      18     
end_path:out1:RootOutput                         3.166e-06    4.65878e-06   2.2362e-05     3.612e-06     4.303e-06       18     
end_path:out1:RootOutput(write)                   4.3489        4.56702       4.92165       4.54744      0.166123        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5206.09 MB
  Peak resident set size usage (VmHWM): 3322.04 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181144_keepup.root
\tHists: np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181145_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181144_keepup.root
Ran successfully
{
  "name": "np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181144_keepup.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757787105.0,
    "core.end_time": 1757787105.0,
    "core.events": [
      53286,
      53306,
      53326,
      53346,
      53366,
      53386,
      53406,
      53426,
      53446,
      53466,
      53486,
      53506,
      53526,
      53546,
      53566,
      53586,
      53606,
      53626
    ],
    "core.event_count": 18,
    "core.first_event_number": 53286,
    "core.last_event_number": 53626,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135_reco_stage1_20250913T181145_keepup_hists.root",
  "namespace": "vd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_10_00d00",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
    "dune.campaign": "vd-protodune-reco-keepup-v0",
    "core.start_time": 1757787105.0,
    "core.end_time": 1757787105.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      39275
    ],
    "core.runs_subruns": [
      3927500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics"
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run039275_0148_df-s03-d0_dw_0_20250901T193135.hdf5"
    }
  ]
}
justIN time: 2025-09-19 02:08:05 UTC       justIN version: 01.05.00