justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303653.176@dunegpschedd01.fnal.gov

Jobsub ID303653.176@dunegpschedd01.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:24
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod03
Last heartbeat2026-02-04 00:50:19
From worker nodeHostnamewm00.grid.hep.ph.ic.ac.uk
cpuinfoAMD EPYC 7742 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:49:04
Input filesvd-protodune:np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256.hdf5
JobscriptExit code0
Real time58m (3516s)
CPU time49m (2996s = 85%)
Max RSS bytes2589491200 (2469 MiB)
Outputting started2026-02-04 00:47:42
Output files
Finished2026-02-04 00:50:19
Saved logsjustin-logs:303653.176-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0x3087baa0, 0x42dc0460
Caught exception in looking up space point ptr... 0x42dc0460, 0x15b1f6e0
Caught exception in looking up space point ptr... 0x8de4dd0, 0x60164820
Caught exception in looking up space point ptr... 0x60164820, 0x31009580
Caught exception in looking up space point ptr... 0x4f7bf8a0, 0x4a99a1b0
Caught exception in looking up space point ptr... 0x4a99a1b0, 0xf2e64b0
Caught exception in looking up space point ptr... 0xaf9ff50, 0xaf9da50
Caught exception in looking up space point ptr... 0xaf9e030, 0xaf9fcb0
Caught exception in looking up space point ptr... 0xaf9fcb0, 0xaf9fe00
Caught exception in looking up space point ptr... 0xaf9fe00, 0xaf9ff50
Caught exception in looking up space point ptr... 0x2b1c2950, 0x2b1c0140
Caught exception in looking up space point ptr... 0x2b1c0140, 0x2b1be830
Caught exception in looking up space point ptr... 0x32f33c50, 0x32f2f7f0
Caught exception in looking up space point ptr... 0x32f2f7f0, 0x32f33c50
Caught exception in looking up space point ptr... 0x3fb47bf0, 0x3fb47d30
Caught exception in looking up space point ptr... 0x3fb47d30, 0x3fb47bf0
Caught exception in looking up space point ptr... 0x2ac6b3f0, 0x2ac6b8a0
Caught exception in looking up space point ptr... 0x2ac6b8a0, 0x2ac6b590
Caught exception in looking up space point ptr... 0x51502c00, 0x51509790
Caught exception in looking up space point ptr... 0x51509790, 0xaebaf40
Caught exception in looking up space point ptr... 0xaebaf40, 0x51507070
Caught exception in looking up space point ptr... 0x51507070, 0xaebaf40
Caught exception in looking up space point ptr... 0x2d9f7b60, 0x30cd3ae0
Caught exception in looking up space point ptr... 0x30cd3ae0, 0x3337f5d0
Caught exception in looking up space point ptr... 0x4f150610, 0x4f150480
Caught exception in looking up space point ptr... 0x4f150480, 0x4f150610
Caught exception in looking up space point ptr... 0x1419d090, 0x42e2dae0
Caught exception in looking up space point ptr... 0x42e2dae0, 0x1419d090
Caught exception in looking up space point ptr... 0x33361ed0, 0xb09f780
Caught exception in looking up space point ptr... 0xb09f780, 0x33361ed0
Caught exception in looking up space point ptr... 0x33361ed0, 0x141868a0
Caught exception in looking up space point ptr... 0x141868a0, 0x33361ed0
Caught exception in looking up space point ptr... 0x33361ed0, 0x59762240
Caught exception in looking up space point ptr... 0x59762240, 0x33361ed0
Caught exception in looking up space point ptr... 0x4143fbe0, 0x4143f800
Caught exception in looking up space point ptr... 0x4143f800, 0x4143fbe0
Caught exception in looking up space point ptr... 0x3fb460d0, 0x4f406630
Caught exception in looking up space point ptr... 0x4f406630, 0x3fb460d0
Caught exception in looking up space point ptr... 0x35f37560, 0x497e5110
Caught exception in looking up space point ptr... 0x497e5110, 0x35f37560
Caught exception in looking up space point ptr... 0x14182fe0, 0x14182e00
Caught exception in looking up space point ptr... 0x14182e00, 0x14182950
Caught exception in looking up space point ptr... 0x8de2c50, 0x8dd7980
Caught exception in looking up space point ptr... 0x8dd7980, 0x8de0870
Caught exception in looking up space point ptr... 0x3977b3a0, 0x39776c40
Caught exception in looking up space point ptr... 0x39776c40, 0x3977b3a0
Caught exception in looking up space point ptr... 0x28ce7ea0, 0x28ce7fa0
Caught exception in looking up space point ptr... 0x28ce7fa0, 0x28ce6a80
Caught exception in looking up space point ptr... 0x25176c20, 0x25174750
Caught exception in looking up space point ptr... 0x25174750, 0x25176c20
Caught exception in looking up space point ptr... 0xaeb9f70, 0xaeba650
Caught exception in looking up space point ptr... 0xaeba650, 0xaeb9f70
Caught exception in looking up space point ptr... 0xaeb9f70, 0xaeba7e0
Caught exception in looking up space point ptr... 0xaeba7e0, 0xaeb9f70
Caught exception in looking up space point ptr... 0xaeb9f70, 0xaeba4c0
Caught exception in looking up space point ptr... 0xaeba4c0, 0xaeb9f70
Caught exception in looking up space point ptr... 0x4cebca70, 0x3961b5a0
Caught exception in looking up space point ptr... 0x3961b5a0, 0x3961b730
Caught exception in looking up space point ptr... 0x2e8c2210, 0x48829d00
Caught exception in looking up space point ptr... 0x48829d00, 0x2e8c2210
Caught exception in looking up space point ptr... 0x28cce6c0, 0x28ccb3e0
Caught exception in looking up space point ptr... 0x28ccb3e0, 0x2506a560
Caught exception in looking up space point ptr... 0x4f9b4ff0, 0x4f9b6090
Caught exception in looking up space point ptr... 0x4f9b6090, 0x4f9b4ff0
Caught exception in looking up space point ptr... 0x3fdb33d0, 0x20e7bea0
Caught exception in looking up space point ptr... 0x20e7bea0, 0x3fdb33d0
++++>>>> total num hits: 58624, num free: 56675
04-Feb-2026 00:47:31 GMT  Closed output file "np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256_reco_stage0_20260204T004731_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         103.79        125.998       164.069       121.408       17.7176        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       6.5825e-05    0.000125402   0.000295118   0.000102013   6.14022e-05      27     
produce:tpcrawdecoder:PDVDTPCReader                10.894        12.8647       15.2653       12.6689       1.02966        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.506225      0.628323       1.95842      0.559507      0.266653        27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0162189     0.0570496     0.891583      0.0169693     0.165071        27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0802532     0.0822685     0.0900246     0.0822722     0.0016887       27     
produce:beamevent:BeamEvent                      7.8508e-05    0.000155368   0.000950287   0.000117853   0.00015867       27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.21429       4.40558       5.63943       4.35871      0.619343        27     
produce:ophit:OpHitFinder                         0.031701      0.0486626     0.0605849     0.0486076    0.00671625       27     
produce:wclsdatavd:WireCellToolkit                 67.9482       86.7413       109.404       86.4802       12.1519        27     
produce:gaushit:GausHitFinder                     0.542979       1.16099       2.03306       1.06448      0.390691        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000184259   0.000323292   0.000722517   0.000325666   0.000108837      27     
produce:reco3d:SpacePointSolver                    4.19465       10.7338       19.0034       10.4615       3.77823        27     
produce:hitpdune:DisambigFromSpacePoints          0.0471773     0.125339      0.213375      0.132257      0.0415689       27     
produce:cluster3d:Cluster3D                       0.894821       2.59747       5.13494       2.64759      0.989697        27     
[art]:TriggerResults:TriggerResultInserter       1.9808e-05    3.30621e-05   8.4941e-05    2.7161e-05    1.53524e-05      27     
end_path:out1:RootOutput                          3.156e-06    1.18471e-05   5.1568e-05     9.026e-06    1.09815e-05      27     
end_path:out1:RootOutput(write)                    4.80899       6.52426       10.3589       5.92234       1.62338        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4494.77 MB
  Peak resident set size usage (VmHWM): 2589.49 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256_reco_stage0_20260204T004731_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256_reco_stage0_20260204T004731_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256_reco_stage0_20260204T004731_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162558.0,
    "core.end_time": 1770166054.0,
    "core.events": [
      21617,
      21623,
      21629,
      21635,
      21641,
      21647,
      21653,
      21659,
      21665,
      21671,
      21677,
      21683,
      21689,
      21695,
      21701,
      21707,
      21713,
      21719,
      21725,
      21731,
      21737,
      21743,
      21749,
      21755,
      21761,
      21767,
      21773
    ],
    "core.event_count": 27,
    "core.first_event_number": 21617,
    "core.last_event_number": 21773,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0133_df-s04-d1_dw_0_20260203T144256.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:33:12 UTC       justIN version: 01.06.00