justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303654.113@dunegpschedd01.fnal.gov

Jobsub ID303654.113@dunegpschedd01.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:52:26
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2026-02-04 01:05:41
From worker nodeHostnamedune001-1846110.0-lcg2727.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:09:49
Input filesvd-protodune:np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052.hdf5
JobscriptExit code0
Real time54m (3252s)
CPU time49m (2952s = 90%)
Max RSS bytes3385843712 (3228 MiB)
Outputting started2026-02-04 01:04:02
Output files
Finished2026-02-04 01:05:41
Saved logsjustin-logs:303654.113-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

. 0x797074a0, 0x79706e10
Caught exception in looking up space point ptr... 0x79706e10, 0x79706c80
Caught exception in looking up space point ptr... 0xbee9ee50, 0xc2e63170
Caught exception in looking up space point ptr... 0xc2e63170, 0x45487320
Caught exception in looking up space point ptr... 0x8313ce00, 0x8313d0e0
Caught exception in looking up space point ptr... 0xbbf141a0, 0x8313ce00
Caught exception in looking up space point ptr... 0xa929b5f0, 0xa929c8a0
Caught exception in looking up space point ptr... 0xa929c8a0, 0xa929b7f0
Caught exception in looking up space point ptr... 0xa929d230, 0xa929b5f0
Caught exception in looking up space point ptr... 0x47cd5aa0, 0x6e7a71a0
Caught exception in looking up space point ptr... 0x6e7a71a0, 0x47cd5aa0
Caught exception in looking up space point ptr... 0x47cd5aa0, 0xb69d6080
Caught exception in looking up space point ptr... 0xb69d6080, 0x47cd5aa0
Caught exception in looking up space point ptr... 0xb3233e90, 0x6e262650
Caught exception in looking up space point ptr... 0x6e262650, 0x87478600
Caught exception in looking up space point ptr... 0x6c4c3570, 0x6e5861e0
Caught exception in looking up space point ptr... 0x74e9d6a0, 0x6c4c3570
Caught exception in looking up space point ptr... 0x76aa30d0, 0x76aaa540
Caught exception in looking up space point ptr... 0x76aaa540, 0x76aa30d0
Caught exception in looking up space point ptr... 0xa9c39270, 0xa9442f90
Caught exception in looking up space point ptr... 0xa9442f90, 0xbee8f900
Caught exception in looking up space point ptr... 0xba578f00, 0x854e41f0
Caught exception in looking up space point ptr... 0x854e41f0, 0x76d803e0
Caught exception in looking up space point ptr... 0xbdec30b0, 0x88f3f8c0
Caught exception in looking up space point ptr... 0x88f3f8c0, 0xbde97ae0
Caught exception in looking up space point ptr... 0x72890cd0, 0xba5bccc0
Caught exception in looking up space point ptr... 0xba5bccc0, 0x72890cd0
Caught exception in looking up space point ptr... 0x8b5b21b0, 0xa1f35a10
Caught exception in looking up space point ptr... 0xa1f35a10, 0x76bb6720
Caught exception in looking up space point ptr... 0x76aa4d50, 0x76aa4e80
Caught exception in looking up space point ptr... 0x76aa4e80, 0x76ab0690
Caught exception in looking up space point ptr... 0x4edfe640, 0x4edfd920
Caught exception in looking up space point ptr... 0x4edfd920, 0x4edfe640
Caught exception in looking up space point ptr... 0xb3fbef60, 0x54749ff0
Caught exception in looking up space point ptr... 0x54749ff0, 0xb3fbef60
Caught exception in looking up space point ptr... 0xc3d484d0, 0xc3d49240
Caught exception in looking up space point ptr... 0xc3d49240, 0xc3d484d0
Caught exception in looking up space point ptr... 0x83148130, 0x83147d70
Caught exception in looking up space point ptr... 0x83147d70, 0x83148130
Caught exception in looking up space point ptr... 0x54ecef30, 0x4ad95250
Caught exception in looking up space point ptr... 0x4ad95250, 0x54ed0140
Caught exception in looking up space point ptr... 0x6e2decb0, 0x7d7cec20
Caught exception in looking up space point ptr... 0x7d7cec20, 0x6e2decb0
Caught exception in looking up space point ptr... 0xa7808c40, 0x6e98a480
Caught exception in looking up space point ptr... 0x6e98a480, 0xa7808c40
Caught exception in looking up space point ptr... 0xb695abe0, 0xb695af00
Caught exception in looking up space point ptr... 0xb695af00, 0xb695b090
Caught exception in looking up space point ptr... 0x830c9780, 0x995a50c0
Caught exception in looking up space point ptr... 0x995a50c0, 0xa98faf00
Caught exception in looking up space point ptr... 0x980249d0, 0x98024dc0
Caught exception in looking up space point ptr... 0x98024dc0, 0x980249d0
Caught exception in looking up space point ptr... 0x980249d0, 0x98024f50
Caught exception in looking up space point ptr... 0x98024f50, 0x980249d0
Caught exception in looking up space point ptr... 0x54de8d70, 0x54de9100
Caught exception in looking up space point ptr... 0x54de9100, 0x54de8d70
Caught exception in looking up space point ptr... 0x980250e0, 0x98025400
Caught exception in looking up space point ptr... 0x98025400, 0x980250e0
Caught exception in looking up space point ptr... 0x441ee870, 0x4ca06190
Caught exception in looking up space point ptr... 0x4ca06190, 0x80152ee0
Caught exception in looking up space point ptr... 0x980263a0, 0x98026850
Caught exception in looking up space point ptr... 0x98026850, 0x980263a0
Caught exception in looking up space point ptr... 0xa7f01e70, 0xa7f029b0
Caught exception in looking up space point ptr... 0xa7f029b0, 0xa7f01e70
Caught exception in looking up space point ptr... 0x49ce54e0, 0x7b3e8b40
Caught exception in looking up space point ptr... 0x7b3e8b40, 0x49ce54e0
Caught exception in looking up space point ptr... 0x854b6360, 0x854fd020
Caught exception in looking up space point ptr... 0x854fd020, 0x854b6360
++++>>>> total num hits: 86017, num free: 82804
04-Feb-2026 01:03:52 UTC  Closed output file "np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052_reco_stage0_20260204T010352_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         147.242       166.966       182.067       168.712       9.40577        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.1202e-05    0.000157594   0.000472835   9.7327e-05    0.000104852      19     
produce:tpcrawdecoder:PDVDTPCReader                16.1268       16.4626       17.1176       16.4388      0.286552        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.669859      0.708374      0.781207      0.705658      0.0244733       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0230312     0.0244987     0.0315756     0.0232523    0.00271248       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0230371     0.0232001     0.0234849     0.0231499    0.000124591      19     
produce:beamevent:BeamEvent                      0.000111409   0.000228616   0.00103657    0.000195566   0.00019388       19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.73576       3.22671       3.50662       3.24621      0.210398        19     
produce:ophit:OpHitFinder                         0.0365831     0.0422678     0.0476407     0.0435622    0.00308081       19     
produce:wclsdatavd:WireCellToolkit                 103.655       115.022       129.092       113.855       6.48663        19     
produce:gaushit:GausHitFinder                     0.842819       1.56626       2.33741       1.61108      0.409232        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000215376   0.000394112   0.000690494   0.000402438   0.000102819      19     
produce:reco3d:SpacePointSolver                    14.9859       21.2897       27.9525       21.9623       3.71044        19     
produce:hitpdune:DisambigFromSpacePoints          0.117861      0.215564       0.3307       0.227216      0.0710435       19     
produce:cluster3d:Cluster3D                        1.42651       2.98996       4.82149       2.89533       1.03245        19     
[art]:TriggerResults:TriggerResultInserter       2.6921e-05    4.11669e-05   0.000138249   3.5864e-05    2.34071e-05      19     
end_path:out1:RootOutput                          4.257e-06    1.01211e-05    3.956e-05     9.285e-06    7.31927e-06      19     
end_path:out1:RootOutput(write)                    3.98418       5.3385        6.6375        5.85548       0.91182        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5681.05 MB
  Peak resident set size usage (VmHWM): 3385.84 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052_reco_stage0_20260204T010352_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052_reco_stage0_20260204T010352_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052_reco_stage0_20260204T010352_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163800.0,
    "core.end_time": 1770167034.0,
    "core.events": [
      50625,
      50631,
      50637,
      50643,
      50649,
      50655,
      50661,
      50667,
      50673,
      50679,
      50685,
      50691,
      50697,
      50703,
      50709,
      50715,
      50721,
      50727,
      50733
    ],
    "core.event_count": 19,
    "core.first_event_number": 50625,
    "core.last_event_number": 50733,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0444_df-s03-d1_dw_0_20260203T213052.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:48:09 UTC       justIN version: 01.06.00