justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297923.149@dunegpschedd02.fnal.gov

Jobsub ID297923.149@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 23:00:26
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2026-02-04 01:33:57
From worker nodeHostnameheplnc153.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 00:08:59
Input filesvd-protodune:np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906.hdf5
JobscriptExit code0
Real time1h (4901s)
CPU time1h (4542s = 92%)
Max RSS bytes3570880512 (3405 MiB)
Outputting started2026-02-04 01:30:43
Output files
Finished2026-02-04 01:33:57
Saved logsjustin-logs:297923.149-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing up space point ptr... 0x6cb2c1a0, 0x6cb1d690
Caught exception in looking up space point ptr... 0x6cb1d690, 0x6cb25660
Caught exception in looking up space point ptr... 0x37c0be40, 0x37c0b720
Caught exception in looking up space point ptr... 0x37c0b720, 0x37c0b8c0
Caught exception in looking up space point ptr... 0x16cb8130, 0x16cb0460
Caught exception in looking up space point ptr... 0x16cb12e0, 0x16cb8130
Caught exception in looking up space point ptr... 0x46313c30, 0x2b3bb360
Caught exception in looking up space point ptr... 0x69776ee0, 0x46313c30
Caught exception in looking up space point ptr... 0x6cb35210, 0x6cb37c30
Caught exception in looking up space point ptr... 0x6cb37c30, 0x6cb3abf0
Caught exception in looking up space point ptr... 0x6cb3abf0, 0x6cb349c0
Caught exception in looking up space point ptr... 0x56134c70, 0x6ef35f50
Caught exception in looking up space point ptr... 0x6ef35f50, 0x56134c70
Caught exception in looking up space point ptr... 0x429f2020, 0x5baffce0
Caught exception in looking up space point ptr... 0x5baffce0, 0x394609f0
Caught exception in looking up space point ptr... 0x19e84960, 0x3ce0d4b0
Caught exception in looking up space point ptr... 0x3ce0d4b0, 0x19e84960
Caught exception in looking up space point ptr... 0x67485ed0, 0x674861f0
Caught exception in looking up space point ptr... 0x674861f0, 0x67485ed0
Caught exception in looking up space point ptr... 0x320d5a90, 0x320d58b0
Caught exception in looking up space point ptr... 0x320d58b0, 0x320d5a90
Caught exception in looking up space point ptr... 0x1741ecf0, 0x67969770
Caught exception in looking up space point ptr... 0x67969770, 0x6796a290
Caught exception in looking up space point ptr... 0x37c11320, 0x37c0add0
Caught exception in looking up space point ptr... 0x37c0add0, 0x37c11320
Caught exception in looking up space point ptr... 0x37c11320, 0x37c09750
Caught exception in looking up space point ptr... 0x37c09750, 0x37c11320
Caught exception in looking up space point ptr... 0x44c4b640, 0x1293ed20
Caught exception in looking up space point ptr... 0x1293ed20, 0x1293ea00
Caught exception in looking up space point ptr... 0x6cbe6bb0, 0x6cbe7180
Caught exception in looking up space point ptr... 0x6cbe7180, 0x6cbe6950
Caught exception in looking up space point ptr... 0x148b6190, 0x148adea0
Caught exception in looking up space point ptr... 0x148adea0, 0x148b6190
Caught exception in looking up space point ptr... 0x561347e0, 0x69c1ff50
Caught exception in looking up space point ptr... 0x69c1ff50, 0x68132910
Caught exception in looking up space point ptr... 0x58ba0670, 0x59770c20
Caught exception in looking up space point ptr... 0x59770c20, 0x58ba0670
Caught exception in looking up space point ptr... 0x69827d90, 0x69833c40
Caught exception in looking up space point ptr... 0x69833c40, 0x6983ed80
Caught exception in looking up space point ptr... 0x463464e0, 0x46346190
Caught exception in looking up space point ptr... 0x46346190, 0x46346290
Caught exception in looking up space point ptr... 0xcaeedd0, 0xcaeec40
Caught exception in looking up space point ptr... 0xcaeec40, 0x44c0d500
Caught exception in looking up space point ptr... 0x82e5e520, 0x82e5e390
Caught exception in looking up space point ptr... 0x82e65d40, 0x82e5e520
Caught exception in looking up space point ptr... 0x359687f0, 0x35973160
Caught exception in looking up space point ptr... 0x35973160, 0x35981110
Caught exception in looking up space point ptr... 0x3972d880, 0x3ac9e6a0
Caught exception in looking up space point ptr... 0x3ac9e6a0, 0x3972d880
Caught exception in looking up space point ptr... 0x62b965f0, 0x62b96780
Caught exception in looking up space point ptr... 0x62b96780, 0x62b96910
Caught exception in looking up space point ptr... 0x793c77e0, 0x793c6e30
Caught exception in looking up space point ptr... 0x793c6e30, 0x793c7a40
Caught exception in looking up space point ptr... 0x394605a0, 0x394608c0
Caught exception in looking up space point ptr... 0x394608c0, 0x394605a0
Caught exception in looking up space point ptr... 0x394605a0, 0x592fa410
Caught exception in looking up space point ptr... 0x592fa410, 0x394605a0
Caught exception in looking up space point ptr... 0x333359c0, 0x8b071d10
Caught exception in looking up space point ptr... 0x8b071d10, 0x333359c0
Caught exception in looking up space point ptr... 0x195304e0, 0x195303a0
Caught exception in looking up space point ptr... 0x195303a0, 0x67a00f20
Caught exception in looking up space point ptr... 0x5906cbe0, 0x16c6abc0
Caught exception in looking up space point ptr... 0x16c6abc0, 0x5bb96c10
Caught exception in looking up space point ptr... 0x5bb96c10, 0x5bb96ad0
Caught exception in looking up space point ptr... 0x5bb96ad0, 0x5906cbe0
Caught exception in looking up space point ptr... 0x12ff44a0, 0x44d8c090
Caught exception in looking up space point ptr... 0x5bbbb340, 0x12ff44a0
++++>>>> total num hits: 97829, num free: 94532
04-Feb-2026 01:30:32 GMT  Closed output file "np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906_reco_stage0_20260204T013032_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         202.698       253.097       303.069       253.782       27.2194        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        9.078e-05    0.000164773   0.000254176   0.000167654   4.77421e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                13.0415        14.68        15.9308       14.8196      0.737174        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.530351      0.555919       0.6217       0.550151      0.0257144       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0178412     0.0184631     0.0266565     0.0179919    0.00193436       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0177112     0.0180438     0.0191865     0.0179901    0.000296939      19     
produce:beamevent:BeamEvent                      0.00010701    0.000205001   0.00127607    0.000145843   0.000253749      19     
produce:pdvddaphne:DAPHNEReaderPDVD                2.9806        3.63717       4.49652       3.69659      0.330031        19     
produce:ophit:OpHitFinder                         0.0375895     0.0448954     0.0531441     0.0449201    0.00420462       19     
produce:wclsdatavd:WireCellToolkit                 116.677       138.026       156.583       138.78        10.1656        19     
produce:gaushit:GausHitFinder                      1.82198       2.86285       4.04374       3.0549       0.727193        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000269154   0.000616845   0.00265239    0.000467195   0.00050601       19     
produce:reco3d:SpacePointSolver                    9.48009       21.0382       37.2656       20.1804       8.64096        19     
produce:hitpdune:DisambigFromSpacePoints          0.239189       0.42205      0.674016      0.405092       0.1431         19     
produce:cluster3d:Cluster3D                        3.12789       63.5197       78.474        72.3447       21.774         19     
[art]:TriggerResults:TriggerResultInserter       2.7442e-05    5.06948e-05   0.000212888    4.273e-05    3.88023e-05      19     
end_path:out1:RootOutput                          5.069e-06    1.22551e-05    4.237e-05    1.0019e-05    8.48239e-06      19     
end_path:out1:RootOutput(write)                    5.81661       8.22379       10.2677       8.46326       1.40301        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5552.54 MB
  Peak resident set size usage (VmHWM): 3570.88 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906_reco_stage0_20260204T013032_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906_reco_stage0_20260204T013032_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906_reco_stage0_20260204T013032_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163755.0,
    "core.end_time": 1770168634.0,
    "core.events": [
      8103,
      8109,
      8115,
      8121,
      8127,
      8133,
      8139,
      8145,
      8151,
      8157,
      8163,
      8169,
      8175,
      8181,
      8187,
      8193,
      8199,
      8205,
      8211
    ],
    "core.event_count": 19,
    "core.first_event_number": 8103,
    "core.last_event_number": 8211,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0071_df-s03-d1_dw_0_20260203T190906.hdf5"
    }
  ]
}
justIN time: 2026-02-04 04:28:16 UTC       justIN version: 01.06.00