justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.171@dunegpschedd02.fnal.gov

Jobsub ID297921.171@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-02-04 02:12:41
From worker nodeHostnamecn051.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:56:23
Input filesvd-protodune:np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223.hdf5
JobscriptExit code0
Real time2h (8025s)
CPU time1h (4880s = 60%)
Max RSS bytes3341459456 (3186 MiB)
Outputting started2026-02-04 02:10:12
Output files
Finished2026-02-04 02:12:41
Saved logsjustin-logs:297921.171-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng up space point ptr... 0x78813b60, 0x78813e00
Caught exception in looking up space point ptr... 0x4a664970, 0x3be26a50
Caught exception in looking up space point ptr... 0x3be26a50, 0x4a664970
Caught exception in looking up space point ptr... 0x493bf1f0, 0x493e6d50
Caught exception in looking up space point ptr... 0x493e6d50, 0x493bf1f0
Caught exception in looking up space point ptr... 0x493bf1f0, 0x424a9270
Caught exception in looking up space point ptr... 0x424a9270, 0x493bf1f0
Caught exception in looking up space point ptr... 0x4b71dc40, 0x4b71f800
Caught exception in looking up space point ptr... 0x4b71f800, 0x4b71dc40
Caught exception in looking up space point ptr... 0x1330e9f0, 0x1330e6d0
Caught exception in looking up space point ptr... 0x1330e6d0, 0x1330e9f0
Caught exception in looking up space point ptr... 0x453ce2f0, 0x453c7f30
Caught exception in looking up space point ptr... 0x453c7f30, 0x453cb8b0
Caught exception in looking up space point ptr... 0x37ab4a00, 0x37ab4220
Caught exception in looking up space point ptr... 0x37ab4220, 0x37ab4a00
Caught exception in looking up space point ptr... 0x37ab4a00, 0x37ab4370
Caught exception in looking up space point ptr... 0x37ab4370, 0x37ab4a00
Caught exception in looking up space point ptr... 0x194c6070, 0x194c5d50
Caught exception in looking up space point ptr... 0x194c5d50, 0x194c5ee0
Caught exception in looking up space point ptr... 0x5f3b7dd0, 0x5f3b8970
Caught exception in looking up space point ptr... 0x5f3b8970, 0x5f3ba1b0
Caught exception in looking up space point ptr... 0x42512040, 0x42511eb0
Caught exception in looking up space point ptr... 0x42511eb0, 0x42512040
Caught exception in looking up space point ptr... 0x519f1520, 0x519f1200
Caught exception in looking up space point ptr... 0x519f1200, 0x519f1520
Caught exception in looking up space point ptr... 0x4b720ba0, 0x7c155c60
Caught exception in looking up space point ptr... 0x7c155c60, 0x7c156240
Caught exception in looking up space point ptr... 0x484f0cc0, 0x484f1020
Caught exception in looking up space point ptr... 0x484f1020, 0x484f1ad0
Caught exception in looking up space point ptr... 0x1b4bae10, 0x1b4bb2c0
Caught exception in looking up space point ptr... 0x1b4bb2c0, 0x1b4bae10
Caught exception in looking up space point ptr... 0x1336c650, 0x1336c4b0
Caught exception in looking up space point ptr... 0x1336c4b0, 0x1336c8f0
Caught exception in looking up space point ptr... 0x3eec49a0, 0x3e294020
Caught exception in looking up space point ptr... 0x3e294020, 0x56d83670
Caught exception in looking up space point ptr... 0x4827e220, 0x4827df70
Caught exception in looking up space point ptr... 0x4827df70, 0x4827ac70
Caught exception in looking up space point ptr... 0x519fa000, 0x519f9e70
Caught exception in looking up space point ptr... 0x519f9e70, 0x519fa000
Caught exception in looking up space point ptr... 0x6e926db0, 0x6e926c20
Caught exception in looking up space point ptr... 0x6e926c20, 0x6e926db0
Caught exception in looking up space point ptr... 0x379858f0, 0x3797c100
Caught exception in looking up space point ptr... 0x3797c100, 0x379858f0
Caught exception in looking up space point ptr... 0x379858f0, 0x37985710
Caught exception in looking up space point ptr... 0x37985710, 0x379858f0
Caught exception in looking up space point ptr... 0x6e8f8140, 0x6e8ff330
Caught exception in looking up space point ptr... 0x6e8ff330, 0x6e8f8140
Caught exception in looking up space point ptr... 0x7bcc3f90, 0x158ea3d0
Caught exception in looking up space point ptr... 0x158ea3d0, 0x7bcc3f90
Caught exception in looking up space point ptr... 0x1a74f2b0, 0x1a74fa60
Caught exception in looking up space point ptr... 0x1a74fa60, 0x1a74f2b0
Caught exception in looking up space point ptr... 0x78838be0, 0x7883ade0
Caught exception in looking up space point ptr... 0x7883ade0, 0x78838be0
Caught exception in looking up space point ptr... 0x4892e8c0, 0x4892e770
Caught exception in looking up space point ptr... 0x4892e770, 0x4892e8c0
Caught exception in looking up space point ptr... 0xba21880, 0xba2e510
Caught exception in looking up space point ptr... 0xba2e510, 0xba21880
Caught exception in looking up space point ptr... 0x412ec560, 0x412ec1f0
Caught exception in looking up space point ptr... 0x412ec1f0, 0x412ebf60
Caught exception in looking up space point ptr... 0x16319e90, 0x481d5eb0
Caught exception in looking up space point ptr... 0x481d5eb0, 0x136a8650
Caught exception in looking up space point ptr... 0x1a7304b0, 0x7882f290
Caught exception in looking up space point ptr... 0x7882f290, 0x3afe12a0
Caught exception in looking up space point ptr... 0x377a9f90, 0x4741c200
Caught exception in looking up space point ptr... 0x4741c200, 0x377a9f90
Caught exception in looking up space point ptr... 0x5a3f7920, 0x5a3f7a20
Caught exception in looking up space point ptr... 0x5a3f7a20, 0x5a3f7820
++++>>>> total num hits: 153566, num free: 147418
04-Feb-2026 02:09:58 GMT  Closed output file "np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223_reco_stage0_20260204T020958_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                          374.2        417.22        479.112       415.415       28.0362        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       9.2075e-05    0.000120166   0.000235692   0.000101275   3.59388e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                110.803       158.636       229.729       150.075       27.5003        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.591819      0.788684       1.07325      0.818412      0.134683        19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0165474     0.0222273     0.0664714     0.0192275     0.0110401       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0165368     0.018387      0.048749      0.0167113    0.00715706       19     
produce:beamevent:BeamEvent                      0.000125612   0.000198466   0.00111409    0.000142372   0.000216728      19     
produce:pdvddaphne:DAPHNEReaderPDVD                14.1363       17.865        22.3218       17.7669       2.62116        19     
produce:ophit:OpHitFinder                         0.0562825     0.0659476     0.0839061     0.0645256    0.00664655       19     
produce:wclsdatavd:WireCellToolkit                 151.653       195.99        206.294       200.398       12.3083        19     
produce:gaushit:GausHitFinder                      1.54789       2.35404       3.25572       2.31515      0.445438        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000779842   0.000955435   0.00138172    0.000914838   0.000143917      19     
produce:reco3d:SpacePointSolver                    15.2326       29.5256       45.5937       29.6425       7.21801        19     
produce:hitpdune:DisambigFromSpacePoints          0.210177      0.420227      0.705517      0.393757      0.130357        19     
produce:cluster3d:Cluster3D                        1.75215       3.99112       6.28896       3.73349       1.41351        19     
[art]:TriggerResults:TriggerResultInserter       3.1088e-05    4.74758e-05   0.000116479   4.3519e-05    1.74646e-05      19     
end_path:out1:RootOutput                          5.902e-06    1.90319e-05   0.000173345    9.479e-06    3.67934e-05      19     
end_path:out1:RootOutput(write)                    5.38934       7.47876       8.68875       7.63351      0.890824        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5330.47 MB
  Peak resident set size usage (VmHWM): 3341.46 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223_reco_stage0_20260204T020958_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223_reco_stage0_20260204T020958_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223_reco_stage0_20260204T020958_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162992.0,
    "core.end_time": 1770171001.0,
    "core.events": [
      3088,
      3094,
      3100,
      3106,
      3112,
      3118,
      3124,
      3130,
      3136,
      3142,
      3148,
      3154,
      3160,
      3166,
      3172,
      3178,
      3184,
      3190,
      3196
    ],
    "core.event_count": 19,
    "core.first_event_number": 3088,
    "core.last_event_number": 3196,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0027_df-s04-d0_dw_0_20260203T185223.hdf5"
    }
  ]
}
justIN time: 2026-02-04 09:54:59 UTC       justIN version: 01.06.00