justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297917.81@dunegpschedd02.fnal.gov

Jobsub ID297917.81@dunegpschedd02.fnal.gov
Workflow ID12717
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:48:25
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_klomp_multicore
Last heartbeat2026-02-04 00:18:56
From worker nodeHostnamewn-snel-018.farm.nikhef.nl
cpuinfoAMD EPYC 7H12 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:18:52
Input filesvd-protodune:np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208.hdf5
JobscriptExit code0
Real time57m (3455s)
CPU time53m (3182s = 92%)
Max RSS bytes2660769792 (2537 MiB)
Outputting started2026-02-04 00:16:28
Output files
Finished2026-02-04 00:18:49
Saved logsjustin-logs:297917.81-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

s: 1, count: 3
******************************************************************************************************************
******************************************************************************************************************
******************************************************************************************************************
****> vertexList containted 0 vertices
Caught exception in looking up space point ptr... 0x318c9670, 0x28775910
Caught exception in looking up space point ptr... 0x28775910, 0x2aafb460
Caught exception in looking up space point ptr... 0x2aafb460, 0x105260f0
Caught exception in looking up space point ptr... 0x105260f0, 0x2e84e910
Caught exception in looking up space point ptr... 0x2e84e910, 0x4ae91ed0
Caught exception in looking up space point ptr... 0x4ae91ed0, 0x5aeec870
Caught exception in looking up space point ptr... 0x46101740, 0x460ffc20
Caught exception in looking up space point ptr... 0x460ffc20, 0x461018c0
Caught exception in looking up space point ptr... 0x461018c0, 0x63410680
Caught exception in looking up space point ptr... 0x63414ef0, 0x46101740
Caught exception in looking up space point ptr... 0x3e112210, 0x3e112470
Caught exception in looking up space point ptr... 0x3e112470, 0x43ce17f0
Caught exception in looking up space point ptr... 0x3c872100, 0x3e112210
Caught exception in looking up space point ptr... 0x3c9588d0, 0x3c959500
Caught exception in looking up space point ptr... 0x3c959500, 0x3c959650
Caught exception in looking up space point ptr... 0x3c959650, 0x3c959ea0
Caught exception in looking up space point ptr... 0x3c959d10, 0x3c9599a0
Caught exception in looking up space point ptr... 0x3c9599a0, 0x3c9588d0
Caught exception in looking up space point ptr... 0x33a2b330, 0x33a28c20
Caught exception in looking up space point ptr... 0x2e686ee0, 0x33a2b330
Caught exception in looking up space point ptr... 0x3dab04e0, 0x30ee5550
Caught exception in looking up space point ptr... 0x30ee5550, 0x31c87540
Caught exception in looking up space point ptr... 0xc4891d0, 0x24ffb6b0
Caught exception in looking up space point ptr... 0x24ffb6b0, 0x2497ffb0
Caught exception in looking up space point ptr... 0x2637bd40, 0x5b3efd50
Caught exception in looking up space point ptr... 0x5b3efd50, 0x2637bd40
Caught exception in looking up space point ptr... 0x5155efd0, 0x5155f270
Caught exception in looking up space point ptr... 0x5155f270, 0x5155efd0
Caught exception in looking up space point ptr... 0x3b5ca4f0, 0x40d24220
Caught exception in looking up space point ptr... 0x40d24220, 0x5aaf9ca0
Caught exception in looking up space point ptr... 0xd66f690, 0xd66f8f0
Caught exception in looking up space point ptr... 0xd66f8f0, 0xd66f7c0
Caught exception in looking up space point ptr... 0x5aae6b20, 0x28783310
Caught exception in looking up space point ptr... 0x28783310, 0x5aae6b20
Caught exception in looking up space point ptr... 0x2e67e660, 0x2e67e3a0
Caught exception in looking up space point ptr... 0x2e67e3a0, 0x2e67e4e0
Caught exception in looking up space point ptr... 0x22ad0970, 0x4d612750
Caught exception in looking up space point ptr... 0x4d612750, 0xeee82f0
Caught exception in looking up space point ptr... 0xeee82f0, 0x3b5cb620
Caught exception in looking up space point ptr... 0x3b5cb620, 0x383abfe0
Caught exception in looking up space point ptr... 0x383abfe0, 0x22ad0970
Caught exception in looking up space point ptr... 0x30c66510, 0xff0e420
Caught exception in looking up space point ptr... 0xff0e420, 0x30c66510
Caught exception in looking up space point ptr... 0x3ec6f540, 0x3061ebc0
Caught exception in looking up space point ptr... 0x5c4f1200, 0x3ec6f540
Caught exception in looking up space point ptr... 0xa1a1530, 0x30e33cd0
Caught exception in looking up space point ptr... 0x30e33cd0, 0xf9ab180
Caught exception in looking up space point ptr... 0x460fe880, 0x46100860
Caught exception in looking up space point ptr... 0x46100860, 0x46100780
Caught exception in looking up space point ptr... 0x46100780, 0x46100cc0
Caught exception in looking up space point ptr... 0x11aad060, 0x28405b30
Caught exception in looking up space point ptr... 0x28405b30, 0x28405870
Caught exception in looking up space point ptr... 0x2e677c50, 0x2e6779b0
Caught exception in looking up space point ptr... 0x2e6779b0, 0x2e677c50
Caught exception in looking up space point ptr... 0x2e679090, 0x2e674b60
Caught exception in looking up space point ptr... 0x2e674b60, 0x2e676a20
Caught exception in looking up space point ptr... 0x2a9df980, 0x4ce3f210
Caught exception in looking up space point ptr... 0x4ce3f210, 0x2a9df980
Caught exception in looking up space point ptr... 0x250adea0, 0x27db33c0
Caught exception in looking up space point ptr... 0x27db33c0, 0x250adea0
++++>>>> total num hits: 41814, num free: 40733
04-Feb-2026 01:16:17 CET  Closed output file "np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208_reco_stage0_20260204T001617_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         112.267       124.039       138.18        123.77        6.82602        27     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.8056e-05    0.000111116   0.000322435   0.00010114     4.397e-05       27     
produce:tpcrawdecoder:PDVDTPCReader                11.795        12.2845       13.4973       12.1943      0.381148        27     
produce:triggerrawdecoder:PDVDTriggerReader4      0.512518      0.533756      0.559173      0.537236      0.0123812       27     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0173717     0.0176483     0.0180425     0.0176345    0.000178022      27     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0858337     0.0898779     0.104545      0.0869004    0.00660765       27     
produce:beamevent:BeamEvent                      9.9457e-05    0.000182234   0.00138681    0.000136827   0.000237463      27     
produce:pdvddaphne:DAPHNEReaderPDVD                3.39217       3.79943       4.20474       3.85152      0.208296        27     
produce:ophit:OpHitFinder                         0.0332714     0.0457369     0.0551323     0.0462679     0.0047601       27     
produce:wclsdatavd:WireCellToolkit                 72.8876       82.9425       91.3926       82.8896       5.36268        27     
produce:gaushit:GausHitFinder                     0.681629       1.23883       1.98974       1.13743      0.340201        27     
produce:nhitsfilter:NumberOfHitsFilter           0.000256672   0.000534201   0.000901252   0.00047427    0.000164856      27     
produce:reco3d:SpacePointSolver                    8.15532       14.707        21.6133       14.7237       3.45993        27     
produce:hitpdune:DisambigFromSpacePoints          0.0922688     0.145609      0.238971      0.140577      0.0359469       27     
produce:cluster3d:Cluster3D                        1.43849       3.04925       4.54664       3.05002      0.852072        27     
[art]:TriggerResults:TriggerResultInserter        2.173e-05    3.13332e-05    9.581e-05    2.8103e-05    1.33377e-05      27     
end_path:out1:RootOutput                          3.767e-06    5.31107e-06   2.4265e-05     4.579e-06    3.75405e-06      27     
end_path:out1:RootOutput(write)                    4.10577       5.14931       6.25685       5.27714      0.574074        27     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4623.35 MB
  Peak resident set size usage (VmHWM): 2660.77 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208_reco_stage0_20260204T001617_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208_reco_stage0_20260204T001617_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208_reco_stage0_20260204T001617_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770160746.0,
    "core.end_time": 1770164179.0,
    "core.events": [
      6371,
      6377,
      6383,
      6389,
      6395,
      6401,
      6407,
      6413,
      6419,
      6425,
      6431,
      6437,
      6443,
      6449,
      6455,
      6461,
      6467,
      6473,
      6479,
      6485,
      6491,
      6497,
      6503,
      6509,
      6515,
      6521,
      6527
    ],
    "core.event_count": 27,
    "core.first_event_number": 6371,
    "core.last_event_number": 6527,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42421
    ],
    "core.runs_subruns": [
      4242100001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042421_0039_df-s04-d1_dw_0_20260203T135208.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:42:50 UTC       justIN version: 01.06.00