justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.139@dunegpschedd02.fnal.gov

Jobsub ID297921.139@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-02-04 01:13:59
From worker nodeHostnamecn006.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:56:34
Input filesvd-protodune:np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508.hdf5
JobscriptExit code0
Real time1h (4488s)
CPU time1h (4209s = 93%)
Max RSS bytes3399487488 (3242 MiB)
Outputting started2026-02-04 01:11:24
Output files
Finished2026-02-04 01:13:59
Saved logsjustin-logs:297921.139-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x389f41c0, 0x612b9c10
Caught exception in looking up space point ptr... 0xa4b5fd0, 0xa4b0b50
Caught exception in looking up space point ptr... 0xa4b0b50, 0xa4b68a0
Caught exception in looking up space point ptr... 0x6311e540, 0x6311e150
Caught exception in looking up space point ptr... 0x6311e150, 0x6311e540
Caught exception in looking up space point ptr... 0x6311e540, 0x6311e2a0
Caught exception in looking up space point ptr... 0x6311e2a0, 0x6311e540
Caught exception in looking up space point ptr... 0x3766ba40, 0x3766b7a0
Caught exception in looking up space point ptr... 0x3766b7a0, 0x3766ba40
Caught exception in looking up space point ptr... 0x17dc7fa0, 0x17dc6520
Caught exception in looking up space point ptr... 0x17dc6520, 0x17dc7fa0
Caught exception in looking up space point ptr... 0x439ca3a0, 0x30193b70
Caught exception in looking up space point ptr... 0x30193b70, 0x439ca3a0
Caught exception in looking up space point ptr... 0x708154e0, 0x70815290
Caught exception in looking up space point ptr... 0x70815290, 0x708154e0
Caught exception in looking up space point ptr... 0x6312fbd0, 0x6312f980
Caught exception in looking up space point ptr... 0x63138eb0, 0x6312fbd0
Caught exception in looking up space point ptr... 0x45485050, 0x45485210
Caught exception in looking up space point ptr... 0x45485210, 0x45485050
Caught exception in looking up space point ptr... 0x69dcb660, 0x7b70bb70
Caught exception in looking up space point ptr... 0x7b70bb70, 0x69dcb660
Caught exception in looking up space point ptr... 0x50972d90, 0x9784f60
Caught exception in looking up space point ptr... 0x9784f60, 0x50972d90
Caught exception in looking up space point ptr... 0x581159c0, 0x5cd59a40
Caught exception in looking up space point ptr... 0x5cd59a40, 0x5b3dbf30
Caught exception in looking up space point ptr... 0x45482710, 0x45482630
Caught exception in looking up space point ptr... 0x45482630, 0x45482710
Caught exception in looking up space point ptr... 0x508d4750, 0x508d48e0
Caught exception in looking up space point ptr... 0x508d48e0, 0x4c6885d0
Caught exception in looking up space point ptr... 0x4c6885d0, 0x814429d0
Caught exception in looking up space point ptr... 0x814429d0, 0x508d4750
Caught exception in looking up space point ptr... 0x3bd05b60, 0x4d70f860
Caught exception in looking up space point ptr... 0x4d70f860, 0x3bd05e20
Caught exception in looking up space point ptr... 0x7b7134a0, 0x3b8b0a40
Caught exception in looking up space point ptr... 0x3b8b0a40, 0x4404b400
Caught exception in looking up space point ptr... 0x6251a4d0, 0xbf7db90
Caught exception in looking up space point ptr... 0xbf7db90, 0x62514ca0
Caught exception in looking up space point ptr... 0xec5c930, 0xec4a380
Caught exception in looking up space point ptr... 0xec4a380, 0xec5c930
Caught exception in looking up space point ptr... 0x3c9a3190, 0x3c9a2e70
Caught exception in looking up space point ptr... 0x3c9a2e70, 0x3c9a3190
Caught exception in looking up space point ptr... 0x6a02c170, 0x6a02c020
Caught exception in looking up space point ptr... 0x6a02c020, 0x6a018150
Caught exception in looking up space point ptr... 0x6241a2e0, 0x6241a150
Caught exception in looking up space point ptr... 0x6241a150, 0x62407f80
Caught exception in looking up space point ptr... 0x39bdd650, 0x39bdf700
Caught exception in looking up space point ptr... 0x39bdf700, 0x39bdd650
Caught exception in looking up space point ptr... 0x454a5540, 0x454a52e0
Caught exception in looking up space point ptr... 0x454a52e0, 0x454a5540
Caught exception in looking up space point ptr... 0x454a5540, 0x454a5410
Caught exception in looking up space point ptr... 0x454a5410, 0x454a5540
Caught exception in looking up space point ptr... 0x5868c5f0, 0x5868c460
Caught exception in looking up space point ptr... 0x5868c460, 0x5868c5f0
Caught exception in looking up space point ptr... 0x5868c5f0, 0x5868c2d0
Caught exception in looking up space point ptr... 0x5868c2d0, 0x5868c5f0
Caught exception in looking up space point ptr... 0x52ed5280, 0x335cdac0
Caught exception in looking up space point ptr... 0x335cdac0, 0x52ed5280
Caught exception in looking up space point ptr... 0x7b98f440, 0x104a4360
Caught exception in looking up space point ptr... 0x104a4360, 0x7b98f440
Caught exception in looking up space point ptr... 0x52ecb230, 0x52ed36d0
Caught exception in looking up space point ptr... 0x52ed36d0, 0x52ecb230
Caught exception in looking up space point ptr... 0x5b3b70b0, 0x79e2f530
Caught exception in looking up space point ptr... 0x79e2f530, 0x5b3b70b0
Caught exception in looking up space point ptr... 0x45473f10, 0x45473b20
Caught exception in looking up space point ptr... 0x45473b20, 0x45473f10
Caught exception in looking up space point ptr... 0x59306260, 0x37668ca0
Caught exception in looking up space point ptr... 0x37668ca0, 0x59306260
++++>>>> total num hits: 89996, num free: 86609
04-Feb-2026 01:11:09 GMT  Closed output file "np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508_reco_stage0_20260204T011109_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         201.631       230.288       284.616       223.483       19.6011        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       8.1956e-05    0.000144819   0.000376099   0.000127907   6.45243e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                15.1796       17.2096       29.6607       15.9483       3.36342        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.496786      0.516224       0.59691      0.507465      0.0245826       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0167485     0.0175953     0.0275999     0.0169724    0.00236971       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0167483     0.0170054     0.0174063     0.0169621    0.000188239      19     
produce:beamevent:BeamEvent                      0.000144008   0.000284213   0.00131082    0.000233371   0.000250041      19     
produce:pdvddaphne:DAPHNEReaderPDVD                6.11636       7.08302       8.38657       7.08416      0.649874        19     
produce:ophit:OpHitFinder                         0.0365639     0.0557864     0.077201      0.057063      0.013109        19     
produce:wclsdatavd:WireCellToolkit                 138.414       169.093       206.019       166.078       15.1719        19     
produce:gaushit:GausHitFinder                      1.50276       2.10674       2.94175       2.20549      0.349487        19     
produce:nhitsfilter:NumberOfHitsFilter           0.000570084   0.00090929    0.00182562    0.000837181   0.000279918      19     
produce:reco3d:SpacePointSolver                    14.2395       23.9519       34.8408       25.5441       6.56471        19     
produce:hitpdune:DisambigFromSpacePoints          0.253747      0.364534      0.554931      0.356711      0.0819528       19     
produce:cluster3d:Cluster3D                        1.97213       3.11336       5.05525       3.16758       0.77485        19     
[art]:TriggerResults:TriggerResultInserter       3.1626e-05    4.86852e-05   0.000131426   4.3476e-05    2.09276e-05      19     
end_path:out1:RootOutput                          6.842e-06    1.11768e-05   4.0311e-05     9.522e-06    7.23209e-06      19     
end_path:out1:RootOutput(write)                    5.20866       6.68734       8.37562       6.4173       0.952875        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5316.55 MB
  Peak resident set size usage (VmHWM): 3399.49 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508_reco_stage0_20260204T011109_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508_reco_stage0_20260204T011109_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508_reco_stage0_20260204T011109_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770163005.0,
    "core.end_time": 1770167474.0,
    "core.events": [
      39907,
      39913,
      39919,
      39925,
      39931,
      39937,
      39943,
      39949,
      39955,
      39961,
      39967,
      39973,
      39979,
      39985,
      39991,
      39997,
      40003,
      40009,
      40015
    ],
    "core.event_count": 19,
    "core.first_event_number": 39907,
    "core.last_event_number": 40015,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0350_df-s02-d1_dw_0_20260203T205508.hdf5"
    }
  ]
}
justIN time: 2026-02-04 05:49:56 UTC       justIN version: 01.06.00