justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297921.141@dunegpschedd02.fnal.gov

Jobsub ID297921.141@dunegpschedd02.fnal.gov
Workflow ID12719
Stage ID1
User nameykermaid@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-02-03 22:58:26
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-04 01:15:52
From worker nodeHostnamecn530.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 23:56:14
Input filesvd-protodune:np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002.hdf5
JobscriptExit code0
Real time1h (4627s)
CPU time1h (4380s = 94%)
Max RSS bytes3381473280 (3224 MiB)
Outputting started2026-02-04 01:13:23
Output files
Finished2026-02-04 01:15:52
Saved logsjustin-logs:297921.141-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

point ptr... 0x6333e680, 0x1a39d020
Caught exception in looking up space point ptr... 0x1a39d020, 0x6333e680
Caught exception in looking up space point ptr... 0x4eb505b0, 0x4eb53110
Caught exception in looking up space point ptr... 0x4eb50690, 0x36dafc90
Caught exception in looking up space point ptr... 0x36dafc90, 0x4eb505b0
Caught exception in looking up space point ptr... 0x53553cc0, 0x53553e50
Caught exception in looking up space point ptr... 0x53553e50, 0x6410e1d0
Caught exception in looking up space point ptr... 0x377d2260, 0x377d23f0
Caught exception in looking up space point ptr... 0x377d23f0, 0x377d2580
Caught exception in looking up space point ptr... 0x14288930, 0x142897b0
Caught exception in looking up space point ptr... 0x142897b0, 0x14288930
Caught exception in looking up space point ptr... 0x4d332c40, 0x4d3334a0
Caught exception in looking up space point ptr... 0x4d3334a0, 0x4d3331e0
Caught exception in looking up space point ptr... 0x6ed85d00, 0x6ed85890
Caught exception in looking up space point ptr... 0x6ed85790, 0x6ed85d00
Caught exception in looking up space point ptr... 0x3b2855c0, 0x3b285370
Caught exception in looking up space point ptr... 0x3b285370, 0x1a716da0
Caught exception in looking up space point ptr... 0x14268d90, 0x626d25d0
Caught exception in looking up space point ptr... 0x626d25d0, 0x14268d90
Caught exception in looking up space point ptr... 0x636140a0, 0x63614260
Caught exception in looking up space point ptr... 0x63614260, 0x636140a0
Caught exception in looking up space point ptr... 0x8f98f30, 0x52dfda50
Caught exception in looking up space point ptr... 0x52dff230, 0x8f98f30
Caught exception in looking up space point ptr... 0x1427db40, 0x1427fda0
Caught exception in looking up space point ptr... 0x1427fda0, 0x1427db40
Caught exception in looking up space point ptr... 0x6333bf50, 0x6333baa0
Caught exception in looking up space point ptr... 0x6333baa0, 0x6333bf50
Caught exception in looking up space point ptr... 0x459c4df0, 0x52cc50e0
Caught exception in looking up space point ptr... 0x52cc50e0, 0x459c4df0
Caught exception in looking up space point ptr... 0x1b61bac0, 0x1b61b300
Caught exception in looking up space point ptr... 0x1b61b300, 0x1b61b1a0
Caught exception in looking up space point ptr... 0xfaf1ec0, 0xfaf1cc0
Caught exception in looking up space point ptr... 0xfaf1cc0, 0xfaf1ec0
Caught exception in looking up space point ptr... 0x85f304e0, 0x3bd6e6e0
Caught exception in looking up space point ptr... 0x3bd6e6e0, 0x85f30a00
Caught exception in looking up space point ptr... 0xd3d0b40, 0x62a73290
Caught exception in looking up space point ptr... 0x62a73290, 0xd3d0b40
Caught exception in looking up space point ptr... 0x1426b720, 0x1426b520
Caught exception in looking up space point ptr... 0x1426b520, 0x1426b720
Caught exception in looking up space point ptr... 0x63f576c0, 0x63f575a0
Caught exception in looking up space point ptr... 0x63f575a0, 0x63f576c0
Caught exception in looking up space point ptr... 0x14251ef0, 0x14251bd0
Caught exception in looking up space point ptr... 0x14251bd0, 0x14251ef0
Caught exception in looking up space point ptr... 0x14251ef0, 0x14251a60
Caught exception in looking up space point ptr... 0x14251a60, 0x14251ef0
Caught exception in looking up space point ptr... 0x41dff5e0, 0x41e00d00
Caught exception in looking up space point ptr... 0x41e00d00, 0x41e00fa0
Caught exception in looking up space point ptr... 0x154664e0, 0x41df7630
Caught exception in looking up space point ptr... 0x41df7630, 0x626ae230
Caught exception in looking up space point ptr... 0x626c87d0, 0x41e0af90
Caught exception in looking up space point ptr... 0x41e0af90, 0x626c87d0
Caught exception in looking up space point ptr... 0x6f973130, 0xaf95070
Caught exception in looking up space point ptr... 0xaf95070, 0x3b728660
Caught exception in looking up space point ptr... 0x5dd83110, 0xecce190
Caught exception in looking up space point ptr... 0xecce190, 0x29f205c0
Caught exception in looking up space point ptr... 0x3bd6f390, 0x85f298f0
Caught exception in looking up space point ptr... 0x85f298f0, 0x85f29310
Caught exception in looking up space point ptr... 0x41dfcd70, 0x41dfcad0
Caught exception in looking up space point ptr... 0x41dfcad0, 0x41dfcd70
Caught exception in looking up space point ptr... 0x1548c630, 0x1548be20
Caught exception in looking up space point ptr... 0x1548be20, 0x1548c630
Caught exception in looking up space point ptr... 0x1548c630, 0x1548bfc0
Caught exception in looking up space point ptr... 0x1548bfc0, 0x1548c630
Caught exception in looking up space point ptr... 0x1548c630, 0x1548c4a0
Caught exception in looking up space point ptr... 0x1548c4a0, 0x1548c630
Caught exception in looking up space point ptr... 0x5dd236e0, 0x5dd22c20
Caught exception in looking up space point ptr... 0x5dd22c20, 0x5dd23570
++++>>>> total num hits: 60986, num free: 58110
04-Feb-2026 01:13:11 GMT  Closed output file "np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002_reco_stage0_20260204T011311_offline.root"

===================================================================================================================================
TimeTracker printout (sec)                           Min           Avg           Max         Median          RMS         nEvts   
===================================================================================================================================
Full event                                         211.351       237.838       322.038       230.789       27.3069        19     
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                       0.000106469   0.000148157   0.000303981   0.000134606   4.68753e-05      19     
produce:tpcrawdecoder:PDVDTPCReader                14.8669       16.9096       19.5361       16.8426      0.928655        19     
produce:triggerrawdecoder:PDVDTriggerReader4      0.496956       0.52926      0.630718      0.518523      0.0366313       19     
produce:timingrawdecoder:PDHDTimingRawDecoder     0.0167762     0.0199139     0.0327951     0.0170907    0.00508457       19     
produce:ctbrawdecoder:PDHDCTBRawDecoder           0.0166578     0.0169677     0.0174748     0.0169824    0.000202116      19     
produce:beamevent:BeamEvent                      0.000120925   0.000264951   0.00126624    0.000204028   0.000238828      19     
produce:pdvddaphne:DAPHNEReaderPDVD                5.72483       6.62927       7.94465       6.63053      0.622151        19     
produce:ophit:OpHitFinder                         0.046691      0.0597112     0.0839159     0.0520722     0.0137508       19     
produce:wclsdatavd:WireCellToolkit                 158.186       180.954       238.113       176.117       21.627         19     
produce:gaushit:GausHitFinder                      1.4441        2.20507       3.24492       2.2302       0.460448        19     
produce:nhitsfilter:NumberOfHitsFilter           0.00049448    0.000652969   0.000906286   0.000626329   0.000113199      19     
produce:reco3d:SpacePointSolver                    12.1945       19.9916       30.7473       18.794        4.14468        19     
produce:hitpdune:DisambigFromSpacePoints          0.174499      0.315893      0.592174      0.287129      0.112182        19     
produce:cluster3d:Cluster3D                        1.4008        3.38516       13.323        2.71723       2.46907        19     
[art]:TriggerResults:TriggerResultInserter       2.5411e-05    4.22645e-05   9.3499e-05    3.6267e-05    1.61755e-05      19     
end_path:out1:RootOutput                          4.467e-06    1.93133e-05   0.000202108    8.093e-06    4.33632e-05      19     
end_path:out1:RootOutput(write)                    5.79357       6.76308       10.4687       6.16454       1.21348        19     
===================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5294.74 MB
  Peak resident set size usage (VmHWM): 3381.47 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002_reco_stage0_20260204T011311_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002_reco_stage0_20260204T011311_offline.root
Ran successfully
{
  "name": "np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002_reco_stage0_20260204T011311_offline.root",
  "namespace": "usertests",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v10_17_00d00",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage0_protodunevd_offline.fcl",
    "dune.campaign": "vd-protodune-det-reco-test",
    "core.start_time": 1770162985.0,
    "core.end_time": 1770167594.0,
    "core.events": [
      23376,
      23382,
      23388,
      23394,
      23400,
      23406,
      23412,
      23418,
      23424,
      23430,
      23436,
      23442,
      23448,
      23454,
      23460,
      23466,
      23472,
      23478,
      23484
    ],
    "core.event_count": 19,
    "core.first_event_number": 23376,
    "core.last_event_number": 23484,
    "core.data_stream": "cosmics",
    "core.file_type": "detector",
    "core.run_type": "vd-protodune",
    "core.runs": [
      42433
    ],
    "core.runs_subruns": [
      4243300001
    ]
  },
  "parents": [
    {
      "did": "vd-protodune:np02vd_raw_run042433_0205_df-s02-d0_dw_0_20260203T200002.hdf5"
    }
  ]
}
justIN time: 2026-02-04 07:45:06 UTC       justIN version: 01.06.00