Jobsub ID 294228.2@dunegpschedd02.fnal.gov
| Jobsub ID | 294228.2@dunegpschedd02.fnal.gov |
| Workflow ID | 12303 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2026-01-23 10:42:09 |
| Site | UK_Edinburgh |
| Entry | DUNE_UK_SGridECDF_ce1_multicore |
| Last heartbeat | 2026-01-23 13:38:44 |
| From worker node | Hostname | node2b04.ecdf.ed.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2026-01-23 10:42:43 |
| Input files | vd-protodune:np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 2h (10415s) |
| CPU time | 1h (4214s = 40%) |
| Max RSS bytes | 3685433344 (3514 MiB) |
| Outputting started | 2026-01-23 13:36:20 |
| Output files | |
| Finished | 2026-01-23 13:38:44 |
| Saved logs | justin-logs:294228.2-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
on in looking up space point ptr... 0x5140b080, 0x571a5050
Caught exception in looking up space point ptr... 0x67ffc070, 0x324e13f0
Caught exception in looking up space point ptr... 0x324e13f0, 0x67ffc070
Caught exception in looking up space point ptr... 0x59bf7b80, 0x9d9b880
Caught exception in looking up space point ptr... 0x9d9b880, 0x3292d010
Caught exception in looking up space point ptr... 0x11ec4250, 0x4df55120
Caught exception in looking up space point ptr... 0x4df55120, 0x46b34ab0
Caught exception in looking up space point ptr... 0x72501890, 0x72502060
Caught exception in looking up space point ptr... 0x72502060, 0x72501890
Caught exception in looking up space point ptr... 0x109aca30, 0x109acee0
Caught exception in looking up space point ptr... 0x109acee0, 0x109aca30
Caught exception in looking up space point ptr... 0x109aca30, 0x109acd50
Caught exception in looking up space point ptr... 0x109acd50, 0x109aca30
Caught exception in looking up space point ptr... 0x109aca30, 0x109ac8a0
Caught exception in looking up space point ptr... 0x109ac8a0, 0x109aca30
Caught exception in looking up space point ptr... 0x39e702f0, 0x39e70100
Caught exception in looking up space point ptr... 0x39e70100, 0x5e4f7f10
Caught exception in looking up space point ptr... 0x53956800, 0x53956cb0
Caught exception in looking up space point ptr... 0x53956cb0, 0x53956800
Caught exception in looking up space point ptr... 0x53956800, 0x53956b20
Caught exception in looking up space point ptr... 0x53956b20, 0x53956800
Caught exception in looking up space point ptr... 0x53956800, 0x53956670
Caught exception in looking up space point ptr... 0x53956670, 0x53956800
Caught exception in looking up space point ptr... 0x5394cbc0, 0x5394cd50
Caught exception in looking up space point ptr... 0x5394cd50, 0x5394cbc0
Caught exception in looking up space point ptr... 0x24db56d0, 0x44227090
Caught exception in looking up space point ptr... 0x44227090, 0x165b3bf0
Caught exception in looking up space point ptr... 0x61689c50, 0x146ee5d0
Caught exception in looking up space point ptr... 0x146ee5d0, 0x50170c20
++++>>>> total num hits: 124486, num free: 117439
23-Jan-2026 13:36:12 GMT Closed output file "np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452_reco_stage0_20260123T133612_offline.root"
[2026-01-23 13:36:12.574211 +0000][Debug ][File ][ 1143] [0x88f22e0@root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/37/82/np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452.hdf5?xrdcl.requuid=f2e9d0e3-6050-4822-b905-9bd5d9bad299] Sending a close command for handle 0x0 to st-096-ee8093d0.cern.ch:1095
[2026-01-23 13:36:12.576923 +0000][Debug ][ExDbgMsg ][ 1143] [st-096-ee8093d0.cern.ch:1095] MsgHandler created: 0x575463d0 (message: kXR_close (handle: 0x00000000) ).
[2026-01-23 13:36:12.578677 +0000][Debug ][ExDbgMsg ][ 1143] [st-096-ee8093d0.cern.ch:1095] Moving MsgHandler: 0x575463d0 (message: kXR_close (handle: 0x00000000) ) from out-queu to in-queue.
[2026-01-23 13:36:12.607849 +0000][Debug ][ExDbgMsg ][ 1143] [msg: 0x454334a0] Assigned MsgHandler: 0x575463d0.
[2026-01-23 13:36:12.607889 +0000][Debug ][ExDbgMsg ][ 1143] [handler: 0x575463d0] Removed MsgHandler: 0x575463d0 from the in-queue.
[2026-01-23 13:36:12.608016 +0000][Debug ][ExDbgMsg ][ 1143] [st-096-ee8093d0.cern.ch:1095] Calling MsgHandler: 0x575463d0 (message: kXR_close (handle: 0x00000000) ) with status: [SUCCESS] .
[2026-01-23 13:36:12.608057 +0000][Debug ][File ][ 1143] [0x88f22e0@root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/vd-protodune/37/82/np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452.hdf5?xrdcl.requuid=f2e9d0e3-6050-4822-b905-9bd5d9bad299] Close returned from st-096-ee8093d0.cern.ch:1095 with: [SUCCESS]
[2026-01-23 13:36:12.608083 +0000][Debug ][ExDbgMsg ][ 1143] [st-096-ee8093d0.cern.ch:1095] Destroying MsgHandler: 0x575463d0.
===================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
===================================================================================================================================
Full event 342.366 449.203 583.941 461.554 74.1741 23
-----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.0759e-05 7.76104e-05 0.000137894 7.0739e-05 1.87638e-05 23
produce:tpcrawdecoder:PDVDTPCReader 179.571 270.884 386.29 262.692 66.9336 23
produce:triggerrawdecoder:PDVDTriggerReader4 1.23343 1.85974 4.21237 1.754 0.636345 23
produce:timingrawdecoder:PDHDTimingRawDecoder 0.0279066 0.0431553 0.086758 0.0392783 0.0158943 23
produce:ctbrawdecoder:PDHDCTBRawDecoder 0.13915 0.142678 0.168126 0.140183 0.00775696 23
produce:beamevent:BeamEvent 7.7587e-05 0.000109585 0.000581095 8.8739e-05 0.000100761 23
produce:pdvddaphne:DAPHNEReaderPDVD 0.000365269 0.000395544 0.000611672 0.000386276 4.69101e-05 23
produce:ophit:OpHitFinder 5.4454e-05 7.81599e-05 0.000397887 6.1602e-05 6.85372e-05 23
produce:wclsdatavd:WireCellToolkit 105.555 117.119 128.658 118.4 6.07915 23
produce:gaushit:GausHitFinder 1.25798 6.16181 11.9056 7.99731 3.3233 23
produce:nhitsfilter:NumberOfHitsFilter 0.000326975 0.00060479 0.000807018 0.000675477 0.000150656 23
produce:reco3d:SpacePointSolver 1.85199 10.6675 23.8102 11.1813 7.26411 23
produce:hitpdune:DisambigFromSpacePoints 0.157034 0.828386 1.72973 0.932341 0.456149 23
produce:cluster3d:Cluster3D 1.80499 35.8849 190.797 15.328 49.5194 23
[art]:TriggerResults:TriggerResultInserter 2.3815e-05 3.22092e-05 8.4489e-05 3.1011e-05 1.16162e-05 23
end_path:out1:RootOutput 4.593e-06 5.86561e-06 1.7917e-05 5.34e-06 2.60073e-06 23
end_path:out1:RootOutput(write) 4.63475 5.57668 8.52533 5.10826 1.04902 23
===================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5606.07 MB
Peak resident set size usage (VmHWM): 3685.43 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
[2026-01-23 13:36:14.517756 +0000][Debug ][JobMgr ][ 1143] Stopping the job manager...
[2026-01-23 13:36:14.518265 +0000][Debug ][JobMgr ][ 1143] Job manager stopped
[2026-01-23 13:36:14.518640 +0000][Debug ][TaskMgr ][ 1143] Stopping the task manager...
[2026-01-23 13:36:14.518920 +0000][Debug ][TaskMgr ][ 1143] Task manager stopped
[2026-01-23 13:36:14.519166 +0000][Debug ][Poller ][ 1143] Stopping the poller...
[2026-01-23 13:36:14.519611 +0000][Debug ][AsyncSock ][ 1143] [st-096-ee8093d0.cern.ch:1095.0] Closing the socket
[2026-01-23 13:36:14.519915 +0000][Debug ][Poller ][ 1143] <[::ffff:192.41.105.37]:46094><--><[::ffff:128.142.52.213]:1095> Removing socket from the poller
[2026-01-23 13:36:14.520093 +0000][Debug ][PostMaster ][ 1143] [st-096-ee8093d0.cern.ch:1095] Destroying stream
[2026-01-23 13:36:14.520334 +0000][Debug ][AsyncSock ][ 1143] [st-096-ee8093d0.cern.ch:1095.0] Closing the socket
Output files:
\tReco: np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452_reco_stage0_20260123T133612_offline.root
Forming reco metadata
Successfully opened file np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452_reco_stage0_20260123T133612_offline.root
Ran successfully
{
"name": "np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452_reco_stage0_20260123T133612_offline.root",
"namespace": "usertests",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_17_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage0_protodunevd_offline_hv220.fcl",
"dune.campaign": "vd-protodune-det-reco-test",
"core.start_time": 1769164973.0,
"core.end_time": 1769175374.0,
"core.events": [
6081,
6085,
6089,
6093,
6097,
6101,
6105,
6109,
6113,
6117,
6121,
6125,
6129,
6133,
6137,
6141,
6145,
6149,
6153,
6157,
6161,
6165,
6169
],
"core.event_count": 23,
"core.first_event_number": 6081,
"core.last_event_number": 6169,
"core.data_stream": "cosmics",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
40362
],
"core.runs_subruns": [
4036200001
]
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run040362_0065_df-s04-d1_dw_0_20251105T101452.hdf5"
}
]
} 