Jobsub ID 36362.93@dunegpschedd02.fnal.gov
| Jobsub ID | 36362.93@dunegpschedd02.fnal.gov |
| Workflow ID | 2330 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 12:47:59 |
| Site | UK_RAL-Tier1 |
| Entry | LIGO_UK_RAL_arc_ce03 |
| Last heartbeat | 2025-09-08 15:52:47 |
| From worker node | Hostname | dune001-8551170.0-lcg2702.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 9654 96-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:44:16 |
| Input files | vd-protodune:np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 57m (3470s) |
| CPU time | 50m (3018s = 86%) |
| Max RSS bytes | 2891616256 (2757 MiB) |
| Outputting started | 2025-09-08 15:42:06 |
| Output files | |
| Finished | 2025-09-08 15:52:47 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
desired window for channel 7461
wclsFrameSaver: no samples within desired window for channel 7808
wclsFrameSaver: no samples within desired window for channel 8056
wclsFrameSaver: no samples within desired window for channel 8056
wclsFrameSaver: no samples within desired window for channel 8057
wclsFrameSaver: no samples within desired window for channel 8058
wclsFrameSaver: no samples within desired window for channel 8059
wclsFrameSaver: no samples within desired window for channel 8060
wclsFrameSaver: no samples within desired window for channel 8061
wclsFrameSaver: no samples within desired window for channel 9584
wclsFrameSaver: no samples within desired window for channel 9735
wclsFrameSaver: no samples within desired window for channel 9736
wclsFrameSaver: no samples within desired window for channel 9737
wclsFrameSaver: no samples within desired window for channel 9737
wclsFrameSaver: no samples within desired window for channel 9738
wclsFrameSaver: no samples within desired window for channel 9738
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9749
wclsFrameSaver: no samples within desired window for channel 9750
wclsFrameSaver: no samples within desired window for channel 9750
wclsFrameSaver: no samples within desired window for channel 10301
wclsFrameSaver: no samples within desired window for channel 10329
wclsFrameSaver: no samples within desired window for channel 10380
wclsFrameSaver: no samples within desired window for channel 10763
wclsFrameSaver: no samples within desired window for channel 10765
wclsFrameSaver: no samples within desired window for channel 10766
wclsFrameSaver: no samples within desired window for channel 10767
wclsFrameSaver: no samples within desired window for channel 10768
wclsFrameSaver: no samples within desired window for channel 10769
wclsFrameSaver: no samples within desired window for channel 10769
wclsFrameSaver: no samples within desired window for channel 10770
wclsFrameSaver: no samples within desired window for channel 10794
wclsFrameSaver: no samples within desired window for channel 10794
wclsFrameSaver: no samples within desired window for channel 10817
wclsFrameSaver: no samples within desired window for channel 10930
wclsFrameSaver: no samples within desired window for channel 11710
wclsFrameSaver: no samples within desired window for channel 11729
FrameSaver: q=8.32136e+06 n=1004329 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 678 XUs and 971 XVs -> 33 XUVs
C:0 T:1 1164 XUs and 984 XVs -> 31 XUVs
C:0 T:2 2120 XUs and 2436 XVs -> 111 XUVs
C:0 T:3 1601 XUs and 2328 XVs -> 141 XUVs
C:0 T:4 859 XUs and 955 XVs -> 43 XUVs
C:0 T:5 550 XUs and 783 XVs -> 43 XUVs
C:0 T:6 2440 XUs and 2947 XVs -> 262 XUVs
C:0 T:7 2126 XUs and 3081 XVs -> 168 XUVs
C:0 T:8 2589 XUs and 2663 XVs -> 153 XUVs
C:0 T:9 1310 XUs and 1370 XVs -> 91 XUVs
C:0 T:10 2285 XUs and 3082 XVs -> 116 XUVs
C:0 T:11 1280 XUs and 1114 XVs -> 60 XUVs
C:0 T:12 167 XUs and 135 XVs -> 7 XUVs
C:0 T:13 3410 XUs and 2581 XVs -> 150 XUVs
C:0 T:14 142 XUs and 159 XVs -> 14 XUVs
C:0 T:15 2575 XUs and 1893 XVs -> 158 XUVs
1581 XUVs total
1184 collection wire objects
1581 potential space points
Neighbour search...
18527 tests to find 8240 neighbours
Iterating with no regularization...
Begin: 4.35448e+08
0 4.24218e+08
1 4.2384e+08
Now with regularization...
Begin: 4.13185e+08
0 4.13164e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
08-Sep-2025 15:40:50 UTC Closed output file "np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 63.8092 166.801 222.348 158.139 35.1168 20
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 8.8663e-05 0.000134193 0.000428164 0.000115169 7.11648e-05 20
produce:tpcrawdecoder:PDVDTPCReader 11.5345 19.9637 24.8221 20.4439 2.41207 20
produce:triggerrawdecoder:PDVDTriggerReader4 0.688042 0.710198 0.73821 0.712169 0.015313 20
produce:pdvddaphne:DAPHNEReaderPDVD 0.000238598 0.000513424 0.00221318 0.000406161 0.000404445 20
produce:ophit:OpHitFinder 0.000239791 0.000536335 0.0025999 0.000322074 0.000550757 20
produce:opflash:OpFlashFinderVerticalDrift 5.6224e-05 0.000160784 0.00099722 0.000107021 0.000201077 20
produce:wclsdatavd:WireCellToolkit 30.7653 71.6484 97.6679 69.4518 12.7318 20
produce:gaushit:GausHitFinder 0.914607 1.45182 2.11644 1.36415 0.344682 20
produce:nhitsfilter:NumberOfHitsFilter 0.000291318 0.000695782 0.00157555 0.000635956 0.000332145 20
produce:reco3d:SpacePointSolver 5.73184 14.9056 22.2347 14.4189 4.21028 20
produce:hitpdune:DisambigFromSpacePoints 0.104737 0.22213 0.407236 0.214677 0.0819766 20
produce:pandora:StandardPandora 11.1264 51.0971 79.3124 44.4569 18.0721 20
produce:pandoraTrack:LArPandoraTrackCreation 0.716442 1.50445 3.51054 1.3118 0.647732 20
produce:pandoraGnocalo:GnocchiCalorimetry 0.021992 0.0390104 0.0520254 0.0407116 0.00868952 20
[art]:TriggerResults:TriggerResultInserter 3.4081e-05 6.12275e-05 0.00021224 5.1923e-05 3.81764e-05 20
end_path:out1:RootOutput 5.489e-06 1.7258e-05 7.332e-05 1.39355e-05 1.38942e-05 20
end_path:out1:RootOutput(write) 1.67762 5.16639 5.70981 5.4153 0.848293 20
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5104.83 MB
Peak resident set size usage (VmHWM): 2891.62 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup.root
\tHists: np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346051.0,
"core.end_time": 1757346051.0,
"core.events": [
1089,
1093,
1097,
1101,
1105,
1109,
1113,
1117,
1121,
1125,
1129,
1133,
1137,
1141,
1145,
1149,
1153,
1157,
1161,
1165
],
"core.event_count": 20,
"core.first_event_number": 1089,
"core.last_event_number": 1165,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732_reco_stage1_20250908T154050_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346051.0,
"core.end_time": 1757346051.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0014_df-s04-d1_dw_0_20250908T104732.hdf5"
}
]
} 