Jobsub ID 36362.154@dunegpschedd02.fnal.gov
| Jobsub ID | 36362.154@dunegpschedd02.fnal.gov |
| Workflow ID | 2330 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 12:47:59 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc03 |
| Last heartbeat | 2025-09-08 15:52:41 |
| From worker node | Hostname | wn-lb-08.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:44:09 |
| Input files | vd-protodune:np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 57m (3463s) |
| CPU time | 52m (3153s = 91%) |
| Max RSS bytes | 2768388096 (2640 MiB) |
| Outputting started | 2025-09-08 15:41:53 |
| Output files | |
| Finished | 2025-09-08 15:52:41 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
6
wclsFrameSaver: no samples within desired window for channel 9598
wclsFrameSaver: no samples within desired window for channel 9599
wclsFrameSaver: no samples within desired window for channel 9599
wclsFrameSaver: no samples within desired window for channel 9601
wclsFrameSaver: no samples within desired window for channel 9856
wclsFrameSaver: no samples within desired window for channel 9858
wclsFrameSaver: no samples within desired window for channel 9871
wclsFrameSaver: no samples within desired window for channel 9871
wclsFrameSaver: no samples within desired window for channel 10218
wclsFrameSaver: no samples within desired window for channel 10303
wclsFrameSaver: no samples within desired window for channel 10366
wclsFrameSaver: no samples within desired window for channel 10423
wclsFrameSaver: no samples within desired window for channel 10426
wclsFrameSaver: no samples within desired window for channel 10466
wclsFrameSaver: no samples within desired window for channel 10526
wclsFrameSaver: no samples within desired window for channel 10528
wclsFrameSaver: no samples within desired window for channel 10529
wclsFrameSaver: no samples within desired window for channel 10548
wclsFrameSaver: no samples within desired window for channel 10823
wclsFrameSaver: no samples within desired window for channel 10865
wclsFrameSaver: no samples within desired window for channel 10865
wclsFrameSaver: no samples within desired window for channel 10899
wclsFrameSaver: no samples within desired window for channel 10915
wclsFrameSaver: no samples within desired window for channel 10938
wclsFrameSaver: no samples within desired window for channel 11638
wclsFrameSaver: no samples within desired window for channel 11640
wclsFrameSaver: no samples within desired window for channel 11641
wclsFrameSaver: no samples within desired window for channel 11641
wclsFrameSaver: no samples within desired window for channel 11642
wclsFrameSaver: no samples within desired window for channel 11643
wclsFrameSaver: no samples within desired window for channel 11643
wclsFrameSaver: no samples within desired window for channel 11735
wclsFrameSaver: no samples within desired window for channel 11735
wclsFrameSaver: no samples within desired window for channel 11736
wclsFrameSaver: no samples within desired window for channel 11825
wclsFrameSaver: no samples within desired window for channel 11825
wclsFrameSaver: no samples within desired window for channel 11826
wclsFrameSaver: no samples within desired window for channel 11826
wclsFrameSaver: no samples within desired window for channel 11827
FrameSaver: q=6.51739e+06 n=720436 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 366 XUs and 393 XVs -> 12 XUVs
C:0 T:1 391 XUs and 347 XVs -> 26 XUVs
C:0 T:2 843 XUs and 656 XVs -> 130 XUVs
C:0 T:3 711 XUs and 680 XVs -> 30 XUVs
C:0 T:4 1739 XUs and 2203 XVs -> 105 XUVs
C:0 T:5 296 XUs and 315 XVs -> 20 XUVs
C:0 T:6 1328 XUs and 1472 XVs -> 111 XUVs
C:0 T:7 257 XUs and 459 XVs -> 14 XUVs
C:0 T:8 1342 XUs and 1654 XVs -> 67 XUVs
C:0 T:9 810 XUs and 1737 XVs -> 65 XUVs
C:0 T:10 1564 XUs and 1812 XVs -> 146 XUVs
C:0 T:11 2399 XUs and 2665 XVs -> 117 XUVs
C:0 T:12 1706 XUs and 1997 XVs -> 110 XUVs
C:0 T:13 1440 XUs and 1868 XVs -> 105 XUVs
C:0 T:14 1887 XUs and 2090 XVs -> 79 XUVs
C:0 T:15 1243 XUs and 1604 XVs -> 91 XUVs
1228 XUVs total
954 collection wire objects
1228 potential space points
Neighbour search...
15096 tests to find 8712 neighbours
Iterating with no regularization...
Begin: 4.8203e+08
0 4.73534e+08
1 4.73452e+08
Now with regularization...
Begin: 4.63422e+08
0 4.63415e+08
08-Sep-2025 17:40:22 CEST Closed output file "np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154022_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 134.929 174.712 212.784 179.103 20.2359 19
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 7.9669e-05 0.000106375 0.00026735 8.8933e-05 4.751e-05 19
produce:tpcrawdecoder:PDVDTPCReader 14.4509 15.9468 18.0991 15.876 0.763773 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.560849 0.63565 0.726948 0.617565 0.0444946 19
produce:pdvddaphne:DAPHNEReaderPDVD 0.000277986 0.000345013 0.000708341 0.000316816 9.18678e-05 19
produce:ophit:OpHitFinder 0.000146209 0.000211265 0.00084125 0.000174692 0.000150199 19
produce:opflash:OpFlashFinderVerticalDrift 4.8944e-05 7.34591e-05 0.000345678 5.5944e-05 6.47164e-05 19
produce:wclsdatavd:WireCellToolkit 69.8712 82.3393 102.92 80.8076 9.00638 19
produce:gaushit:GausHitFinder 0.750157 1.41563 1.81336 1.48562 0.297832 19
produce:nhitsfilter:NumberOfHitsFilter 0.000282133 0.000743113 0.00132159 0.000692237 0.000245797 19
produce:reco3d:SpacePointSolver 9.22739 16.4777 21.5833 17.1767 3.21811 19
produce:hitpdune:DisambigFromSpacePoints 0.109519 0.217374 0.304036 0.224285 0.0480695 19
produce:pandora:StandardPandora 31.2107 50.2292 72.9531 49.4368 12.9606 19
produce:pandoraTrack:LArPandoraTrackCreation 0.788594 1.40176 2.25661 1.25209 0.498996 19
produce:pandoraGnocalo:GnocchiCalorimetry 0.0258431 0.032511 0.0430693 0.0318171 0.00487544 19
[art]:TriggerResults:TriggerResultInserter 2.9084e-05 4.27957e-05 0.000111697 3.6946e-05 1.82e-05 19
end_path:out1:RootOutput 6.591e-06 9.50937e-06 3.2388e-05 7.951e-06 5.52265e-06 19
end_path:out1:RootOutput(write) 4.66874 5.93464 6.38141 6.16603 0.508903 19
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4658.48 MB
Peak resident set size usage (VmHWM): 2768.39 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154022_keepup.root
\tHists: np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154023_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154022_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154022_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346023.0,
"core.end_time": 1757346023.0,
"core.events": [
860,
864,
868,
872,
876,
880,
884,
888,
892,
896,
900,
904,
908,
912,
916,
920,
924,
928,
932
],
"core.event_count": 19,
"core.first_event_number": 860,
"core.last_event_number": 932,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350_reco_stage1_20250908T154023_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757346023.0,
"core.end_time": 1757346023.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0011_df-s04-d0_dw_0_20250908T104350.hdf5"
}
]
} 