Jobsub ID 36362.191@dunegpschedd02.fnal.gov
| Jobsub ID | 36362.191@dunegpschedd02.fnal.gov |
| Workflow ID | 2330 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| HTCondor Group | group_dune.prod.mcsim |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2025-09-08 12:47:59 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc01 |
| Last heartbeat | 2025-09-08 15:52:40 |
| From worker node | Hostname | wn-lb-02.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Started | 2025-09-08 14:44:30 |
| Input files | vd-protodune:np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 55m (3320s) |
| CPU time | 50m (3052s = 91%) |
| Max RSS bytes | 2736656384 (2609 MiB) |
| Outputting started | 2025-09-08 15:39:51 |
| Output files | |
| Finished | 2025-09-08 15:52:40 |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
s within desired window for channel 10269
wclsFrameSaver: no samples within desired window for channel 10269
wclsFrameSaver: no samples within desired window for channel 10419
wclsFrameSaver: no samples within desired window for channel 10502
wclsFrameSaver: no samples within desired window for channel 10503
wclsFrameSaver: no samples within desired window for channel 10503
wclsFrameSaver: no samples within desired window for channel 10790
wclsFrameSaver: no samples within desired window for channel 10914
wclsFrameSaver: no samples within desired window for channel 10983
wclsFrameSaver: no samples within desired window for channel 10985
wclsFrameSaver: no samples within desired window for channel 11252
wclsFrameSaver: no samples within desired window for channel 11252
wclsFrameSaver: no samples within desired window for channel 11253
wclsFrameSaver: no samples within desired window for channel 11253
wclsFrameSaver: no samples within desired window for channel 11254
wclsFrameSaver: no samples within desired window for channel 11254
wclsFrameSaver: no samples within desired window for channel 11255
wclsFrameSaver: no samples within desired window for channel 11256
wclsFrameSaver: no samples within desired window for channel 11256
wclsFrameSaver: no samples within desired window for channel 11256
wclsFrameSaver: no samples within desired window for channel 11257
wclsFrameSaver: no samples within desired window for channel 11516
wclsFrameSaver: no samples within desired window for channel 11516
wclsFrameSaver: no samples within desired window for channel 11516
wclsFrameSaver: no samples within desired window for channel 11687
wclsFrameSaver: no samples within desired window for channel 11688
wclsFrameSaver: no samples within desired window for channel 12041
wclsFrameSaver: no samples within desired window for channel 12041
wclsFrameSaver: no samples within desired window for channel 12042
wclsFrameSaver: no samples within desired window for channel 12043
wclsFrameSaver: no samples within desired window for channel 12043
wclsFrameSaver: no samples within desired window for channel 12044
wclsFrameSaver: no samples within desired window for channel 12044
wclsFrameSaver: no samples within desired window for channel 12193
wclsFrameSaver: no samples within desired window for channel 12193
wclsFrameSaver: no samples within desired window for channel 12195
wclsFrameSaver: no samples within desired window for channel 12196
wclsFrameSaver: no samples within desired window for channel 12196
wclsFrameSaver: no samples within desired window for channel 12196
FrameSaver: q=9.20134e+06 n=1022436 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 736 XUs and 847 XVs -> 21 XUVs
C:0 T:1 711 XUs and 899 XVs -> 33 XUVs
C:0 T:2 699 XUs and 684 XVs -> 12 XUVs
C:0 T:3 272 XUs and 339 XVs -> 19 XUVs
C:0 T:4 733 XUs and 709 XVs -> 23 XUVs
C:0 T:5 987 XUs and 797 XVs -> 72 XUVs
C:0 T:6 272 XUs and 804 XVs -> 41 XUVs
C:0 T:7 511 XUs and 645 XVs -> 15 XUVs
C:0 T:8 1242 XUs and 1454 XVs -> 81 XUVs
C:0 T:9 1579 XUs and 1891 XVs -> 112 XUVs
C:0 T:10 2590 XUs and 2919 XVs -> 115 XUVs
C:0 T:11 2247 XUs and 3077 XVs -> 147 XUVs
C:0 T:12 2476 XUs and 3086 XVs -> 191 XUVs
C:0 T:13 2731 XUs and 2871 XVs -> 263 XUVs
C:0 T:14 2004 XUs and 2659 XVs -> 118 XUVs
C:0 T:15 2641 XUs and 2807 XVs -> 199 XUVs
1462 XUVs total
1206 collection wire objects
1462 potential space points
Neighbour search...
12502 tests to find 7038 neighbours
Iterating with no regularization...
Begin: 3.93325e+08
0 3.8563e+08
1 3.85418e+08
Now with regularization...
Begin: 3.78753e+08
0 3.7873e+08
08-Sep-2025 17:39:09 CEST Closed output file "np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup.root"
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 121.909 170.008 228.998 168.978 29.7898 19
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read) 6.7722e-05 9.29223e-05 0.000232498 8.5938e-05 3.47574e-05 19
produce:tpcrawdecoder:PDVDTPCReader 15.0222 16.3345 17.8128 16.2537 0.792887 19
produce:triggerrawdecoder:PDVDTriggerReader4 0.558908 0.626928 0.77015 0.611391 0.0535146 19
produce:pdvddaphne:DAPHNEReaderPDVD 0.000231605 0.000357252 0.000770651 0.000326938 0.000124693 19
produce:ophit:OpHitFinder 0.00013487 0.000214707 0.000884351 0.000168572 0.00015989 19
produce:opflash:OpFlashFinderVerticalDrift 4.1171e-05 7.25093e-05 0.000345576 5.3428e-05 6.54373e-05 19
produce:wclsdatavd:WireCellToolkit 63.9101 74.1654 87.6874 73.3775 7.24824 19
produce:gaushit:GausHitFinder 0.825643 1.37684 2.12817 1.37077 0.326255 19
produce:nhitsfilter:NumberOfHitsFilter 0.000230434 0.000719407 0.00133985 0.000688229 0.000267258 19
produce:reco3d:SpacePointSolver 8.60677 16.4721 24.6906 15.3194 4.43213 19
produce:hitpdune:DisambigFromSpacePoints 0.125424 0.219866 0.335962 0.219813 0.0564412 19
produce:pandora:StandardPandora 20.7065 53.4444 95.5643 52.949 21.5399 19
produce:pandoraTrack:LArPandoraTrackCreation 0.59717 1.71408 3.51958 1.63443 0.823244 19
produce:pandoraGnocalo:GnocchiCalorimetry 0.0194416 0.0328177 0.0428792 0.0327914 0.00677224 19
[art]:TriggerResults:TriggerResultInserter 2.4757e-05 4.64023e-05 0.000135171 4.5597e-05 2.32814e-05 19
end_path:out1:RootOutput 6.1e-06 9.62716e-06 3.5302e-05 8.173e-06 6.20932e-06 19
end_path:out1:RootOutput(write) 4.49262 5.53187 6.39403 5.94426 0.736915 19
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4625.08 MB
Peak resident set size usage (VmHWM): 2736.66 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup.root
\tHists: np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup_hists.root
Forming reco metadata
Successfully opened file np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup.root
Ran successfully
{
"name": "np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757345950.0,
"core.end_time": 1757345950.0,
"core.events": [
1490,
1494,
1498,
1502,
1506,
1510,
1514,
1518,
1522,
1526,
1530,
1534,
1538,
1542,
1546,
1550,
1554,
1558,
1562
],
"core.event_count": 19,
"core.first_event_number": 1490,
"core.last_event_number": 1562,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404_reco_stage1_20250908T153909_keepup_hists.root",
"namespace": "vd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v10_10_00d00",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage1_protodunevd_keepup_all.fcl",
"dune.campaign": "vd-protodune-reco-keepup-v0",
"core.start_time": 1757345950.0,
"core.end_time": 1757345950.0,
"core.data_stream": "cosmics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "vd-protodune",
"core.runs": [
39327
],
"core.runs_subruns": [
3932700001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics"
},
"parents": [
{
"did": "vd-protodune:np02vd_raw_run039327_0019_df-s04-d2_dw_0_20250908T105404.hdf5"
}
]
} 