Jobsub ID 253747.0@dunegpschedd02.fnal.gov
| Jobsub ID | 253747.0@dunegpschedd02.fnal.gov |
| Workflow ID | 10802 |
| Stage ID | 1 |
| User name | ykermaid@fnal.gov |
| Requested | Processors | 4 |
| GPU | No |
| RSS bytes | 6815744000 (6500 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-11-27 05:53:19 |
| Site | NL_SURFsara |
| Entry | DUNE_SurfSARA_arc01 |
| Last heartbeat | 2025-11-27 05:57:17 |
| From worker node | Hostname | wn-la-19.gina.surf.nl |
| cpuinfo | AMD EPYC 9754 128-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 4 |
| RSS bytes | 7864320000 (7500 MiB) |
| Wall seconds limit | 129600 (36 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Started | 2025-11-27 05:54:11 |
| Input files | vd-protodune-top:3430_3_c_pvdt.test
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Max RSS bytes | 0 (0 MiB) |
| Outputting started | |
| Output files | |
| Finished | 2025-11-27 05:57:17 |
| Saved logs | justin-logs:253747.0-dunegpschedd02.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
within desired window for channel 5724
wclsFrameSaver: no samples within desired window for channel 5724
wclsFrameSaver: no samples within desired window for channel 5725
wclsFrameSaver: no samples within desired window for channel 5726
wclsFrameSaver: no samples within desired window for channel 5727
wclsFrameSaver: no samples within desired window for channel 5727
wclsFrameSaver: no samples within desired window for channel 5728
wclsFrameSaver: no samples within desired window for channel 5729
wclsFrameSaver: no samples within desired window for channel 5729
wclsFrameSaver: no samples within desired window for channel 5730
wclsFrameSaver: no samples within desired window for channel 5730
wclsFrameSaver: no samples within desired window for channel 5730
wclsFrameSaver: no samples within desired window for channel 5731
wclsFrameSaver: no samples within desired window for channel 5732
wclsFrameSaver: no samples within desired window for channel 5732
wclsFrameSaver: no samples within desired window for channel 5733
wclsFrameSaver: no samples within desired window for channel 5734
wclsFrameSaver: no samples within desired window for channel 5735
wclsFrameSaver: no samples within desired window for channel 5735
wclsFrameSaver: no samples within desired window for channel 5736
wclsFrameSaver: no samples within desired window for channel 5736
wclsFrameSaver: no samples within desired window for channel 5737
wclsFrameSaver: no samples within desired window for channel 5738
wclsFrameSaver: no samples within desired window for channel 5739
wclsFrameSaver: no samples within desired window for channel 5739
wclsFrameSaver: no samples within desired window for channel 5740
wclsFrameSaver: no samples within desired window for channel 5740
wclsFrameSaver: no samples within desired window for channel 5741
wclsFrameSaver: no samples within desired window for channel 5742
wclsFrameSaver: no samples within desired window for channel 5742
wclsFrameSaver: no samples within desired window for channel 5742
wclsFrameSaver: no samples within desired window for channel 5743
wclsFrameSaver: no samples within desired window for channel 5744
wclsFrameSaver: no samples within desired window for channel 5745
wclsFrameSaver: no samples within desired window for channel 5745
wclsFrameSaver: no samples within desired window for channel 5746
wclsFrameSaver: no samples within desired window for channel 5747
wclsFrameSaver: no samples within desired window for channel 5748
wclsFrameSaver: no samples within desired window for channel 5852
wclsFrameSaver: no samples within desired window for channel 5853
wclsFrameSaver: no samples within desired window for channel 5854
wclsFrameSaver: no samples within desired window for channel 5854
wclsFrameSaver: no samples within desired window for channel 5854
wclsFrameSaver: no samples within desired window for channel 5855
wclsFrameSaver: no samples within desired window for channel 5855
wclsFrameSaver: no samples within desired window for channel 5856
wclsFrameSaver: no samples within desired window for channel 5857
wclsFrameSaver: no samples within desired window for channel 5858
wclsFrameSaver: no samples within desired window for channel 5859
wclsFrameSaver: no samples within desired window for channel 5861
wclsFrameSaver: no samples within desired window for channel 6113
wclsFrameSaver: no samples within desired window for channel 6115
wclsFrameSaver: no samples within desired window for channel 6115
wclsFrameSaver: no samples within desired window for channel 6116
wclsFrameSaver: no samples within desired window for channel 6116
wclsFrameSaver: no samples within desired window for channel 6116
wclsFrameSaver: no samples within desired window for channel 6116
wclsFrameSaver: no samples within desired window for channel 6116
wclsFrameSaver: no samples within desired window for channel 6117
wclsFrameSaver: no samples within desired window for channel 6117
wclsFrameSaver: no samples within desired window for channel 6117
wclsFrameSaver: no samples within desired window for channel 6118
wclsFrameSaver: no samples within desired window for channel 6118
wclsFrameSaver: no samples within desired window for channel 6118
wclsFrameSaver: no samples within desired window for channel 6118
wclsFrameSaver: no samples within desired window for channel 6118
wclsFrameSaver: no samples within desired window for channel 6119
wclsFrameSaver: no samples within desired window for channel 6119
wclsFrameSaver: no samples within desired window for channel 6119
wclsFrameSaver: no samples within desired window for channel 6119
wclsFrameSaver: no samples within desired window for channel 6119
wclsFrameSaver: no samples within desired window for channel 6120
wclsFrameSaver: no samples within desired window for channel 6120
FrameSaver: q=8.9151e+07 n=1700725 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1569 XUs and 4677 XVs -> 15 XUVs
C:0 T:1 766 XUs and 530 XVs -> 0 XUVs
C:0 T:2 84 XUs and 51 XVs -> 0 XUVs
C:0 T:3 695 XUs and 641 XVs -> 1 XUVs
C:0 T:4 881 XUs and 823 XVs -> 2 XUVs
C:0 T:5 686 XUs and 781 XVs -> 2 XUVs
C:0 T:6 894 XUs and 1025 XVs -> 2 XUVs
C:0 T:7 758 XUs and 801 XVs -> 3 XUVs
25 XUVs total
24 collection wire objects
25 potential space points
Neighbour search...
51 tests to find 18 neighbours
Iterating with no regularization...
Begin: 3.35445e+10
0 3.35445e+10
Now with regularization...
Begin: 3.35413e+10
0 3.35413e+10
27-Nov-2025 06:56:25 CET Opened output file with pattern "%ifb_reco_stage1_%tc_laser_reco.root"
27-Nov-2025 06:56:31 CET Closed output file "3430_3_c_pvdt_reco_stage1_20251127T055631_laser_reco.root"
DataPrepModule::endJob: # events processed: 1
DataPrepModule::endJob: # events skipped: 0
==================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
==================================================================================================================================
Full event 83.1495 83.1495 83.1495 83.1495 0 1
----------------------------------------------------------------------------------------------------------------------------------
source:VDColdboxTDERawInput(read) 0.000203724 0.000203724 0.000203724 0.000203724 0 1
produce:caldata:DataPrepModule 0.68391 0.68391 0.68391 0.68391 0 1
produce:wclsdatavd:WireCellToolkit 52.493 52.493 52.493 52.493 0 1
produce:gaushit:GausHitFinder 2.31392 2.31392 2.31392 2.31392 0 1
produce:reco3d:SpacePointSolver 7.55456 7.55456 7.55456 7.55456 0 1
produce:hitpdune:DisambigFromSpacePoints 0.0509632 0.0509632 0.0509632 0.0509632 0 1
produce:pandora:StandardPandora 16.7076 16.7076 16.7076 16.7076 0 1
produce:pandoraTrack:LArPandoraTrackCreation 0.185384 0.185384 0.185384 0.185384 0 1
produce:pandoraGnocalo:GnocchiCalorimetry 0.0172462 0.0172462 0.0172462 0.0172462 0 1
[art]:TriggerResults:TriggerResultInserter 6.8361e-05 6.8361e-05 6.8361e-05 6.8361e-05 0 1
end_path:out1:RootOutput 2.2252e-05 2.2252e-05 2.2252e-05 2.2252e-05 0 1
end_path:out1:RootOutput(write) 3.13052 3.13052 3.13052 3.13052 0 1
==================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4004.52 MB
Peak resident set size usage (VmHWM): 1761.75 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: 3430_3_c_pvdt_reco_stage1_20251127T055631_laser_reco.root
\tHists:
Forming reco metadata
Successfully opened file 3430_3_c_pvdt_reco_stage1_20251127T055631_laser_reco.root
Traceback (most recent call last):
File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/71f643ddd59465043e3cd3712a1e24b9cd0fa631/meta_maker.py", line 48, in <module>
results = inherit_metadata.inherit(args.parent)
File "/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/71f643ddd59465043e3cd3712a1e24b9cd0fa631/inherit_metadata.py", line 72, in inherit
'metadata':get_parent_md(parent_name),
File "/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/71f643ddd59465043e3cd3712a1e24b9cd0fa631/inherit_metadata.py", line 33, in get_parent_md
check_md(required_keys, parent_md, parent_name)
File "/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/71f643ddd59465043e3cd3712a1e24b9cd0fa631/inherit_metadata.py", line 20, in check_md
raise Exception( ##TODO -- better exception
Exception: Error! The following required keys are missing from metadata of vd-protodune-top:3430_3_c_pvdt.test
core.file_content_status, dune.daq_test
Error in reco metadata 