justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 32473.6@dunegpschedd02.fnal.gov

Jobsub ID32473.6@dunegpschedd02.fnal.gov
Workflow ID2135
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-02 21:47:08
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-09-02 23:41:25
From worker nodeHostnameheplnc160.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-09-02 21:50:32
Input filesvd-protodune:np02vd_raw_run039275_0447_df-s02-d1_dw_0_20250902T052648.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-09-02 23:41:25
Saved logsjustin-logs:32473.6-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

08
wclsFrameSaver: no samples within desired window for channel 8105
wclsFrameSaver: no samples within desired window for channel 8259
wclsFrameSaver: no samples within desired window for channel 8335
wclsFrameSaver: no samples within desired window for channel 8336
wclsFrameSaver: no samples within desired window for channel 8337
wclsFrameSaver: no samples within desired window for channel 8337
wclsFrameSaver: no samples within desired window for channel 8338
wclsFrameSaver: no samples within desired window for channel 8338
wclsFrameSaver: no samples within desired window for channel 8338
wclsFrameSaver: no samples within desired window for channel 8339
wclsFrameSaver: no samples within desired window for channel 8339
wclsFrameSaver: no samples within desired window for channel 8604
wclsFrameSaver: no samples within desired window for channel 8632
wclsFrameSaver: no samples within desired window for channel 8633
wclsFrameSaver: no samples within desired window for channel 8633
wclsFrameSaver: no samples within desired window for channel 8634
wclsFrameSaver: no samples within desired window for channel 8634
wclsFrameSaver: no samples within desired window for channel 8635
wclsFrameSaver: no samples within desired window for channel 8635
wclsFrameSaver: no samples within desired window for channel 8636
wclsFrameSaver: no samples within desired window for channel 8636
wclsFrameSaver: no samples within desired window for channel 8637
wclsFrameSaver: no samples within desired window for channel 8637
wclsFrameSaver: no samples within desired window for channel 8637
wclsFrameSaver: no samples within desired window for channel 8638
wclsFrameSaver: no samples within desired window for channel 8638
wclsFrameSaver: no samples within desired window for channel 8639
wclsFrameSaver: no samples within desired window for channel 8639
wclsFrameSaver: no samples within desired window for channel 8856
wclsFrameSaver: no samples within desired window for channel 8866
wclsFrameSaver: no samples within desired window for channel 8866
wclsFrameSaver: no samples within desired window for channel 8866
wclsFrameSaver: no samples within desired window for channel 8868
wclsFrameSaver: no samples within desired window for channel 8868
wclsFrameSaver: no samples within desired window for channel 9012
wclsFrameSaver: no samples within desired window for channel 9012
wclsFrameSaver: no samples within desired window for channel 9012
wclsFrameSaver: no samples within desired window for channel 9078
wclsFrameSaver: no samples within desired window for channel 9078
wclsFrameSaver: no samples within desired window for channel 9079
wclsFrameSaver: no samples within desired window for channel 9079
wclsFrameSaver: no samples within desired window for channel 9080
wclsFrameSaver: no samples within desired window for channel 9081
wclsFrameSaver: no samples within desired window for channel 9082
wclsFrameSaver: no samples within desired window for channel 9083
wclsFrameSaver: no samples within desired window for channel 9083
wclsFrameSaver: no samples within desired window for channel 9318
wclsFrameSaver: no samples within desired window for channel 9502
wclsFrameSaver: no samples within desired window for channel 9907
wclsFrameSaver: no samples within desired window for channel 9978
wclsFrameSaver: no samples within desired window for channel 9980
wclsFrameSaver: no samples within desired window for channel 9981
wclsFrameSaver: no samples within desired window for channel 10302
wclsFrameSaver: no samples within desired window for channel 10483
wclsFrameSaver: no samples within desired window for channel 10484
wclsFrameSaver: no samples within desired window for channel 10575
wclsFrameSaver: no samples within desired window for channel 10710
wclsFrameSaver: no samples within desired window for channel 10765
wclsFrameSaver: no samples within desired window for channel 10765
wclsFrameSaver: no samples within desired window for channel 10907
wclsFrameSaver: no samples within desired window for channel 10937
wclsFrameSaver: no samples within desired window for channel 10947
wclsFrameSaver: no samples within desired window for channel 10948
wclsFrameSaver: no samples within desired window for channel 10948
wclsFrameSaver: no samples within desired window for channel 10950
wclsFrameSaver: no samples within desired window for channel 10951
wclsFrameSaver: no samples within desired window for channel 11583
wclsFrameSaver: no samples within desired window for channel 11583
wclsFrameSaver: no samples within desired window for channel 11585
wclsFrameSaver: no samples within desired window for channel 11585
wclsFrameSaver: no samples within desired window for channel 11585
wclsFrameSaver: no samples within desired window for channel 11990
wclsFrameSaver: no samples within desired window for channel 12111
wclsFrameSaver: no samples within desired window for channel 12115
FrameSaver: q=8.97763e+06 n=1021003 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 1939 XUs and 2367 XVs -> 123 XUVs
C:0 T:1 1485 XUs and 1489 XVs -> 88 XUVs
C:0 T:2 979 XUs and 1145 XVs -> 39 XUVs
C:0 T:3 183 XUs and 174 XVs -> 11 XUVs
C:0 T:4 124 XUs and 225 XVs -> 14 XUVs
C:0 T:5 1790 XUs and 1963 XVs -> 90 XUVs
C:0 T:6 270 XUs and 226 XVs -> 20 XUVs
C:0 T:7 117 XUs and 372 XVs -> 13 XUVs
C:0 T:8 15178 XUs and 40870 XVs -> 4633 XUVs
C:0 T:9 2086 XUs and 4477 XVs -> 394 XUVs
C:0 T:10 2800 XUs and 3629 XVs -> 234 XUVs
C:0 T:11 1853 XUs and 1970 XVs -> 55 XUVs
C:0 T:12 1428 XUs and 1835 XVs -> 130 XUVs
C:0 T:13 5665 XUs and 6446 XVs -> 1652 XUVs
C:0 T:14 1454 XUs and 1017 XVs -> 59 XUVs
C:0 T:15 1980 XUs and 2995 XVs -> 385 XUVs
7940 XUVs total
1641 collection wire objects
7940 potential space points
Neighbour search...
871518 tests to find 389716 neighbours
Iterating with no regularization...
Begin: 4.00112e+08
0 3.51029e+08
1 3.47261e+08
2 3.46683e+08
3 3.46502e+08
Now with regularization...
Begin: 3.37895e+08
0 3.37701e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
03-Sep-2025 00:40:56 BST  Closed output file "np02vd_raw_run039275_0447_df-s02-d1_dw_0_20250902T052648_reco_stage1_20250902T234056_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        192.218       364.875       700.499       291.92        162.373        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      9.3625e-05    0.000417583    0.0052255    0.000140167   0.00116661       18     
produce:tpcrawdecoder:PDVDTPCReader               15.2518       16.5745       17.8996       16.6337      0.691499        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0355596     0.0371411     0.0447102     0.0361512    0.00240243       18     
produce:pdvddaphne:DAPHNEReaderPDVD               4.91367       5.71875       6.69176       5.75317      0.475757        18     
produce:ophit:OpHitFinder                        0.0643041     0.0761999     0.0907695      0.07636     0.00579206       18     
produce:opflash:OpFlashFinderVerticalDrift        0.0104       0.0190324     0.0293178     0.0181086    0.00478937       18     
produce:wclsdatavd:WireCellToolkit                89.0676       101.749       126.227       99.5136       9.90211        18     
produce:gaushit:GausHitFinder                     1.74994       2.44929       3.90514       2.31651      0.586434        18     
produce:nhitsfilter:NumberOfHitsFilter          0.00037409    0.000600708   0.00119116    0.000555059   0.000196864      18     
produce:reco3d:SpacePointSolver                   14.9503       29.1269       54.3377       28.0286       10.5349        18     
produce:hitpdune:DisambigFromSpacePoints         0.254606      0.557352       1.05179      0.512753      0.246166        18     
produce:pandora:StandardPandora                   47.4154       198.19        482.18        125.847       145.451        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.844887       2.41429       4.73025       2.38841      0.936701        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0411681     0.0615395     0.0832117     0.0594087     0.0127917       18     
[art]:TriggerResults:TriggerResultInserter      2.8844e-05    3.89851e-05   7.8977e-05    3.6804e-05    1.04537e-05      18     
end_path:out1:RootOutput                         4.338e-06    1.12288e-05   2.8774e-05     1.052e-05    4.53895e-06      18     
end_path:out1:RootOutput(write)                   7.28597       7.82662       8.58527       7.70348       0.36128        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5468.57 MB
  Peak resident set size usage (VmHWM): 3515.67 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0447_df-s02-d1_dw_0_20250902T052648_reco_stage1_20250902T234056_keepup.root
\tHists: np02vd_raw_run039275_0447_df-s02-d1_dw_0_20250902T052648_reco_stage1_20250902T234057_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2025-11-05 14:43:54 UTC       justIN version: 01.05.01