justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 32473.10@dunegpschedd02.fnal.gov

Jobsub ID32473.10@dunegpschedd02.fnal.gov
Workflow ID2135
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-02 21:47:08
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-09-02 23:10:41
From worker nodeHostnameheplnc160.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-09-02 21:50:25
Input filesvd-protodune:np02vd_raw_run039275_0446_df-s03-d2_dw_0_20250902T052416.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-09-02 23:10:41
Saved logsjustin-logs:32473.10-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ples within desired window for channel 7423
wclsFrameSaver: no samples within desired window for channel 7424
wclsFrameSaver: no samples within desired window for channel 7443
wclsFrameSaver: no samples within desired window for channel 7446
wclsFrameSaver: no samples within desired window for channel 7609
wclsFrameSaver: no samples within desired window for channel 7761
wclsFrameSaver: no samples within desired window for channel 7762
wclsFrameSaver: no samples within desired window for channel 7763
wclsFrameSaver: no samples within desired window for channel 7965
wclsFrameSaver: no samples within desired window for channel 7966
wclsFrameSaver: no samples within desired window for channel 7967
wclsFrameSaver: no samples within desired window for channel 7969
wclsFrameSaver: no samples within desired window for channel 8525
wclsFrameSaver: no samples within desired window for channel 8525
wclsFrameSaver: no samples within desired window for channel 8526
wclsFrameSaver: no samples within desired window for channel 8526
wclsFrameSaver: no samples within desired window for channel 8527
wclsFrameSaver: no samples within desired window for channel 8528
wclsFrameSaver: no samples within desired window for channel 8528
wclsFrameSaver: no samples within desired window for channel 8530
wclsFrameSaver: no samples within desired window for channel 8564
wclsFrameSaver: no samples within desired window for channel 8565
wclsFrameSaver: no samples within desired window for channel 8567
wclsFrameSaver: no samples within desired window for channel 8567
wclsFrameSaver: no samples within desired window for channel 8568
wclsFrameSaver: no samples within desired window for channel 8568
wclsFrameSaver: no samples within desired window for channel 8568
wclsFrameSaver: no samples within desired window for channel 8569
wclsFrameSaver: no samples within desired window for channel 8569
wclsFrameSaver: no samples within desired window for channel 8875
wclsFrameSaver: no samples within desired window for channel 8876
wclsFrameSaver: no samples within desired window for channel 8876
wclsFrameSaver: no samples within desired window for channel 8877
wclsFrameSaver: no samples within desired window for channel 8877
wclsFrameSaver: no samples within desired window for channel 8878
wclsFrameSaver: no samples within desired window for channel 8878
wclsFrameSaver: no samples within desired window for channel 9160
wclsFrameSaver: no samples within desired window for channel 9359
wclsFrameSaver: no samples within desired window for channel 9361
wclsFrameSaver: no samples within desired window for channel 9769
wclsFrameSaver: no samples within desired window for channel 9769
wclsFrameSaver: no samples within desired window for channel 9770
wclsFrameSaver: no samples within desired window for channel 9771
wclsFrameSaver: no samples within desired window for channel 9771
wclsFrameSaver: no samples within desired window for channel 9797
wclsFrameSaver: no samples within desired window for channel 9798
wclsFrameSaver: no samples within desired window for channel 10268
wclsFrameSaver: no samples within desired window for channel 10268
wclsFrameSaver: no samples within desired window for channel 10663
wclsFrameSaver: no samples within desired window for channel 10664
wclsFrameSaver: no samples within desired window for channel 10665
wclsFrameSaver: no samples within desired window for channel 10666
wclsFrameSaver: no samples within desired window for channel 10666
wclsFrameSaver: no samples within desired window for channel 10666
wclsFrameSaver: no samples within desired window for channel 10743
wclsFrameSaver: no samples within desired window for channel 10743
wclsFrameSaver: no samples within desired window for channel 10818
wclsFrameSaver: no samples within desired window for channel 11089
wclsFrameSaver: no samples within desired window for channel 11334
wclsFrameSaver: no samples within desired window for channel 11335
wclsFrameSaver: no samples within desired window for channel 11335
wclsFrameSaver: no samples within desired window for channel 11858
wclsFrameSaver: no samples within desired window for channel 11933
wclsFrameSaver: no samples within desired window for channel 11933
wclsFrameSaver: no samples within desired window for channel 11934
wclsFrameSaver: no samples within desired window for channel 11935
wclsFrameSaver: no samples within desired window for channel 11935
wclsFrameSaver: no samples within desired window for channel 11936
wclsFrameSaver: no samples within desired window for channel 11937
wclsFrameSaver: no samples within desired window for channel 11937
wclsFrameSaver: no samples within desired window for channel 11938
wclsFrameSaver: no samples within desired window for channel 11939
wclsFrameSaver: no samples within desired window for channel 11939
wclsFrameSaver: no samples within desired window for channel 11940
FrameSaver: q=1.37171e+07 n=1419796 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 962 XUs and 1056 XVs -> 69 XUVs
C:0 T:1 564 XUs and 390 XVs -> 25 XUVs
C:0 T:2 1017 XUs and 1191 XVs -> 43 XUVs
C:0 T:3 2591 XUs and 3458 XVs -> 126 XUVs
C:0 T:4 671 XUs and 522 XVs -> 21 XUVs
C:0 T:5 2277 XUs and 2698 XVs -> 126 XUVs
C:0 T:6 1217 XUs and 1589 XVs -> 72 XUVs
C:0 T:7 1911 XUs and 2388 XVs -> 112 XUVs
C:0 T:8 13309 XUs and 21409 XVs -> 2546 XUVs
C:0 T:9 17809 XUs and 43256 XVs -> 3969 XUVs
C:0 T:10 15725 XUs and 42841 XVs -> 7017 XUVs
C:0 T:11 4604 XUs and 3443 XVs -> 182 XUVs
C:0 T:12 6644 XUs and 7818 XVs -> 1584 XUVs
C:0 T:13 2849 XUs and 2724 XVs -> 157 XUVs
C:0 T:14 5064 XUs and 6572 XVs -> 914 XUVs
C:0 T:15 8480 XUs and 11245 XVs -> 1029 XUVs
17992 XUVs total
2819 collection wire objects
17992 potential space points
Neighbour search...
2560678 tests to find 1044508 neighbours
Iterating with no regularization...
Begin: 9.86841e+08
0 8.65474e+08
1 8.54675e+08
2 8.53416e+08
3 8.52986e+08
Now with regularization...
Begin: 8.37919e+08
0 8.37601e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
03-Sep-2025 00:10:14 BST  Closed output file "np02vd_raw_run039275_0446_df-s03-d2_dw_0_20250902T052416_reco_stage1_20250902T231014_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        174.758       262.686       460.57        246.225       77.7611        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      8.9898e-05    0.00013997    0.000375723   0.000115832   6.58748e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               15.4245       16.2404       17.1174       16.2148      0.475126        18     
produce:triggerrawdecoder:PDVDTriggerReader4     0.0355581     0.0360248     0.0377874     0.035889     0.00054157       18     
produce:pdvddaphne:DAPHNEReaderPDVD               4.82506       5.34736       5.81139       5.40532      0.296861        18     
produce:ophit:OpHitFinder                        0.0682089     0.0789922     0.0880211     0.0781118    0.00555208       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00810191     0.0204229     0.0331998     0.0198215    0.00600415       18     
produce:wclsdatavd:WireCellToolkit                82.5713       93.4812       107.675       93.1416       6.33401        18     
produce:gaushit:GausHitFinder                     1.35617       2.13808       2.95973       2.13202      0.461428        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000306223   0.000514279   0.000845261   0.00049452    0.000144734      18     
produce:reco3d:SpacePointSolver                   12.3097       23.4337       39.5457       22.7682       7.87926        18     
produce:hitpdune:DisambigFromSpacePoints          0.18539      0.416034      0.771506      0.378328      0.167543        18     
produce:pandora:StandardPandora                   39.4209       111.748       280.183       94.3789       66.3586        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.646137       1.81714       3.28138       1.50355      0.791429        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0376022     0.0569794     0.0832505     0.0571303     0.0155091       18     
[art]:TriggerResults:TriggerResultInserter      2.9095e-05    4.58265e-05   0.000136406   3.6353e-05    2.60567e-05      18     
end_path:out1:RootOutput                         9.979e-06    1.32761e-05   4.7129e-05    1.1206e-05    8.26301e-06      18     
end_path:out1:RootOutput(write)                   7.22445       7.81445       8.46058       7.72787      0.355955        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5108.1 MB
  Peak resident set size usage (VmHWM): 3197.36 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0446_df-s03-d2_dw_0_20250902T052416_reco_stage1_20250902T231014_keepup.root
\tHists: np02vd_raw_run039275_0446_df-s03-d2_dw_0_20250902T052416_reco_stage1_20250902T231015_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2025-11-05 14:41:28 UTC       justIN version: 01.05.01