justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 39221.1@dunegpschedd01.fnal.gov

Jobsub ID39221.1@dunegpschedd01.fnal.gov
Workflow ID2135
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-02 21:01:08
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2025-09-02 21:53:27
From worker nodeHostnameccwcondor0051
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-09-02 21:02:35
Input filesvd-protodune:np02vd_raw_run039275_0435_df-s03-d3_dw_0_20250902T045730.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-09-02 21:53:27
Saved logsjustin-logs:39221.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

FrameSaver: saving 83403 traces tagged "wiener"
wclsFrameSaver: no samples within desired window for channel 6217
wclsFrameSaver: no samples within desired window for channel 6217
wclsFrameSaver: no samples within desired window for channel 6218
wclsFrameSaver: no samples within desired window for channel 6218
wclsFrameSaver: no samples within desired window for channel 6218
wclsFrameSaver: no samples within desired window for channel 6219
wclsFrameSaver: no samples within desired window for channel 6219
wclsFrameSaver: no samples within desired window for channel 6219
wclsFrameSaver: no samples within desired window for channel 6220
wclsFrameSaver: no samples within desired window for channel 6220
wclsFrameSaver: no samples within desired window for channel 6221
wclsFrameSaver: no samples within desired window for channel 6222
wclsFrameSaver: no samples within desired window for channel 6223
wclsFrameSaver: no samples within desired window for channel 6225
wclsFrameSaver: no samples within desired window for channel 6388
wclsFrameSaver: no samples within desired window for channel 6744
wclsFrameSaver: no samples within desired window for channel 6746
wclsFrameSaver: no samples within desired window for channel 6746
wclsFrameSaver: no samples within desired window for channel 7118
wclsFrameSaver: no samples within desired window for channel 7260
wclsFrameSaver: no samples within desired window for channel 7706
wclsFrameSaver: no samples within desired window for channel 7707
wclsFrameSaver: no samples within desired window for channel 7708
wclsFrameSaver: no samples within desired window for channel 7709
wclsFrameSaver: no samples within desired window for channel 7728
wclsFrameSaver: no samples within desired window for channel 7728
wclsFrameSaver: no samples within desired window for channel 7894
wclsFrameSaver: no samples within desired window for channel 8269
wclsFrameSaver: no samples within desired window for channel 8269
wclsFrameSaver: no samples within desired window for channel 8270
wclsFrameSaver: no samples within desired window for channel 8270
wclsFrameSaver: no samples within desired window for channel 8271
wclsFrameSaver: no samples within desired window for channel 8271
wclsFrameSaver: no samples within desired window for channel 8272
wclsFrameSaver: no samples within desired window for channel 8272
wclsFrameSaver: no samples within desired window for channel 8273
wclsFrameSaver: no samples within desired window for channel 8274
wclsFrameSaver: no samples within desired window for channel 8274
wclsFrameSaver: no samples within desired window for channel 8274
wclsFrameSaver: no samples within desired window for channel 8275
wclsFrameSaver: no samples within desired window for channel 8276
wclsFrameSaver: no samples within desired window for channel 8756
wclsFrameSaver: no samples within desired window for channel 8764
wclsFrameSaver: no samples within desired window for channel 8764
wclsFrameSaver: no samples within desired window for channel 9797
wclsFrameSaver: no samples within desired window for channel 9799
wclsFrameSaver: no samples within desired window for channel 9799
wclsFrameSaver: no samples within desired window for channel 9800
wclsFrameSaver: no samples within desired window for channel 9801
wclsFrameSaver: no samples within desired window for channel 9801
wclsFrameSaver: no samples within desired window for channel 9802
wclsFrameSaver: no samples within desired window for channel 9802
wclsFrameSaver: no samples within desired window for channel 9803
wclsFrameSaver: no samples within desired window for channel 9804
wclsFrameSaver: no samples within desired window for channel 9852
wclsFrameSaver: no samples within desired window for channel 10721
wclsFrameSaver: no samples within desired window for channel 10721
wclsFrameSaver: no samples within desired window for channel 10740
wclsFrameSaver: no samples within desired window for channel 10741
wclsFrameSaver: no samples within desired window for channel 10742
wclsFrameSaver: no samples within desired window for channel 10743
wclsFrameSaver: no samples within desired window for channel 10745
wclsFrameSaver: no samples within desired window for channel 10746
wclsFrameSaver: no samples within desired window for channel 11075
wclsFrameSaver: no samples within desired window for channel 11088
wclsFrameSaver: no samples within desired window for channel 11859
wclsFrameSaver: no samples within desired window for channel 11860
wclsFrameSaver: no samples within desired window for channel 11862
wclsFrameSaver: no samples within desired window for channel 11862
wclsFrameSaver: no samples within desired window for channel 11863
wclsFrameSaver: no samples within desired window for channel 11863
wclsFrameSaver: no samples within desired window for channel 11864
wclsFrameSaver: no samples within desired window for channel 11974
FrameSaver: q=1.27177e+07 n=1264260 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 199 XUs and 264 XVs -> 6 XUVs
C:0 T:1 209 XUs and 244 XVs -> 4 XUVs
C:0 T:2 1189 XUs and 2016 XVs -> 160 XUVs
C:0 T:3 508 XUs and 293 XVs -> 52 XUVs
C:0 T:4 1491 XUs and 1464 XVs -> 71 XUVs
C:0 T:5 3671 XUs and 4184 XVs -> 365 XUVs
C:0 T:6 1757 XUs and 1762 XVs -> 91 XUVs
C:0 T:7 823 XUs and 1427 XVs -> 85 XUVs
C:0 T:8 13602 XUs and 17722 XVs -> 2494 XUVs
C:0 T:9 11751 XUs and 14312 XVs -> 1968 XUVs
C:0 T:10 14794 XUs and 30846 XVs -> 2627 XUVs
C:0 T:11 17361 XUs and 34354 XVs -> 5511 XUVs
C:0 T:12 7480 XUs and 18919 XVs -> 2418 XUVs
C:0 T:13 12047 XUs and 21138 XVs -> 2898 XUVs
C:0 T:14 6584 XUs and 11158 XVs -> 1966 XUVs
C:0 T:15 14028 XUs and 26618 XVs -> 3696 XUVs
24412 XUVs total
4185 collection wire objects
24412 potential space points
Neighbour search...
2711566 tests to find 1185460 neighbours
Iterating with no regularization...
Begin: 1.00487e+09
0 7.88694e+08
1 7.73842e+08
2 7.72193e+08
3 7.71658e+08
Now with regularization...
Begin: 7.46202e+08
0 7.4578e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
02-Sep-2025 23:53:05 CEST  Closed output file "np02vd_raw_run039275_0435_df-s03-d3_dw_0_20250902T045730_reco_stage1_20250902T215305_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        115.552       166.263       330.136       145.554       51.9529        18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.4618e-05    9.79881e-05   0.000268077   7.6992e-05    4.63489e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               6.64095       7.17249       7.56733       7.15353      0.224704        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00581955    0.00626768    0.00723892    0.00616011    0.000411906      18     
produce:pdvddaphne:DAPHNEReaderPDVD               3.68943       4.01753       4.35264       4.02456      0.204623        18     
produce:ophit:OpHitFinder                        0.0341207     0.0598174     0.0683525     0.0618276     0.0083873       18     
produce:opflash:OpFlashFinderVerticalDrift       0.0047396     0.0143307     0.0209539     0.0137054    0.00479842       18     
produce:wclsdatavd:WireCellToolkit                58.9072       68.0548       81.9601       68.0313       6.22016        18     
produce:gaushit:GausHitFinder                    0.683865       1.2833        2.26546       1.2459       0.397905        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000180023   0.000289432   0.000443112   0.000287295   8.67522e-05      18     
produce:reco3d:SpacePointSolver                   8.52916       16.2036       28.7808       15.1103       6.20805        18     
produce:hitpdune:DisambigFromSpacePoints         0.114859      0.282127      0.489477       0.24555      0.124026        18     
produce:pandora:StandardPandora                   23.9707       62.1735       198.259       44.6373       42.4904        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.603792       1.27719       2.12008       1.17865      0.466653        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0212629     0.0367875     0.0523938     0.0357198    0.00948616       18     
[art]:TriggerResults:TriggerResultInserter      2.5018e-05    4.38393e-05   0.000141394   3.64455e-05   2.5073e-05       18     
end_path:out1:RootOutput                         5.649e-06    1.1102e-05    4.3116e-05    9.5345e-06    7.94861e-06      18     
end_path:out1:RootOutput(write)                   4.42699       5.6268        6.41705       5.68066      0.519499        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5239.09 MB
  Peak resident set size usage (VmHWM): 3308.69 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0435_df-s03-d3_dw_0_20250902T045730_reco_stage1_20250902T215305_keepup.root
\tHists: np02vd_raw_run039275_0435_df-s03-d3_dw_0_20250902T045730_reco_stage1_20250902T215305_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2025-09-19 02:43:58 UTC       justIN version: 01.05.00