justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 39108.34@dunegpschedd01.fnal.gov

Jobsub ID39108.34@dunegpschedd01.fnal.gov
Workflow ID2178
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2025-09-02 17:21:03
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2025-09-02 18:14:31
From worker nodeHostnameccwcondor0051
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statejobscript_error
Started2025-09-02 17:21:41
Input filesvd-protodune:np02vd_raw_run039275_0243_df-s01-d2_dw_0_20250901T225517.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Max RSS bytes0 (0 MiB)
Outputting started 
Output files
Finished2025-09-02 18:14:31
Saved logsjustin-logs:39108.34-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

amples within desired window for channel 9137
wclsFrameSaver: no samples within desired window for channel 9137
wclsFrameSaver: no samples within desired window for channel 9138
wclsFrameSaver: no samples within desired window for channel 9138
wclsFrameSaver: no samples within desired window for channel 9138
wclsFrameSaver: no samples within desired window for channel 9139
wclsFrameSaver: no samples within desired window for channel 9139
wclsFrameSaver: no samples within desired window for channel 9139
wclsFrameSaver: no samples within desired window for channel 9140
wclsFrameSaver: no samples within desired window for channel 9140
wclsFrameSaver: no samples within desired window for channel 9141
wclsFrameSaver: no samples within desired window for channel 9142
wclsFrameSaver: no samples within desired window for channel 9144
wclsFrameSaver: no samples within desired window for channel 9146
wclsFrameSaver: no samples within desired window for channel 9149
wclsFrameSaver: no samples within desired window for channel 9332
wclsFrameSaver: no samples within desired window for channel 9473
wclsFrameSaver: no samples within desired window for channel 9474
wclsFrameSaver: no samples within desired window for channel 9475
wclsFrameSaver: no samples within desired window for channel 9475
wclsFrameSaver: no samples within desired window for channel 9476
wclsFrameSaver: no samples within desired window for channel 9476
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9495
wclsFrameSaver: no samples within desired window for channel 9583
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9631
wclsFrameSaver: no samples within desired window for channel 9631
wclsFrameSaver: no samples within desired window for channel 9632
wclsFrameSaver: no samples within desired window for channel 9633
wclsFrameSaver: no samples within desired window for channel 9634
wclsFrameSaver: no samples within desired window for channel 10205
wclsFrameSaver: no samples within desired window for channel 10274
wclsFrameSaver: no samples within desired window for channel 10333
wclsFrameSaver: no samples within desired window for channel 10444
wclsFrameSaver: no samples within desired window for channel 10490
wclsFrameSaver: no samples within desired window for channel 10491
wclsFrameSaver: no samples within desired window for channel 10492
wclsFrameSaver: no samples within desired window for channel 10503
wclsFrameSaver: no samples within desired window for channel 10503
wclsFrameSaver: no samples within desired window for channel 10505
wclsFrameSaver: no samples within desired window for channel 10505
wclsFrameSaver: no samples within desired window for channel 10506
wclsFrameSaver: no samples within desired window for channel 10507
wclsFrameSaver: no samples within desired window for channel 10507
wclsFrameSaver: no samples within desired window for channel 10508
wclsFrameSaver: no samples within desired window for channel 10509
wclsFrameSaver: no samples within desired window for channel 10509
wclsFrameSaver: no samples within desired window for channel 10510
wclsFrameSaver: no samples within desired window for channel 10511
wclsFrameSaver: no samples within desired window for channel 10708
wclsFrameSaver: no samples within desired window for channel 10830
wclsFrameSaver: no samples within desired window for channel 10996
wclsFrameSaver: no samples within desired window for channel 11088
wclsFrameSaver: no samples within desired window for channel 11089
wclsFrameSaver: no samples within desired window for channel 11353
wclsFrameSaver: no samples within desired window for channel 11353
wclsFrameSaver: no samples within desired window for channel 11353
wclsFrameSaver: no samples within desired window for channel 11436
wclsFrameSaver: no samples within desired window for channel 11436
wclsFrameSaver: no samples within desired window for channel 11437
wclsFrameSaver: no samples within desired window for channel 11465
wclsFrameSaver: no samples within desired window for channel 11465
wclsFrameSaver: no samples within desired window for channel 11466
wclsFrameSaver: no samples within desired window for channel 11469
wclsFrameSaver: no samples within desired window for channel 11469
wclsFrameSaver: no samples within desired window for channel 11678
wclsFrameSaver: no samples within desired window for channel 11678
wclsFrameSaver: no samples within desired window for channel 11679
wclsFrameSaver: no samples within desired window for channel 11679
wclsFrameSaver: no samples within desired window for channel 11680
wclsFrameSaver: no samples within desired window for channel 11681
wclsFrameSaver: no samples within desired window for channel 11682
FrameSaver: q=5.66353e+06 n=605479 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 629 XUs and 958 XVs -> 65 XUVs
C:0 T:1 718 XUs and 1139 XVs -> 52 XUVs
C:0 T:2 285 XUs and 469 XVs -> 20 XUVs
C:0 T:3 424 XUs and 425 XVs -> 27 XUVs
C:0 T:4 134 XUs and 117 XVs -> 3 XUVs
C:0 T:5 26 XUs and 22 XVs -> 0 XUVs
C:0 T:6 343 XUs and 616 XVs -> 26 XUVs
C:0 T:7 352 XUs and 482 XVs -> 17 XUVs
C:0 T:8 991 XUs and 761 XVs -> 61 XUVs
C:0 T:9 2290 XUs and 1387 XVs -> 129 XUVs
C:0 T:10 431 XUs and 491 XVs -> 26 XUVs
C:0 T:11 1577 XUs and 1560 XVs -> 132 XUVs
C:0 T:12 2780 XUs and 2081 XVs -> 153 XUVs
C:0 T:13 2397 XUs and 3005 XVs -> 130 XUVs
C:0 T:14 4976 XUs and 3905 XVs -> 327 XUVs
C:0 T:15 2291 XUs and 1851 XVs -> 107 XUVs
1275 XUVs total
1109 collection wire objects
1275 potential space points
Neighbour search...
10803 tests to find 5124 neighbours
Iterating with no regularization...
Begin: 4.81774e+08
0 4.70474e+08
1 4.70352e+08
Now with regularization...
Begin: 4.65523e+08
0 4.65514e+08
02-Sep-2025 20:13:58 CEST  Closed output file "np02vd_raw_run039275_0243_df-s01-d2_dw_0_20250901T225517_reco_stage1_20250902T181358_keepup.root"

==================================================================================================================================
TimeTracker printout (sec)                          Min           Avg           Max         Median          RMS         nEvts   
==================================================================================================================================
Full event                                        120.609       172.289       301.133       164.483       51.154         18     
----------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                      6.8895e-05    0.000104172   0.000267776   9.1328e-05    4.39838e-05      18     
produce:tpcrawdecoder:PDVDTPCReader               6.52929        7.474        8.4212        7.39697       0.63487        18     
produce:triggerrawdecoder:PDVDTriggerReader4    0.00592042    0.00722661     0.0159788     0.0064201    0.00245513       18     
produce:pdvddaphne:DAPHNEReaderPDVD               3.39525       4.02541        4.857        4.06662      0.447475        18     
produce:ophit:OpHitFinder                        0.0499409     0.0616364     0.0778961     0.0613789    0.00621178       18     
produce:opflash:OpFlashFinderVerticalDrift      0.00580525     0.0145504     0.0228165     0.014644     0.00404874       18     
produce:wclsdatavd:WireCellToolkit                58.3951       68.4685       80.3495       69.2743       5.41199        18     
produce:gaushit:GausHitFinder                    0.752822       1.41923       2.04936       1.42525      0.418638        18     
produce:nhitsfilter:NumberOfHitsFilter          0.000161395   0.000298805   0.000425225   0.000302328   7.91489e-05      18     
produce:reco3d:SpacePointSolver                   8.7762        16.0837       25.3399       15.5673       5.36925        18     
produce:hitpdune:DisambigFromSpacePoints         0.112577      0.275103      0.480887       0.27318      0.114235        18     
produce:pandora:StandardPandora                   19.8054       67.1785       181.493       60.3469       43.3629        18     
produce:pandoraTrack:LArPandoraTrackCreation     0.525135       1.18994       2.76833       1.08544      0.631075        18     
produce:pandoraGnocalo:GnocchiCalorimetry        0.0194039     0.0346609     0.0563457     0.0326882     0.0116192       18     
[art]:TriggerResults:TriggerResultInserter       1.984e-05    3.96557e-05   0.000143127   3.3265e-05    2.62662e-05      18     
end_path:out1:RootOutput                         4.427e-06    1.0216e-05    3.1608e-05    9.0785e-06    5.41408e-06      18     
end_path:out1:RootOutput(write)                   4.26513       6.00798       6.65236       6.11266      0.553355        18     
==================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5226.07 MB
  Peak resident set size usage (VmHWM): 3312.39 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np02vd_raw_run039275_0243_df-s01-d2_dw_0_20250901T225517_reco_stage1_20250902T181358_keepup.root
\tHists: np02vd_raw_run039275_0243_df-s01-d2_dw_0_20250901T225517_reco_stage1_20250902T181358_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2025-09-19 04:34:48 UTC       justIN version: 01.05.00