justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303843.8@dunegpschedd01.fnal.gov

Jobsub ID303843.8@dunegpschedd01.fnal.gov
Workflow ID12725
Stage ID1
User namecalcuttj@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-02-04 03:46:37
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2026-02-04 04:07:15
From worker nodeHostnameb9p35p0043.cern.ch
cpuinfoAMD EPYC 9655P 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 03:47:13
Input filesvd-protodune:np02vd_raw_run040380_0007_df-s04-d0_dw_0_20251105T175934.hdf5
JobscriptExit code0
Real time19m (1184s)
CPU time16m (1016s = 85%)
Max RSS bytes2217336832 (2114 MiB)
Outputting started2026-02-04 04:06:58
Output files
Finished2026-02-04 04:07:14
Saved logsjustin-logs:303843.8-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

16  TRIGGER  680 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 18:59:46 2025  +  656478464.0  ns 
Trigger timestamp  1762365586.6564786

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365586.6552157
number of samples =  7872
----- Number Of Hits found :  [3526 3386 3519]
---- Number Of 2D tracks found :  [22 16 25]
--- Number of 3D tracks found :  8
-- Found  52  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365586.6552157
number of samples =  7872
----- Number Of Hits found :  [4426 4699 4732]
---- Number Of 2D tracks found :  [76 63 91]
--- Number of 3D tracks found :  27
-- Found /local_software/lardon/src/lardon/track_2d.py:168: RuntimeWarning: divide by zero encountered in divide
  slope_start = 1. / slope_start
/local_software/lardon/src/lardon/track_2d.py:169: RuntimeWarning: divide by zero encountered in divide
  slope_end = 1. / slope_end
/local_software/lardon/src/lardon/track_2d.py:146: UserWarning: 
The maximal number of iterations maxit (set to 20 by the program)
allowed for finding a smoothing spline with fp=s has been reached: s
too small.
There is an approximation returned but the corresponding weighted sum
of squared residuals does not satisfy the condition abs(fp-s)/s < tol.
  spline_x = UnivariateSpline(x_r_unique, z_r_ux)
/local_software/lardon/src/lardon/track_2d.py:146: UserWarning: 
The maximal number of iterations maxit (set to 20 by the program)
allowed for finding a smoothing spline with fp=s has been reached: s
too small.
There is an approximation returned but the corresponding weighted sum
of squared residuals does not satisfy the condition abs(fp-s)/s < tol.
  spline_x = UnivariateSpline(x_r_unique, z_r_ux)
 105  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365586.6551998
number of samples =  8064
----- Number Of Hits found :  [1149 1447 2088]
---- Number Of 2D tracks found :  [ 88  76 107]
--- Number of 3D tracks found :  32
-- Found  184  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365586.6551998
number of samples =  8064
----- Number Of Hits found :  [1284 1266 2179]
---- Number Of 2D tracks found :  [103  90 128]
--- Number of 3D tracks found :  37
-- Found  259  Single Hits!
Found  0  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  17
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  17  TRIGGER  684 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 18:59:48 2025  +  268144128.0  ns 
Trigger timestamp  1762365588.2681441

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365588.266877
number of samples =  7872
----- Number Of Hits found :  [2975 2804 2474]
---- Number Of 2D tracks found :  [17 20 21]
--- Number of 3D tracks found :  8
-- Found  73  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365588.266877
number of samples =  7872
----- Number Of Hits found :  [2552 2437 2308]
---- Number Of 2D tracks found :  [29 33 32]
--- Number of 3D tracks found :  12
-- Found  146  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365588.26688
number of samples =  8064
----- Number Of Hits found :  [1048 1690  926]
---- Number Of 2D tracks found :  [42 61 44]
--- Number of 3D tracks found :  15
-- Found  212  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365588.26688
number of samples =  8064
----- Number Of Hits found :  [1910 1920 2436]
---- Number Of 2D tracks found :  [66 80 78]
--- Number of 3D tracks found :  21
-- Found  294  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Found  1  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  18
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  18  TRIGGER  688 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 18:59:49 2025  +  345022720.0  ns 
Trigger timestamp  1762365589.345023

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365589.3437645
number of samples =  8448
----- Number Of Hits found :  [2729 2741 2021]
---- Number Of 2D tracks found :  [15 16 16]
--- Number of 3D tracks found :  7
-- Found  62  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365589.3437645
number of samples =  8448
----- Number Of Hits found :  [2624 2323 2009]
---- Number Of 2D tracks found :  [24 26 27]
--- Number of 3D tracks found :  11
-- Found  125  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365589.343744
number of samples =  8704
----- Number Of Hits found :  [1457 1807 2605]
---- Number Of 2D tracks found :  [48 48 46]
--- Number of 3D tracks found :  22
-- Found  243  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365589.343744
number of samples =  8704
----- Number Of Hits found :  [3781 3901 5352]
---- Number Of 2D tracks found :  [108 116 124]
--- Number of 3D tracks found :  39
-- Found  339  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  2  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  19
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  19  TRIGGER  692 : kCTBBeamSpillStart ( 32 )
Taken at  Wed Nov  5 18:59:50 2025  +  205328896.0  ns 
Trigger timestamp  1762365590.205329

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365590.2028105
number of samples =  9856
----- Number Of Hits found :  [3713 4010 3715]
---- Number Of 2D tracks found :  [30 27 32]
--- Number of 3D tracks found :  10
-- Found  88  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365590.2028105
number of samples =  9856
----- Number Of Hits found :  [4144 3276 3290]
---- Number Of 2D tracks found :  [51 54 51]
--- Number of 3D tracks found :  17
-- Found  162  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365590.202816
number of samples =  10048
----- Number Of Hits found :  [1855 1976 2884]
---- Number Of 2D tracks found :  [67 71 81]
--- Number of 3D tracks found :  24
-- Found  280  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365590.202816
number of samples =  10048
----- Number Of Hits found :  [1999 1815 2777]
---- Number Of 2D tracks found :  [ 81  86 109]
--- Number of 3D tracks found :  31
-- Found  400  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Found  1  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  20
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  20  TRIGGER  696 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 18:59:50 2025  +  379705088.0  ns 
Trigger timestamp  1762365590.3797052

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365590.378447
number of samples =  7872
----- Number Of Hits found :  [3903 3828 3666]
---- Number Of 2D tracks found :  [25 25 28]
--- Number of 3D tracks found :  10
-- Found  88  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365590.378447
number of samples =  7872
----- Number Of Hits found :  [2669 2793 2158]
---- Number Of 2D tracks found :  [42 44 51]
--- Number of 3D tracks found :  19
-- Found  144  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365590.378432
number of samples =  8064
----- Number Of Hits found :  [5704 8956 8634]
---- Number Of 2D tracks found :  [107 114 145]
--- Number of 3D tracks found :  25
-- Found  250  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365590.378432
number of samples =  8064
----- Number Of Hits found :  [3879 6347 6550]
---- Number Of 2D tracks found :  [145 160 215]
--- Number of 3D tracks found :  32
-- Found  319  Single Hits!
Across modules  [0, 1] merged  2  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  2  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  21
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  21  TRIGGER  700 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 18:59:50 2025  +  429480960.0  ns 
Trigger timestamp  1762365590.429481

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762365590.4282215
number of samples =  16896
----- Number Of Hits found :  [5749 5892 5075]
---- Number Of 2D tracks found :  [35 40 39]
--- Number of 3D tracks found :  9
-- Found  27  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762365590.4282215
number of samples =  16896
----- Number Of Hits found :  [4937 4546 3703]
---- Number Of 2D tracks found :  [55 59 61]
--- Number of 3D tracks found :  14
-- Found  89  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762365590.428224
number of samples =  17280
----- Number Of Hits found :  [15012 23519 24621]
---- Number Of 2D tracks found :  [250 342 383]
--- Number of 3D tracks found :  93
-- Found  184  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762365590.428224
number of samples =  17280
----- Number Of Hits found :  [13535 21035 23482]
---- Number Of 2D tracks found :  [402 530 656]
--- Number of 3D tracks found :  119
-- Found  297  Single Hits!
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  2  cathode crossing tracks 
file closed!
**************
* Reco total *
**************
* Nb of Hits 1204428
* Nb of 2D Tracks 12287
* Nb of 3D Tracks 1282
* Nb of Ghosts 0
* Nb of Single Hits 6886
* Nb of PDS Peaks 0
* Nb of PDS Clusters 0
**************
it took 1089.75 s to run 22 events (average of 49.53 per event)
Select jobs to execute...
Select jobs to execute...
Complete log: .snakemake/log/2026-02-04T044805.472657.snakemake.log
Done running snakemake
justIN time: 2026-02-04 06:00:43 UTC       justIN version: 01.06.00