justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303843.3@dunegpschedd01.fnal.gov

Jobsub ID303843.3@dunegpschedd01.fnal.gov
Workflow ID12725
Stage ID1
User namecalcuttj@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-02-04 03:46:37
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2026-02-04 04:15:37
From worker nodeHostnameb9p17p7474.cern.ch
cpuinfoIntel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 03:47:16
Input filesvd-protodune:np02vd_raw_run040380_0061_df-s04-d2_dw_0_20251105T182417.hdf5
JobscriptExit code0
Real time28m (1680s)
CPU time25m (1551s = 92%)
Max RSS bytes1863786496 (1777 MiB)
Outputting started2026-02-04 04:15:17
Output files
Finished2026-02-04 04:15:37
Saved logsjustin-logs:303843.3-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tamp  1762367046.7968116

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367046.795559
number of samples =  7872
----- Number Of Hits found :  [3875 3202 3066]
---- Number Of 2D tracks found :  [40 32 27]
--- Number of 3D tracks found :  8
-- Found  64  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367046.795559
number of samples =  7872
----- Number Of Hits found :  [3333 2881 3389]
---- Number Of 2D tracks found :  [62 53 49]
--- Number of 3D tracks found :  17
-- Found  144  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367046.7955518
number of samples =  8064
----- Number Of Hits found :  [1780 1836 2523]
---- Number Of 2D tracks found :  [80 70 76]
--- Number of 3D tracks found :  25
-- Found  237  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367046.7955518
number of samples =  8064
----- Number Of Hits found :  [2169 2489 3762]
---- Number Of 2D tracks found :  [102  86 110]
--- Number of 3D tracks found :  32
-- Found  338  Single Hits!
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  5  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  18
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  18  TRIGGER  5621 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:07 2025  +  199299072.0  ns 
Trigger timestamp  1762367047.199299

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367047.1980484
number of samples =  7872
----- Number Of Hits found :  [2904 2595 2146]
---- Number Of 2D tracks found :  [19 24 21]
--- Number of 3D tracks found :  3
-- Found  70  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367047.1980484
number of samples =  7872
----- Number Of Hits found :  [2424 2645 1980]
---- Number Of 2D tracks found :  [32 37 36]
--- Number of 3D tracks found :  14
-- Found  138  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367047.1980479
number of samples =  8064
----- Number Of Hits found :  [2076 2364 2608]
---- Number Of 2D tracks found :  [49 59 67]
--- Number of 3D tracks found :  20
-- Found  238  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367047.1980479
number of samples =  8064
----- Number Of Hits found :  [2900 2874 3253]
---- Number Of 2D tracks found :  [ 76  90 116]
--- Number of 3D tracks found :  27
-- Found  314  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Found  2  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  19
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  19  TRIGGER  5624 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:07 2025  +  580471808.0  ns 
Trigger timestamp  1762367047.580472

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367047.5792058
number of samples =  7872
----- Number Of Hits found :  [2937 2808 2534]
---- Number Of 2D tracks found :  [17 21 23]
--- Number of 3D tracks found :  8
-- Found  64  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367047.5792058
number of samples =  7872
----- Number Of Hits found :  [2127 2032 1320]
---- Number Of 2D tracks found :  [27 32 41]
--- Number of 3D tracks found :  16
-- Found  130  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367047.5792
number of samples =  8064
----- Number Of Hits found :  [2649 2506 3765]
---- Number Of 2D tracks found :  [62 65 96]
--- Number of 3D tracks found :  27
-- Found  210  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367047.5792
number of samples =  8064
----- Number Of Hits found :  [1430 1457 1926]
---- Number Of 2D tracks found :  [ 76  76 111]
--- Number of 3D tracks found :  35
-- Found  294  Single Hits!
Across modules  [0, 1] merged  2  3D tracks together across module !!! 
Found  5  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  20
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  20  TRIGGER  5627 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:07 2025  +  814136064.0  ns 
Trigger timestamp  1762367047.8141363

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367047.8128743
number of samples =  8448
----- Number Of Hits found :  [2545 2183 1760]
---- Number Of 2D tracks found :  [13 12 13]
--- Number of 3D tracks found :  8
-- Found  97  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367047.8128743
number of samples =  8448
----- Number Of Hits found :  [2384 2455 2294]
---- Number Of 2D tracks found :  [29 27 29]
--- Number of 3D tracks found :  18
-- Found  157  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367047.812864
number of samples =  8704
----- Number Of Hits found :  [2147 3296 3585]
---- Number Of 2D tracks found :  [51 54 67]
--- Number of 3D tracks found :  25
-- Found  233  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367047.812864
number of samples =  8704
----- Number Of Hits found :  [3458 4290 6196]
---- Number Of 2D tracks found :  [ 99 104 131]
--- Number of 3D tracks found :  37
-- Found  324  Single Hits!
Found  8  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  21
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  21  TRIGGER  5631 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:08 2025  +  154236416.0  ns 
Trigger timestamp  1762367048.1542366

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367048.1529734
number of samples =  7872
----- Number Of Hits found :  [2183 1916 1239]
---- Number Of 2D tracks found :  [10 11 12]
--- Number of 3D tracks found :  6
-- Found  71  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367048.1529734
number of samples =  7872
----- Number Of Hits found :  [1663 1628  775]
---- Number Of 2D tracks found :  [18 20 20]
--- Number of 3D tracks found :  11
-- Found  130  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367048.15296
number of samples =  8064
----- Number Of Hits found :  [2452 3269 4231]
---- Number Of 2D tracks found :  [45 59 69]
--- Number of 3D tracks found :  13
-- Found  211  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367048.15296
number of samples =  8064
----- Number Of Hits found :  [1953 2900 3669]
---- Number Of 2D tracks found :  [ 68  81 106]
--- Number of 3D tracks found :  17
-- Found  294  Single Hits!
Found  1  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  22
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  22  TRIGGER  5635 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:08 2025  +  666786048.0  ns 
Trigger timestamp  1762367048.6667862

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367048.6655304
number of samples =  7872
----- Number Of Hits found :  [4045 3903 3336]
---- Number Of 2D tracks found :  [41 40 51]
--- Number of 3D tracks found :  13
-- Found  70  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367048.6655304
number of samples =  7872
----- Number Of Hits found :  [2819 2322 2356]
---- Number Of 2D tracks found :  [54 52 67]
--- Number of 3D tracks found :  17
-- Found  140  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367048.6655037
number of samples =  8064
----- Number Of Hits found :  [1367 1373 1923]
---- Number Of 2D tracks found :  [66 64 85]
--- Number of 3D tracks found :  22
-- Found  222  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367048.6655037
number of samples =  8064
----- Number Of Hits found :  [1441 1511 2586]
---- Number Of 2D tracks found :  [ 76  79 101]
--- Number of 3D tracks found :  29
-- Found  303  Single Hits!
Found  1  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  23
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  23  TRIGGER  5638 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:24:09 2025  +  227809024.0  ns 
Trigger timestamp  1762367049.2278092

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367049.2265513
number of samples =  8512
----- Number Of Hits found :  [2838 2658 1739]
---- Number Of 2D tracks found :  [11 10 21]
--- Number of 3D tracks found :  6
-- Found  66  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367049.2265513
number of samples =  8512
----- Number Of Hits found :  [2819 2866 2441]
---- Number Of 2D tracks found :  [36 32 39]
--- Number of 3D tracks found :  11
-- Found  129  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367049.226528
number of samples =  8704
----- Number Of Hits found :  [3157 3498 4062]
---- Number Of 2D tracks found :  [75 64 89]
--- Number of 3D tracks found :  21
-- Found  240  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367049.226528
number of samples =  8704
----- Number Of Hits found :  [2552 3137 4314]
---- Number Of 2D tracks found :  [104  94 120]
--- Number of 3D tracks found :  29
-- Found  326  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  3  cathode crossing tracks 
file closed!
**************
* Reco total *
**************
* Nb of Hits 778223
* Nb of 2D Tracks 7136
* Nb of 3D Tracks 1059
* Nb of Ghosts 0
* Nb of Single Hits 7314
* Nb of PDS Peaks 0
* Nb of PDS Clusters 0
**************
it took 1570.99 s to run 24 events (average of 65.46 per event)
Select jobs to execute...
Select jobs to execute...
Complete log: .snakemake/log/2026-02-04T044819.786082.snakemake.log
Done running snakemake
justIN time: 2026-02-04 05:56:25 UTC       justIN version: 01.06.00