justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 303843.5@dunegpschedd01.fnal.gov

Jobsub ID303843.5@dunegpschedd01.fnal.gov
Workflow ID12725
Stage ID1
User namecalcuttj@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-02-04 03:46:37
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce513
Last heartbeat2026-02-04 04:01:21
From worker nodeHostnameb9p35p0059.cern.ch
cpuinfoAMD EPYC 9655P 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-04 03:47:14
Input filesvd-protodune:np02vd_raw_run040380_0069_df-s04-d3_dw_0_20251105T182923.hdf5
JobscriptExit code0
Real time13m (832s)
CPU time11m (670s = 80%)
Max RSS bytes1749905408 (1668 MiB)
Outputting started2026-02-04 04:01:07
Output files
Finished2026-02-04 04:01:21
Saved logsjustin-logs:303843.5-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

3 63 65]
--- Number of 3D tracks found :  23
-- Found  297  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367266.7514882
number of samples =  9280
----- Number Of Hits found :  [4022 3415 6709]
---- Number Of 2D tracks found :  [124 104 124]
--- Number of 3D tracks found :  40
-- Found  404  Single Hits!
WHAAAAT ? too many possibility for cathode stitcher, do not do anything 
Found  2  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  17
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  17  TRIGGER  6404 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:47 2025  +  563036160.0  ns 
Trigger timestamp  1762367267.5630362

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367267.561767
number of samples =  8512
----- Number Of Hits found :  [2732 2461 1574]
---- Number Of 2D tracks found :  [13 13 20]
--- Number of 3D tracks found :  6
-- Found  48  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367267.561767
number of samples =  8512
----- Number Of Hits found :  [2483 2225 1621]
---- Number Of 2D tracks found :  [29 28 37]
--- Number of 3D tracks found :  10
-- Found  128  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367267.56176
number of samples =  8768
----- Number Of Hits found :  [3002 3311 5089]
---- Number Of 2D tracks found :  [ 88  67 116]
--- Number of 3D tracks found :  27
-- Found  208  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367267.56176
number of samples =  8768
----- Number Of Hits found :  [1421 1438 2483]
---- Number Of 2D tracks found :  [102  78 134]
--- Number of 3D tracks found :  34
-- Found  302  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Found  4  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  18
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  18  TRIGGER  6408 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:48 2025  +  55163136.0  ns 
Trigger timestamp  1762367268.0551631

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367268.0539095
number of samples =  7872
----- Number Of Hits found :  [2764 2533 1676]
---- Number Of 2D tracks found :  [12 11 21]
--- Number of 3D tracks found :  6
-- Found  72  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367268.0539095
number of samples =  7872
----- Number Of Hits found :  [4088 3956 2668]
---- Number Of 2D tracks found :  [55 63 63]
--- Number of 3D tracks found :  32
-- Found  141  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367268.053888
number of samples =  8064
----- Number Of Hits found :  [627 677 918]
---- Number Of 2D tracks found :  [62 70 75]
--- Number of 3D tracks found :  36
-- Found  191  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367268.053888
number of samples =  8064
----- Number Of Hits found :  [2141 2595 3191]
---- Number Of 2D tracks found :  [ 84  83 103]
--- Number of 3D tracks found :  47
-- Found  286  Single Hits!
Across modules  [0, 1] merged  2  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  1  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  19
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  19  TRIGGER  6412 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:49 2025  +  188950016.0  ns 
Trigger timestamp  1762367269.18895

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367269.1876824
number of samples =  9920
----- Number Of Hits found :  [8397 8078 9487]
---- Number Of 2D tracks found :  [ 87 114 119]
--- Number of 3D tracks found :  183
-- Found  21  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367269.1876824
number of samples =  9920
----- Number Of Hits found :  [3411 3262 2674]
---- Number Of 2D tracks found :  [103 139 142]
--- Number of 3D tracks found :  192
-- Found  116  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367269.1876798
number of samples =  10176
----- Number Of Hits found :  [ 9138  8856 12290]
---- Number Of 2D tracks found :  [229 273 305]
3D finalizing track: problem with interpolation
--- Number of 3D tracks found :  543
-- Found  224  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367269.1876798
number of samples =  10176
----- Number Of Hits found :  [1471 1574 2179]
---- Number Of 2D tracks found :  [245 285 321]
--- Number of 3D tracks found :  546
-- Found  388  Single Hits!
WHAAAAT ? too many possibility for cathode stitcher, do not do anything 
WHAAAAT ? too many possibility for cathode stitcher, do not do anything 
WHAAAAT ? too many possibility for cathode stitcher, do not do anything 
WHAAAAT ? too many possibility for cathode stitcher, do not do anything 
Found  4  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  20
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  20  TRIGGER  6419 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:51 2025  +  184883968.0  ns 
Trigger timestamp  1762367271.184884

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367271.183614
number of samples =  8512
----- Number Of Hits found :  [2749 2630 1918]
---- Number Of 2D tracks found :  [15 18 24]
--- Number of 3D tracks found :  7
-- Found  82  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367271.183614
number of samples =  8512
----- Number Of Hits found :  [3481 3614 3581]
---- Number Of 2D tracks found :  [35 44 53]
--- Number of 3D tracks found :  20
-- Found  163  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367271.1836162
number of samples =  8704
----- Number Of Hits found :  [2053 1852 2564]
---- Number Of 2D tracks found :  [55 60 75]
--- Number of 3D tracks found :  33
-- Found  264  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367271.1836162
number of samples =  8704
----- Number Of Hits found :  [4019 3991 6004]
---- Number Of 2D tracks found :  [122 123 153]
3D finalizing track: problem with interpolation
--- Number of 3D tracks found :  170
-- Found  358  Single Hits!
Across modules  [0, 1] merged  2  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  3  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  21
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  21  TRIGGER  6440 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:54 2025  +  132068608.0  ns 
Trigger timestamp  1762367274.1320686

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367274.1308007
number of samples =  7872
----- Number Of Hits found :  [2107 2055 1300]
---- Number Of 2D tracks found :  [17 14 16]
--- Number of 3D tracks found :  6
-- Found  51  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367274.1308007
number of samples =  7872
----- Number Of Hits found :  [1883 2031 1109]
---- Number Of 2D tracks found :  [27 23 28]
--- Number of 3D tracks found :  12
-- Found  106  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367274.1308162
number of samples =  8064
----- Number Of Hits found :  [2746 2802 4107]
---- Number Of 2D tracks found :  [54 46 65]
--- Number of 3D tracks found :  16
-- Found  208  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367274.1308162
number of samples =  8064
----- Number Of Hits found :  [1652 1924 2181]
---- Number Of 2D tracks found :  [66 59 87]
--- Number of 3D tracks found :  24
-- Found  284  Single Hits!
Across modules  [0, 1] merged  2  3D tracks together across module !!! 
Across modules  [2, 3] merged  1  3D tracks together across module !!! 
Found  4  cathode crossing tracks 
-*-*-*-*-*-*-*-*-*-*-
 READING EVENT  22
-*-*-*-*-*-*-*-*-*-*-
RUN  40380  of  bot  EVENT  22  TRIGGER  6458 : kADCSimpleWindow ( 6 )
Taken at  Wed Nov  5 19:27:56 2025  +  864510208.0  ns 
Trigger timestamp  1762367276.8645103

## Reading TPC ##

MODULE  0
 Removing  12  broken channels
1st sample timestamp:  1762367276.8632588
number of samples =  7872
----- Number Of Hits found :  [2944 2476 2354]
---- Number Of 2D tracks found :  [20 14 26]
--- Number of 3D tracks found :  4
-- Found  60  Single Hits!

MODULE  1
 Removing  12  broken channels
1st sample timestamp:  1762367276.8632588
number of samples =  7872
----- Number Of Hits found :  [3357 3238 3185]
---- Number Of 2D tracks found :  [52 48 66]
--- Number of 3D tracks found :  9
-- Found  126  Single Hits!

MODULE  2
 Removing  12  broken channels
1st sample timestamp:  1762367276.8632321
number of samples =  8064
----- Number Of Hits found :  [1476 1483 2441]
---- Number Of 2D tracks found :  [62 61 90]
--- Number of 3D tracks found :  17
-- Found  209  Single Hits!

MODULE  3
 Removing  12  broken channels
1st sample timestamp:  1762367276.8632321
number of samples =  8064
----- Number Of Hits found :  [2072 2751 3882]
---- Number Of 2D tracks found :  [ 84  80 123]
--- Number of 3D tracks found :  27
-- Found  279  Single Hits!
Across modules  [0, 1] merged  1  3D tracks together across module !!! 
Across modules  [2, 3] merged  2  3D tracks together across module !!! 
Found  3  cathode crossing tracks 
file closed!
**************
* Reco total *
**************
* Nb of Hits 768980
* Nb of 2D Tracks 7372
* Nb of 3D Tracks 1423
* Nb of Ghosts 0
* Nb of Single Hits 7299
* Nb of PDS Peaks 0
* Nb of PDS Clusters 0
**************
it took 748.93 s to run 23 events (average of 32.56 per event)
Select jobs to execute...
Select jobs to execute...
Complete log: .snakemake/log/2026-02-04T044752.587390.snakemake.log
Done running snakemake
justIN time: 2026-02-04 06:00:19 UTC       justIN version: 01.06.00