justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293089.5@dunegpschedd02.fnal.gov

Jobsub ID293089.5@dunegpschedd02.fnal.gov
Workflow ID12206
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 22:58:18
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw7
Last heartbeat2026-01-22 02:03:02
From worker nodeHostnamemh-7763-4.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 22:59:05
Input fileshd-protodune:np04hd_raw_run027308_0018_dataflow0_datawriter_0_20240619T183931.hdf5
JobscriptExit code0
Real time3h (11018s)
CPU time1h (5994s = 54%)
Max RSS bytes2117627904 (2019 MiB)
Outputting started2026-01-22 02:02:44
Output files
Finished2026-01-22 02:03:02
Saved logsjustin-logs:293089.5-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

for channel 9117
wclsFrameSaver: no samples within desired window for channel 9124
wclsFrameSaver: no samples within desired window for channel 9135
wclsFrameSaver: no samples within desired window for channel 9139
wclsFrameSaver: no samples within desired window for channel 9159
wclsFrameSaver: no samples within desired window for channel 9159
wclsFrameSaver: no samples within desired window for channel 9160
wclsFrameSaver: no samples within desired window for channel 9160
wclsFrameSaver: no samples within desired window for channel 9160
wclsFrameSaver: no samples within desired window for channel 9178
wclsFrameSaver: no samples within desired window for channel 9205
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9209
wclsFrameSaver: no samples within desired window for channel 9209
wclsFrameSaver: no samples within desired window for channel 9209
wclsFrameSaver: no samples within desired window for channel 9209
wclsFrameSaver: no samples within desired window for channel 9225
wclsFrameSaver: no samples within desired window for channel 9228
wclsFrameSaver: no samples within desired window for channel 9251
wclsFrameSaver: no samples within desired window for channel 9261
wclsFrameSaver: no samples within desired window for channel 9442
wclsFrameSaver: no samples within desired window for channel 9443
wclsFrameSaver: no samples within desired window for channel 9443
wclsFrameSaver: no samples within desired window for channel 9444
wclsFrameSaver: no samples within desired window for channel 9444
wclsFrameSaver: no samples within desired window for channel 9445
wclsFrameSaver: no samples within desired window for channel 9445
wclsFrameSaver: no samples within desired window for channel 9446
wclsFrameSaver: no samples within desired window for channel 9447
wclsFrameSaver: no samples within desired window for channel 9448
wclsFrameSaver: no samples within desired window for channel 9449
wclsFrameSaver: no samples within desired window for channel 9450
wclsFrameSaver: no samples within desired window for channel 9451
wclsFrameSaver: no samples within desired window for channel 9452
wclsFrameSaver: no samples within desired window for channel 9452
wclsFrameSaver: no samples within desired window for channel 9453
wclsFrameSaver: no samples within desired window for channel 9454
wclsFrameSaver: no samples within desired window for channel 9454
wclsFrameSaver: no samples within desired window for channel 9455
wclsFrameSaver: no samples within desired window for channel 9456
wclsFrameSaver: no samples within desired window for channel 9457
wclsFrameSaver: no samples within desired window for channel 9457
wclsFrameSaver: no samples within desired window for channel 9458
wclsFrameSaver: no samples within desired window for channel 9459
wclsFrameSaver: no samples within desired window for channel 9460
wclsFrameSaver: no samples within desired window for channel 9460
wclsFrameSaver: no samples within desired window for channel 9461
wclsFrameSaver: no samples within desired window for channel 9461
wclsFrameSaver: no samples within desired window for channel 9462
wclsFrameSaver: no samples within desired window for channel 9462
wclsFrameSaver: no samples within desired window for channel 9463
wclsFrameSaver: no samples within desired window for channel 9464
wclsFrameSaver: no samples within desired window for channel 9465
wclsFrameSaver: no samples within desired window for channel 9465
wclsFrameSaver: no samples within desired window for channel 9466
wclsFrameSaver: no samples within desired window for channel 9574
wclsFrameSaver: no samples within desired window for channel 9575
wclsFrameSaver: no samples within desired window for channel 9654
FrameSaver: q=2.49947e+06 n=1156146 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 9765 XUs and 4844 XVs -> 105 XUVs
C:0 T:1 114532 XUs and 71267 XVs -> 3074 XUVs
C:0 T:2 31351 XUs and 12321 XVs -> 374 XUVs
C:0 T:3 59 XUs and 68 XVs -> 6 XUVs
C:0 T:4 90 XUs and 147 XVs -> 1 XUVs
C:0 T:5 98528 XUs and 86177 XVs -> 4510 XUVs
C:0 T:6 21361 XUs and 17539 XVs -> 473 XUVs
C:0 T:7 73 XUs and 73 XVs -> 3 XUVs
8546 XUVs total
3491 collection wire objects
8546 potential space points
Neighbour search...
343048 tests to find 168330 neighbours
Iterating with no regularization...
Begin: 5.21806e+07
0 4.5966e+07
1 4.54222e+07
2 4.5352e+07
3 4.53329e+07
Now with regularization...
Begin: 4.24614e+07
0 4.24463e+07
21-Jan-2026 18:02:41 PST  Closed output file "np04hd_raw_run027308_0018_dataflow0_datawriter_0_20240619T183931_20260122T020241_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          172.643       312.051       853.558       263.208       130.979        35     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        7.5899e-05    0.000146053   0.000271616   0.000126478    5.019e-05       35     
produce:tpcrawdecoder:PDHDTPCReader                 86.5458       138.145       234.183       129.266       35.8893        35     
produce:triggerrawdecoder:PDHDTriggerReader3       0.132342      0.138655      0.251155      0.133063      0.0219184       35     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0662823     0.0695186     0.119129      0.0667377     0.0102653       35     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.330624       0.35537      0.609314       0.33284      0.0665609       35     
produce:beamevent:BeamEvent                        0.0102194      1.05661        36.02       0.0165855      5.99636        35     
produce:fembfilter:PDHDFEMBFilter                 9.9569e-05    0.000175757   0.00080139    0.000131989   0.000123653      35     
produce:wclsdatahd:WireCellToolkit                  61.5303       68.1336       80.7234       67.556        4.25544        35     
produce:gaushit:GausHitFinder                      0.524787      0.936522       1.65811      0.910237      0.228343        35     
produce:nhitsfilter:NumberOfHitsFilter            0.000245307   0.000419809   0.000660331   0.000420485   9.51017e-05      35     
produce:reco3d:SpacePointSolver                    0.424519       1.9206        4.22716       1.74549      0.968487        33     
produce:hitpdune:DisambigFromSpacePoints            0.29777       1.25262       2.55125       1.2423       0.517578        33     
produce:pandora:StandardPandora                     18.1298       98.826        598.452       55.2033       122.207        33     
produce:pandoraWriter:StandardPandora              0.129249      0.242319       0.39096      0.238448      0.0511027       33     
produce:pandoraTrack:LArPandoraTrackCreation       0.736697       3.02844       6.3197        3.18704       1.17338        33     
produce:pandoraShower:LArPandoraShowerCreation     0.619997       2.88622        5.209        3.11269      0.918561        33     
[art]:TriggerResults:TriggerResultInserter        2.6989e-05    4.86272e-05   0.000117919   4.2609e-05    2.07334e-05      35     
end_path:out1:RootOutput                           4.91e-06     1.03718e-05   3.8059e-05     9.01e-06     6.16377e-06      35     
end_path:out1:RootOutput(write)                    0.534028       1.03101       1.72249      0.990703      0.264714        35     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 38305.1 MB
  Peak resident set size usage (VmHWM): 2117.63 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 727M
-rw-r--r--. 1 cuser cuser 523M Jan 21 18:02 np04hd_raw_run027308_0018_dataflow0_datawriter_0_20240619T183931_20260122T020241_keepup.root
-rw-r--r--. 1 cuser cuser 197M Jan 21 18:02 Pandora_Events.pndr
-rw-r--r--. 1 cuser cuser 6.6M Jan 21 18:02 jobscript.log
-rw-r--r--. 1 cuser cuser 268K Jan 21 18:02 mem.db
-rw-r--r--. 1 cuser cuser  52K Jan 21 18:02 time.db
-rw-r--r--. 1 cuser cuser  18K Jan 21 18:02 np04hd_raw_run027308_0018_dataflow0_datawriter_0_20240619T183931.hdf5_calibana.root
-rw-r--r--. 1 cuser cuser 3.5K Jan 21 18:02 Pandora_Geometry.xml
-rw-r--r--. 1 cuser cuser   83 Jan 21 14:59 all-input-dids.txt
-rw-r--r--. 1 cuser cuser    0 Jan 21 14:59 debugprod.log
disk usage
727M	.
justIN time: 2026-02-05 11:16:40 UTC       justIN version: 01.06.00