justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 292996.5@dunegpschedd02.fnal.gov

Jobsub ID292996.5@dunegpschedd02.fnal.gov
Workflow ID12192
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:35:07
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2026-01-21 19:50:15
From worker nodeHostnamemh-7763-18.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:36:45
Input fileshd-protodune:np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408.hdf5
JobscriptExit code0
Real time13m (789s)
CPU time2m (177s = 22%)
Max RSS bytes1588191232 (1514 MiB)
Outputting started2026-01-21 19:49:55
Output files
Finished2026-01-21 19:50:15
Saved logsjustin-logs:292996.5-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

9202
wclsFrameSaver: no samples within desired window for channel 9202
wclsFrameSaver: no samples within desired window for channel 9203
wclsFrameSaver: no samples within desired window for channel 9203
wclsFrameSaver: no samples within desired window for channel 9204
wclsFrameSaver: no samples within desired window for channel 9204
wclsFrameSaver: no samples within desired window for channel 9204
wclsFrameSaver: no samples within desired window for channel 9205
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9206
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9207
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9209
wclsFrameSaver: no samples within desired window for channel 9228
wclsFrameSaver: no samples within desired window for channel 9230
wclsFrameSaver: no samples within desired window for channel 9239
wclsFrameSaver: no samples within desired window for channel 9250
wclsFrameSaver: no samples within desired window for channel 9411
wclsFrameSaver: no samples within desired window for channel 9411
wclsFrameSaver: no samples within desired window for channel 9412
wclsFrameSaver: no samples within desired window for channel 9413
wclsFrameSaver: no samples within desired window for channel 9601
wclsFrameSaver: no samples within desired window for channel 9603
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9606
wclsFrameSaver: no samples within desired window for channel 9606
wclsFrameSaver: no samples within desired window for channel 9607
wclsFrameSaver: no samples within desired window for channel 9607
wclsFrameSaver: no samples within desired window for channel 9608
wclsFrameSaver: no samples within desired window for channel 9608
wclsFrameSaver: no samples within desired window for channel 9608
wclsFrameSaver: no samples within desired window for channel 9609
wclsFrameSaver: no samples within desired window for channel 9609
wclsFrameSaver: no samples within desired window for channel 9610
wclsFrameSaver: no samples within desired window for channel 9611
wclsFrameSaver: no samples within desired window for channel 9612
wclsFrameSaver: no samples within desired window for channel 9612
wclsFrameSaver: no samples within desired window for channel 9613
wclsFrameSaver: no samples within desired window for channel 9613
wclsFrameSaver: no samples within desired window for channel 9614
wclsFrameSaver: no samples within desired window for channel 9614
wclsFrameSaver: no samples within desired window for channel 9615
wclsFrameSaver: no samples within desired window for channel 9615
wclsFrameSaver: no samples within desired window for channel 9615
wclsFrameSaver: no samples within desired window for channel 9616
wclsFrameSaver: no samples within desired window for channel 9617
wclsFrameSaver: no samples within desired window for channel 9618
wclsFrameSaver: no samples within desired window for channel 9618
wclsFrameSaver: no samples within desired window for channel 9618
wclsFrameSaver: no samples within desired window for channel 9619
wclsFrameSaver: no samples within desired window for channel 9620
wclsFrameSaver: no samples within desired window for channel 9621
wclsFrameSaver: no samples within desired window for channel 9622
wclsFrameSaver: no samples within desired window for channel 9623
wclsFrameSaver: no samples within desired window for channel 9711
wclsFrameSaver: no samples within desired window for channel 9720
wclsFrameSaver: no samples within desired window for channel 9720
wclsFrameSaver: no samples within desired window for channel 9721
wclsFrameSaver: no samples within desired window for channel 9722
wclsFrameSaver: no samples within desired window for channel 9723
wclsFrameSaver: no samples within desired window for channel 9724
wclsFrameSaver: no samples within desired window for channel 10146
wclsFrameSaver: no samples within desired window for channel 10147
wclsFrameSaver: no samples within desired window for channel 10148
wclsFrameSaver: no samples within desired window for channel 10149
wclsFrameSaver: no samples within desired window for channel 10150
wclsFrameSaver: no samples within desired window for channel 10151
FrameSaver: q=2.5122e+06 n=1163097 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 9345 XUs and 6177 XVs -> 146 XUVs
C:0 T:1 100653 XUs and 74558 XVs -> 5551 XUVs
C:0 T:2 8361 XUs and 12415 XVs -> 395 XUVs
C:0 T:3 147 XUs and 209 XVs -> 7 XUVs
C:0 T:4 119 XUs and 113 XVs -> 4 XUVs
C:0 T:5 19720 XUs and 16555 XVs -> 507 XUVs
C:0 T:6 28412 XUs and 32188 XVs -> 814 XUVs
C:0 T:7 157 XUs and 222 XVs -> 8 XUVs
7432 XUVs total
2815 collection wire objects
7432 potential space points
Neighbour search...
463738 tests to find 141888 neighbours
Iterating with no regularization...
Begin: 5.00705e+07
0 4.41611e+07
1 4.33961e+07
2 4.33597e+07
Now with regularization...
Begin: 4.17258e+07
0 4.17175e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
21-Jan-2026 11:49:48 PST  Opened output file with pattern "%ifb_%tc_keepup.root"
21-Jan-2026 11:49:51 PST  Closed output file "np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408_20260121T194951_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          714.914       714.914       714.914       714.914          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000296167   0.000296167   0.000296167   0.000296167        0            1     
produce:tpcrawdecoder:PDHDTPCReader                  505.4         505.4         505.4         505.4           0            1     
produce:triggerrawdecoder:PDHDTriggerReader3       0.243131      0.243131      0.243131      0.243131          0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.124458      0.124458      0.124458      0.124458          0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.608001      0.608001      0.608001      0.608001          0            1     
produce:beamevent:BeamEvent                         88.4689       88.4689       88.4689       88.4689          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.00124878    0.00124878    0.00124878    0.00124878         0            1     
produce:wclsdatahd:WireCellToolkit                  78.2982       78.2982       78.2982       78.2982          0            1     
produce:gaushit:GausHitFinder                      0.754244      0.754244      0.754244      0.754244          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000786572   0.000786572   0.000786572   0.000786572        0            1     
produce:reco3d:SpacePointSolver                    0.853323      0.853323      0.853323      0.853323          0            1     
produce:hitpdune:DisambigFromSpacePoints           0.625724      0.625724      0.625724      0.625724          0            1     
produce:pandora:StandardPandora                     33.7765       33.7765       33.7765       33.7765          0            1     
produce:pandoraWriter:StandardPandora               0.21024       0.21024       0.21024       0.21024          0            1     
produce:pandoraTrack:LArPandoraTrackCreation        2.01045       2.01045       2.01045       2.01045          0            1     
produce:pandoraShower:LArPandoraShowerCreation      2.17051       2.17051       2.17051       2.17051          0            1     
[art]:TriggerResults:TriggerResultInserter        6.8659e-05    6.8659e-05    6.8659e-05    6.8659e-05         0            1     
end_path:out1:RootOutput                           1.97e-05      1.97e-05      1.97e-05      1.97e-05          0            1     
end_path:out1:RootOutput(write)                     1.30652       1.30652       1.30652       1.30652          0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 37880.3 MB
  Peak resident set size usage (VmHWM): 1588.19 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 20M
-rw-r--r-- 1 cuser cuser  14M Jan 21 11:49 np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408_20260121T194951_keepup.root
-rw-r--r-- 1 cuser cuser 5.2M Jan 21 11:49 Pandora_Events.pndr
-rw-r--r-- 1 cuser cuser 134K Jan 21 11:49 jobscript.log
-rw-r--r-- 1 cuser cuser  44K Jan 21 11:49 mem.db
-rw-r--r-- 1 cuser cuser  16K Jan 21 11:49 time.db
-rw-r--r-- 1 cuser cuser 7.2K Jan 21 11:49 np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408.hdf5_calibana.root
-rw-r--r-- 1 cuser cuser 3.5K Jan 21 11:49 Pandora_Geometry.xml
-rw-r--r-- 1 cuser cuser   83 Jan 21 11:36 all-input-dids.txt
-rw-r--r-- 1 cuser cuser    0 Jan 21 11:37 debugprod.log
disk usage
20M	.
justIN time: 2026-02-05 11:15:05 UTC       justIN version: 01.06.00