justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 292996.2@dunegpschedd02.fnal.gov

Jobsub ID292996.2@dunegpschedd02.fnal.gov
Workflow ID12192
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:35:07
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2026-01-21 19:49:09
From worker nodeHostnamemh-7763-12.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:36:46
Input fileshd-protodune:np04hd_raw_run027306_0001_dataflow0_datawriter_0_20240619T175445.hdf5
JobscriptExit code0
Real time11m (717s)
CPU time2m (160s = 22%)
Max RSS bytes1543770112 (1472 MiB)
Outputting started2026-01-21 19:48:44
Output files
Finished2026-01-21 19:49:09
Saved logsjustin-logs:292996.2-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ndow for channel 9261
wclsFrameSaver: no samples within desired window for channel 9262
wclsFrameSaver: no samples within desired window for channel 9262
wclsFrameSaver: no samples within desired window for channel 9262
wclsFrameSaver: no samples within desired window for channel 9263
wclsFrameSaver: no samples within desired window for channel 9263
wclsFrameSaver: no samples within desired window for channel 9264
wclsFrameSaver: no samples within desired window for channel 9264
wclsFrameSaver: no samples within desired window for channel 9265
wclsFrameSaver: no samples within desired window for channel 9265
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9299
wclsFrameSaver: no samples within desired window for channel 9310
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9390
wclsFrameSaver: no samples within desired window for channel 9391
wclsFrameSaver: no samples within desired window for channel 9391
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9394
wclsFrameSaver: no samples within desired window for channel 9394
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9396
wclsFrameSaver: no samples within desired window for channel 9397
wclsFrameSaver: no samples within desired window for channel 9398
wclsFrameSaver: no samples within desired window for channel 9399
wclsFrameSaver: no samples within desired window for channel 9399
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9401
wclsFrameSaver: no samples within desired window for channel 9401
wclsFrameSaver: no samples within desired window for channel 9402
wclsFrameSaver: no samples within desired window for channel 9402
wclsFrameSaver: no samples within desired window for channel 9403
wclsFrameSaver: no samples within desired window for channel 9403
wclsFrameSaver: no samples within desired window for channel 9404
wclsFrameSaver: no samples within desired window for channel 9404
wclsFrameSaver: no samples within desired window for channel 9405
wclsFrameSaver: no samples within desired window for channel 9406
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9505
wclsFrameSaver: no samples within desired window for channel 9506
wclsFrameSaver: no samples within desired window for channel 9507
wclsFrameSaver: no samples within desired window for channel 9507
wclsFrameSaver: no samples within desired window for channel 9508
wclsFrameSaver: no samples within desired window for channel 9508
wclsFrameSaver: no samples within desired window for channel 9509
wclsFrameSaver: no samples within desired window for channel 9509
wclsFrameSaver: no samples within desired window for channel 9510
wclsFrameSaver: no samples within desired window for channel 9511
wclsFrameSaver: no samples within desired window for channel 9512
wclsFrameSaver: no samples within desired window for channel 9513
wclsFrameSaver: no samples within desired window for channel 9513
wclsFrameSaver: no samples within desired window for channel 9514
wclsFrameSaver: no samples within desired window for channel 9515
wclsFrameSaver: no samples within desired window for channel 9515
wclsFrameSaver: no samples within desired window for channel 9516
wclsFrameSaver: no samples within desired window for channel 9517
wclsFrameSaver: no samples within desired window for channel 9518
wclsFrameSaver: no samples within desired window for channel 9652
FrameSaver: q=1.56046e+06 n=881277 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 4850 XUs and 3447 XVs -> 65 XUVs
C:0 T:1 79968 XUs and 64903 XVs -> 2410 XUVs
C:0 T:2 5062 XUs and 8517 XVs -> 433 XUVs
C:0 T:3 32 XUs and 43 XVs -> 3 XUVs
C:0 T:4 211 XUs and 255 XVs -> 17 XUVs
C:0 T:5 13120 XUs and 22592 XVs -> 1419 XUVs
C:0 T:6 8681 XUs and 9942 XVs -> 183 XUVs
C:0 T:7 110 XUs and 95 XVs -> 3 XUVs
4533 XUVs total
1939 collection wire objects
4533 potential space points
Neighbour search...
200527 tests to find 100826 neighbours
Iterating with no regularization...
Begin: 1.75938e+07
0 1.58258e+07
1 1.56906e+07
2 1.5679e+07
Now with regularization...
Begin: 1.48016e+07
0 1.47976e+07
21-Jan-2026 11:48:34 PST  Opened output file with pattern "%ifb_%tc_keepup.root"
21-Jan-2026 11:48:37 PST  Closed output file "np04hd_raw_run027306_0001_dataflow0_datawriter_0_20240619T175445_20260121T194837_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          643.717       643.717       643.717       643.717          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000198998   0.000198998   0.000198998   0.000198998        0            1     
produce:tpcrawdecoder:PDHDTPCReader                 401.211       401.211       401.211       401.211          0            1     
produce:triggerrawdecoder:PDHDTriggerReader3       0.180719      0.180719      0.180719      0.180719          0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0898109     0.0898109     0.0898109     0.0898109         0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.439291      0.439291      0.439291      0.439291          0            1     
produce:beamevent:BeamEvent                         141.054       141.054       141.054       141.054          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.000847352   0.000847352   0.000847352   0.000847352        0            1     
produce:wclsdatahd:WireCellToolkit                  79.959        79.959        79.959        79.959           0            1     
produce:gaushit:GausHitFinder                      0.692135      0.692135      0.692135      0.692135          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000782492   0.000782492   0.000782492   0.000782492        0            1     
produce:reco3d:SpacePointSolver                     0.75761       0.75761       0.75761       0.75761          0            1     
produce:hitpdune:DisambigFromSpacePoints           0.395052      0.395052      0.395052      0.395052          0            1     
produce:pandora:StandardPandora                     16.1742       16.1742       16.1742       16.1742          0            1     
produce:pandoraWriter:StandardPandora              0.115125      0.115125      0.115125      0.115125          0            1     
produce:pandoraTrack:LArPandoraTrackCreation       0.954908      0.954908      0.954908      0.954908          0            1     
produce:pandoraShower:LArPandoraShowerCreation      0.80745       0.80745       0.80745       0.80745          0            1     
[art]:TriggerResults:TriggerResultInserter        0.000102129   0.000102129   0.000102129   0.000102129        0            1     
end_path:out1:RootOutput                           2.912e-05     2.912e-05     2.912e-05     2.912e-05         0            1     
end_path:out1:RootOutput(write)                    0.828067      0.828067      0.828067      0.828067          0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 37819.9 MB
  Peak resident set size usage (VmHWM): 1543.77 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 15M
-rw-r--r--. 1 cuser cuser  11M Jan 21 11:48 np04hd_raw_run027306_0001_dataflow0_datawriter_0_20240619T175445_20260121T194837_keepup.root
-rw-r--r--. 1 cuser cuser 3.7M Jan 21 11:48 Pandora_Events.pndr
-rw-r--r--. 1 cuser cuser 125K Jan 21 11:48 jobscript.log
-rw-r--r--. 1 cuser cuser  44K Jan 21 11:48 mem.db
-rw-r--r--. 1 cuser cuser  16K Jan 21 11:48 time.db
-rw-r--r--. 1 cuser cuser 7.2K Jan 21 11:48 np04hd_raw_run027306_0001_dataflow0_datawriter_0_20240619T175445.hdf5_calibana.root
-rw-r--r--. 1 cuser cuser 3.5K Jan 21 11:48 Pandora_Geometry.xml
-rw-r--r--. 1 cuser cuser   83 Jan 21 11:36 all-input-dids.txt
-rw-r--r--. 1 cuser cuser    0 Jan 21 11:37 debugprod.log
disk usage
15M	.
justIN time: 2026-02-05 11:18:34 UTC       justIN version: 01.06.00