justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293047.31@dunegpschedd02.fnal.gov

Jobsub ID293047.31@dunegpschedd02.fnal.gov
Workflow ID12206
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 21:56:16
SiteUS_UChicago
EntryEngage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat2026-01-22 01:46:31
From worker nodeHostnamemwt2-c177.campuscluster.illinois.edu
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 21:57:24
Input fileshd-protodune:np04hd_raw_run027308_0004_dataflow1_datawriter_0_20240619T182351.hdf5
JobscriptExit code0
Real time3h (13729s)
CPU time3h (12852s = 93%)
Max RSS bytes2114576384 (2016 MiB)
Outputting started2026-01-22 01:46:14
Output files
Finished2026-01-22 01:46:31
Saved logsjustin-logs:293047.31-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

samples within desired window for channel 9169
wclsFrameSaver: no samples within desired window for channel 9170
wclsFrameSaver: no samples within desired window for channel 9170
wclsFrameSaver: no samples within desired window for channel 9171
wclsFrameSaver: no samples within desired window for channel 9172
wclsFrameSaver: no samples within desired window for channel 9172
wclsFrameSaver: no samples within desired window for channel 9172
wclsFrameSaver: no samples within desired window for channel 9172
wclsFrameSaver: no samples within desired window for channel 9173
wclsFrameSaver: no samples within desired window for channel 9173
wclsFrameSaver: no samples within desired window for channel 9174
wclsFrameSaver: no samples within desired window for channel 9174
wclsFrameSaver: no samples within desired window for channel 9174
wclsFrameSaver: no samples within desired window for channel 9175
wclsFrameSaver: no samples within desired window for channel 9175
wclsFrameSaver: no samples within desired window for channel 9176
wclsFrameSaver: no samples within desired window for channel 9176
wclsFrameSaver: no samples within desired window for channel 9177
wclsFrameSaver: no samples within desired window for channel 9177
wclsFrameSaver: no samples within desired window for channel 9178
wclsFrameSaver: no samples within desired window for channel 9188
wclsFrameSaver: no samples within desired window for channel 9212
wclsFrameSaver: no samples within desired window for channel 9212
wclsFrameSaver: no samples within desired window for channel 9212
wclsFrameSaver: no samples within desired window for channel 9213
wclsFrameSaver: no samples within desired window for channel 9213
wclsFrameSaver: no samples within desired window for channel 9213
wclsFrameSaver: no samples within desired window for channel 9213
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9215
wclsFrameSaver: no samples within desired window for channel 9249
wclsFrameSaver: no samples within desired window for channel 9388
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9394
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9396
wclsFrameSaver: no samples within desired window for channel 9397
wclsFrameSaver: no samples within desired window for channel 9398
wclsFrameSaver: no samples within desired window for channel 9399
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9401
wclsFrameSaver: no samples within desired window for channel 9402
wclsFrameSaver: no samples within desired window for channel 9402
wclsFrameSaver: no samples within desired window for channel 9403
wclsFrameSaver: no samples within desired window for channel 9404
wclsFrameSaver: no samples within desired window for channel 9421
wclsFrameSaver: no samples within desired window for channel 9423
wclsFrameSaver: no samples within desired window for channel 9424
wclsFrameSaver: no samples within desired window for channel 9425
wclsFrameSaver: no samples within desired window for channel 9426
wclsFrameSaver: no samples within desired window for channel 9427
wclsFrameSaver: no samples within desired window for channel 9428
wclsFrameSaver: no samples within desired window for channel 9428
wclsFrameSaver: no samples within desired window for channel 9429
wclsFrameSaver: no samples within desired window for channel 9430
wclsFrameSaver: no samples within desired window for channel 9431
wclsFrameSaver: no samples within desired window for channel 9432
wclsFrameSaver: no samples within desired window for channel 9433
wclsFrameSaver: no samples within desired window for channel 9434
wclsFrameSaver: no samples within desired window for channel 9434
wclsFrameSaver: no samples within desired window for channel 9435
wclsFrameSaver: no samples within desired window for channel 9435
wclsFrameSaver: no samples within desired window for channel 9560
wclsFrameSaver: no samples within desired window for channel 9667
wclsFrameSaver: no samples within desired window for channel 9668
wclsFrameSaver: no samples within desired window for channel 9723
FrameSaver: q=2.84206e+06 n=1059358 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 3458 XUs and 2492 XVs -> 26 XUVs
C:0 T:1 46007 XUs and 43142 XVs -> 1134 XUVs
C:0 T:2 58252 XUs and 108017 XVs -> 2101 XUVs
C:0 T:3 250 XUs and 191 XVs -> 4 XUVs
C:0 T:4 35 XUs and 50 XVs -> 0 XUVs
C:0 T:5 76986 XUs and 59762 XVs -> 3041 XUVs
C:0 T:6 41748 XUs and 64828 XVs -> 568 XUVs
C:0 T:7 70 XUs and 82 XVs -> 3 XUVs
6877 XUVs total
2736 collection wire objects
6877 potential space points
Neighbour search...
288287 tests to find 134682 neighbours
Iterating with no regularization...
Begin: 9.52497e+07
0 8.92577e+07
1 8.88639e+07
2 8.88236e+07
Now with regularization...
Begin: 8.69859e+07
0 8.69715e+07
21-Jan-2026 19:46:12 CST  Closed output file "np04hd_raw_run027308_0004_dataflow1_datawriter_0_20240619T182351_20260122T014612_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          170.371       388.523       1442.09       306.332       302.009        35     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000103613   0.000148801   0.000317805   0.000134135   4.08035e-05      35     
produce:tpcrawdecoder:PDHDTPCReader                 12.8328       16.804        19.6418       17.0858       1.43058        35     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0496874     0.0506004     0.056169      0.0500871    0.00146995       35     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0249294     0.0254823     0.0311794     0.0252316    0.00111234       35     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.123112      0.126291      0.160052      0.124607     0.00656685       35     
produce:beamevent:BeamEvent                        0.0228002     0.0933663     0.925323      0.0420227     0.167111        35     
produce:fembfilter:PDHDFEMBFilter                 0.000169593   0.000239074   0.000968489   0.000204545   0.000132463      35     
produce:wclsdatahd:WireCellToolkit                  129.507       144.806       161.542       146.555       8.24978        35     
produce:gaushit:GausHitFinder                      0.890859       1.70044       2.77929       1.72345      0.439722        35     
produce:nhitsfilter:NumberOfHitsFilter            0.00052251    0.000856815    0.0014676    0.000827997   0.000203992      35     
produce:reco3d:SpacePointSolver                    0.542909       4.53723       23.435        2.91963       4.50615        34     
produce:hitpdune:DisambigFromSpacePoints            0.45436       2.64774       6.42279       2.43839       1.55269        34     
produce:pandora:StandardPandora                     17.8337       197.624       1227.69       100.135       288.995        34     
produce:pandoraWriter:StandardPandora              0.217436      0.420323      0.705213      0.414005      0.100056        34     
produce:pandoraTrack:LArPandoraTrackCreation        2.86864       12.0852       36.6054       11.4222       6.16071        34     
produce:pandoraShower:LArPandoraShowerCreation      2.56397       12.1379       34.8364       11.019        6.3594         34     
[art]:TriggerResults:TriggerResultInserter        6.7674e-05    9.70011e-05   0.000189496    9.076e-05    2.15552e-05      35     
end_path:out1:RootOutput                          1.2428e-05    2.36758e-05   6.9545e-05    2.0689e-05    1.08222e-05      35     
end_path:out1:RootOutput(write)                    0.908964       1.68729       2.89368       1.66689      0.454274        35     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14599.3 MB
  Peak resident set size usage (VmHWM): 2114.58 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 675M
-rw-r--r-- 1 dune osgvo 485M Jan 21 19:46 np04hd_raw_run027308_0004_dataflow1_datawriter_0_20240619T182351_20260122T014612_keepup.root
-rw-r--r-- 1 dune osgvo 184M Jan 21 19:45 Pandora_Events.pndr
-rw-r--r-- 1 dune osgvo 7.1M Jan 21 19:46 jobscript.log
-rw-r--r-- 1 dune osgvo 272K Jan 21 19:46 mem.db
-rw-r--r-- 1 dune osgvo  56K Jan 21 19:46 time.db
-rw-r--r-- 1 dune osgvo  18K Jan 21 19:46 np04hd_raw_run027308_0004_dataflow1_datawriter_0_20240619T182351.hdf5_calibana.root
-rw-r--r-- 1 dune osgvo 3.5K Jan 21 19:46 Pandora_Geometry.xml
-rw-r--r-- 1 dune osgvo   83 Jan 21 15:57 all-input-dids.txt
-rw-r--r-- 1 dune osgvo    0 Jan 21 15:57 debugprod.log
disk usage
675M	.
justIN time: 2026-02-05 11:17:09 UTC       justIN version: 01.06.00