justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295528.8@dunegpschedd02.fnal.gov

Jobsub ID295528.8@dunegpschedd02.fnal.gov
Workflow ID12398
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-26 11:58:05
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2026-01-26 12:01:54
From worker nodeHostnamedunegli-8279207-0-fnpc23110.fnal.gov
cpuinfoAMD EPYC 9354 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-26 11:59:20
Input fileshd-protodune:np04hd_raw_run027308_0001_dataflow0_datawriter_0_20240619T182104.hdf5
JobscriptExit code0
Real time1m (118s)
CPU time1m (91s = 77%)
Max RSS bytes1607438336 (1532 MiB)
Outputting started2026-01-26 12:01:18
Output files
Finished2026-01-26 12:01:54
Saved logsjustin-logs:295528.8-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9267
wclsFrameSaver: no samples within desired window for channel 9268
wclsFrameSaver: no samples within desired window for channel 9268
wclsFrameSaver: no samples within desired window for channel 9268
wclsFrameSaver: no samples within desired window for channel 9272
wclsFrameSaver: no samples within desired window for channel 9280
wclsFrameSaver: no samples within desired window for channel 9282
wclsFrameSaver: no samples within desired window for channel 9282
wclsFrameSaver: no samples within desired window for channel 9294
wclsFrameSaver: no samples within desired window for channel 9300
wclsFrameSaver: no samples within desired window for channel 9307
wclsFrameSaver: no samples within desired window for channel 9307
wclsFrameSaver: no samples within desired window for channel 9406
wclsFrameSaver: no samples within desired window for channel 9406
wclsFrameSaver: no samples within desired window for channel 9434
wclsFrameSaver: no samples within desired window for channel 9435
wclsFrameSaver: no samples within desired window for channel 9436
wclsFrameSaver: no samples within desired window for channel 9436
wclsFrameSaver: no samples within desired window for channel 9437
wclsFrameSaver: no samples within desired window for channel 9438
wclsFrameSaver: no samples within desired window for channel 9439
wclsFrameSaver: no samples within desired window for channel 9440
wclsFrameSaver: no samples within desired window for channel 9441
wclsFrameSaver: no samples within desired window for channel 9442
wclsFrameSaver: no samples within desired window for channel 9443
wclsFrameSaver: no samples within desired window for channel 9444
wclsFrameSaver: no samples within desired window for channel 9445
wclsFrameSaver: no samples within desired window for channel 9446
wclsFrameSaver: no samples within desired window for channel 9447
wclsFrameSaver: no samples within desired window for channel 9448
wclsFrameSaver: no samples within desired window for channel 9449
wclsFrameSaver: no samples within desired window for channel 9450
wclsFrameSaver: no samples within desired window for channel 9450
wclsFrameSaver: no samples within desired window for channel 9451
wclsFrameSaver: no samples within desired window for channel 9452
wclsFrameSaver: no samples within desired window for channel 9453
wclsFrameSaver: no samples within desired window for channel 9454
wclsFrameSaver: no samples within desired window for channel 9454
wclsFrameSaver: no samples within desired window for channel 9455
wclsFrameSaver: no samples within desired window for channel 9456
wclsFrameSaver: no samples within desired window for channel 9457
wclsFrameSaver: no samples within desired window for channel 9458
wclsFrameSaver: no samples within desired window for channel 9458
wclsFrameSaver: no samples within desired window for channel 9459
wclsFrameSaver: no samples within desired window for channel 9460
wclsFrameSaver: no samples within desired window for channel 9461
wclsFrameSaver: no samples within desired window for channel 9462
wclsFrameSaver: no samples within desired window for channel 9462
wclsFrameSaver: no samples within desired window for channel 9463
wclsFrameSaver: no samples within desired window for channel 9464
wclsFrameSaver: no samples within desired window for channel 9465
wclsFrameSaver: no samples within desired window for channel 9466
wclsFrameSaver: no samples within desired window for channel 9466
wclsFrameSaver: no samples within desired window for channel 9467
wclsFrameSaver: no samples within desired window for channel 9468
wclsFrameSaver: no samples within desired window for channel 9468
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9481
wclsFrameSaver: no samples within desired window for channel 9487
wclsFrameSaver: no samples within desired window for channel 9491
wclsFrameSaver: no samples within desired window for channel 9508
wclsFrameSaver: no samples within desired window for channel 9509
wclsFrameSaver: no samples within desired window for channel 9513
wclsFrameSaver: no samples within desired window for channel 9531
wclsFrameSaver: no samples within desired window for channel 9532
wclsFrameSaver: no samples within desired window for channel 9715
FrameSaver: q=2.19472e+06 n=1054941 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 6570 XUs and 5013 XVs -> 75 XUVs
C:0 T:1 100668 XUs and 80682 XVs -> 2896 XUVs
C:0 T:2 51037 XUs and 54098 XVs -> 846 XUVs
C:0 T:3 37 XUs and 35 XVs -> 1 XUVs
C:0 T:4 217 XUs and 179 XVs -> 2 XUVs
C:0 T:5 33796 XUs and 36374 XVs -> 658 XUVs
C:0 T:6 36333 XUs and 42048 XVs -> 939 XUVs
C:0 T:7 86 XUs and 108 XVs -> 0 XUVs
5417 XUVs total
2819 collection wire objects
5417 potential space points
Neighbour search...
130411 tests to find 67596 neighbours
Iterating with no regularization...
Begin: 4.95452e+07
0 4.62527e+07
1 4.60578e+07
2 4.60456e+07
Now with regularization...
Begin: 4.44212e+07
0 4.44171e+07
26-Jan-2026 12:01:15 UTC  Opened output file with pattern "%ifb_%tc_keepup.root"
26-Jan-2026 12:01:18 UTC  Closed output file "np04hd_raw_run027308_0001_dataflow0_datawriter_0_20240619T182104_20260126T120118_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          82.5554       82.5554       82.5554       82.5554          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000141295   0.000141295   0.000141295   0.000141295        0            1     
produce:tpcrawdecoder:PDHDTPCReader                 11.2286       11.2286       11.2286       11.2286          0            1     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0548147     0.0548147     0.0548147     0.0548147         0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0274757     0.0274757     0.0274757     0.0274757         0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.134391      0.134391      0.134391      0.134391          0            1     
produce:beamevent:BeamEvent                         6.18558       6.18558       6.18558       6.18558          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.000425028   0.000425028   0.000425028   0.000425028        0            1     
produce:wclsdatahd:WireCellToolkit                  42.4793       42.4793       42.4793       42.4793          0            1     
produce:gaushit:GausHitFinder                      0.430547      0.430547      0.430547      0.430547          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000648698   0.000648698   0.000648698   0.000648698        0            1     
produce:reco3d:SpacePointSolver                    0.810098      0.810098      0.810098      0.810098          0            1     
produce:hitpdune:DisambigFromSpacePoints           0.340413      0.340413      0.340413      0.340413          0            1     
produce:pandora:StandardPandora                     17.517        17.517        17.517        17.517           0            1     
produce:pandoraWriter:StandardPandora              0.116121      0.116121      0.116121      0.116121          0            1     
produce:pandoraTrack:LArPandoraTrackCreation        1.27641       1.27641       1.27641       1.27641          0            1     
produce:pandoraShower:LArPandoraShowerCreation      1.19858       1.19858       1.19858       1.19858          0            1     
[art]:TriggerResults:TriggerResultInserter        0.000107184   0.000107184   0.000107184   0.000107184        0            1     
end_path:out1:RootOutput                          3.1969e-05    3.1969e-05    3.1969e-05    3.1969e-05         0            1     
end_path:out1:RootOutput(write)                     0.6898        0.6898        0.6898        0.6898           0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20604.5 MB
  Peak resident set size usage (VmHWM): 1607.44 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 19M
-rw-r--r-- 1 dunegli fnalgrid  13M Jan 26 12:01 np04hd_raw_run027308_0001_dataflow0_datawriter_0_20240619T182104_20260126T120118_keepup.root
-rw-r--r-- 1 dunegli fnalgrid 4.9M Jan 26 12:01 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 684K Jan 26 12:01 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid  44K Jan 26 12:01 mem.db
-rw-r--r-- 1 dunegli fnalgrid  16K Jan 26 12:01 time.db
-rw-r--r-- 1 dunegli fnalgrid 7.2K Jan 26 12:01 np04hd_raw_run027308_0001_dataflow0_datawriter_0_20240619T182104.hdf5_calibana.root
-rw-r--r-- 1 dunegli fnalgrid 3.5K Jan 26 12:01 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid   83 Jan 26 11:59 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid    0 Jan 26 11:59 debugprod.log
disk usage
19M	.
justIN time: 2026-02-04 20:43:37 UTC       justIN version: 01.06.00