justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295531.9@dunegpschedd02.fnal.gov

Jobsub ID295531.9@dunegpschedd02.fnal.gov
Workflow ID12400
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-26 12:06:05
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-26 12:08:35
From worker nodeHostnamedunegli-8436163-0-fnpc22017.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-26 12:06:28
Input fileshd-protodune:np04hd_raw_run027308_0000_dataflow1_datawriter_0_20240619T182030.hdf5
JobscriptExit code0
Real time1m (116s)
CPU time1m (101s = 87%)
Max RSS bytes1608957952 (1534 MiB)
Outputting started2026-01-26 12:08:24
Output files
Finished2026-01-26 12:08:35
Saved logsjustin-logs:295531.9-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

n desired window for channel 8562
wclsFrameSaver: no samples within desired window for channel 8563
wclsFrameSaver: no samples within desired window for channel 8567
wclsFrameSaver: no samples within desired window for channel 8577
wclsFrameSaver: no samples within desired window for channel 8577
wclsFrameSaver: no samples within desired window for channel 8580
wclsFrameSaver: no samples within desired window for channel 8587
wclsFrameSaver: no samples within desired window for channel 8615
wclsFrameSaver: no samples within desired window for channel 8635
wclsFrameSaver: no samples within desired window for channel 8661
wclsFrameSaver: no samples within desired window for channel 8661
wclsFrameSaver: no samples within desired window for channel 8667
wclsFrameSaver: no samples within desired window for channel 8725
wclsFrameSaver: no samples within desired window for channel 8743
wclsFrameSaver: no samples within desired window for channel 8748
wclsFrameSaver: no samples within desired window for channel 8764
wclsFrameSaver: no samples within desired window for channel 8791
wclsFrameSaver: no samples within desired window for channel 8811
wclsFrameSaver: no samples within desired window for channel 8812
wclsFrameSaver: no samples within desired window for channel 8822
wclsFrameSaver: no samples within desired window for channel 8850
wclsFrameSaver: no samples within desired window for channel 8863
wclsFrameSaver: no samples within desired window for channel 8863
wclsFrameSaver: no samples within desired window for channel 8864
wclsFrameSaver: no samples within desired window for channel 8864
wclsFrameSaver: no samples within desired window for channel 8864
wclsFrameSaver: no samples within desired window for channel 8865
wclsFrameSaver: no samples within desired window for channel 8865
wclsFrameSaver: no samples within desired window for channel 8865
wclsFrameSaver: no samples within desired window for channel 8879
wclsFrameSaver: no samples within desired window for channel 8933
wclsFrameSaver: no samples within desired window for channel 8943
wclsFrameSaver: no samples within desired window for channel 8948
wclsFrameSaver: no samples within desired window for channel 8949
wclsFrameSaver: no samples within desired window for channel 8950
wclsFrameSaver: no samples within desired window for channel 8952
wclsFrameSaver: no samples within desired window for channel 8997
wclsFrameSaver: no samples within desired window for channel 9009
wclsFrameSaver: no samples within desired window for channel 9019
wclsFrameSaver: no samples within desired window for channel 9019
wclsFrameSaver: no samples within desired window for channel 9019
wclsFrameSaver: no samples within desired window for channel 9020
wclsFrameSaver: no samples within desired window for channel 9020
wclsFrameSaver: no samples within desired window for channel 9028
wclsFrameSaver: no samples within desired window for channel 9055
wclsFrameSaver: no samples within desired window for channel 9067
wclsFrameSaver: no samples within desired window for channel 9076
wclsFrameSaver: no samples within desired window for channel 9086
wclsFrameSaver: no samples within desired window for channel 9087
wclsFrameSaver: no samples within desired window for channel 9101
wclsFrameSaver: no samples within desired window for channel 9106
wclsFrameSaver: no samples within desired window for channel 9160
wclsFrameSaver: no samples within desired window for channel 9171
wclsFrameSaver: no samples within desired window for channel 9181
wclsFrameSaver: no samples within desired window for channel 9183
wclsFrameSaver: no samples within desired window for channel 9198
wclsFrameSaver: no samples within desired window for channel 9198
wclsFrameSaver: no samples within desired window for channel 9213
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9225
wclsFrameSaver: no samples within desired window for channel 9227
wclsFrameSaver: no samples within desired window for channel 9250
wclsFrameSaver: no samples within desired window for channel 9262
wclsFrameSaver: no samples within desired window for channel 9266
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9481
wclsFrameSaver: no samples within desired window for channel 9650
wclsFrameSaver: no samples within desired window for channel 9689
wclsFrameSaver: no samples within desired window for channel 9698
wclsFrameSaver: no samples within desired window for channel 10167
FrameSaver: q=2.10492e+06 n=960336 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 8121 XUs and 7399 XVs -> 119 XUVs
C:0 T:1 151567 XUs and 139726 XVs -> 5909 XUVs
C:0 T:2 7650 XUs and 6672 XVs -> 1024 XUVs
C:0 T:3 101 XUs and 173 XVs -> 1 XUVs
C:0 T:4 195 XUs and 118 XVs -> 3 XUVs
C:0 T:5 30445 XUs and 18186 XVs -> 542 XUVs
C:0 T:6 12997 XUs and 5674 XVs -> 170 XUVs
C:0 T:7 193 XUs and 267 XVs -> 10 XUVs
7778 XUVs total
3145 collection wire objects
7778 potential space points
Neighbour search...
357654 tests to find 129438 neighbours
Iterating with no regularization...
Begin: 4.85884e+07
0 4.21211e+07
1 4.15542e+07
2 4.15146e+07
Now with regularization...
Begin: 3.97236e+07
0 3.97152e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
26-Jan-2026 12:08:21 UTC  Opened output file with pattern "%ifb_%tc_keepup.root"
26-Jan-2026 12:08:23 UTC  Closed output file "np04hd_raw_run027308_0000_dataflow1_datawriter_0_20240619T182030_20260126T120823_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          77.6657       77.6657       77.6657       77.6657          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        9.7494e-05    9.7494e-05    9.7494e-05    9.7494e-05         0            1     
produce:tpcrawdecoder:PDHDTPCReader                 10.6732       10.6732       10.6732       10.6732          0            1     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0537075     0.0537075     0.0537075     0.0537075         0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0266076     0.0266076     0.0266076     0.0266076         0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.131216      0.131216      0.131216      0.131216          0            1     
produce:beamevent:BeamEvent                        0.262474      0.262474      0.262474      0.262474          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.000730089   0.000730089   0.000730089   0.000730089        0            1     
produce:wclsdatahd:WireCellToolkit                  47.9378       47.9378       47.9378       47.9378          0            1     
produce:gaushit:GausHitFinder                      0.425579      0.425579      0.425579      0.425579          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000531704   0.000531704   0.000531704   0.000531704        0            1     
produce:reco3d:SpacePointSolver                    0.689944      0.689944      0.689944      0.689944          0            1     
produce:hitpdune:DisambigFromSpacePoints            0.40944       0.40944       0.40944       0.40944          0            1     
produce:pandora:StandardPandora                     14.539        14.539        14.539        14.539           0            1     
produce:pandoraWriter:StandardPandora              0.0880636     0.0880636     0.0880636     0.0880636         0            1     
produce:pandoraTrack:LArPandoraTrackCreation       0.837546      0.837546      0.837546      0.837546          0            1     
produce:pandoraShower:LArPandoraShowerCreation     0.993756      0.993756      0.993756      0.993756          0            1     
[art]:TriggerResults:TriggerResultInserter        0.000101602   0.000101602   0.000101602   0.000101602        0            1     
end_path:out1:RootOutput                          4.4745e-05    4.4745e-05    4.4745e-05    4.4745e-05         0            1     
end_path:out1:RootOutput(write)                    0.570264      0.570264      0.570264      0.570264          0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20596.3 MB
  Peak resident set size usage (VmHWM): 1608.96 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 17M
-rw-r--r-- 1 dunegli fnalgrid  12M Jan 26 12:08 np04hd_raw_run027308_0000_dataflow1_datawriter_0_20240619T182030_20260126T120823_keepup.root
-rw-r--r-- 1 dunegli fnalgrid 4.4M Jan 26 12:08 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 230K Jan 26 12:08 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid  44K Jan 26 12:08 mem.db
-rw-r--r-- 1 dunegli fnalgrid  16K Jan 26 12:08 time.db
-rw-r--r-- 1 dunegli fnalgrid 7.2K Jan 26 12:08 calibana.root
-rw-r--r-- 1 dunegli fnalgrid 3.5K Jan 26 12:08 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid   83 Jan 26 12:06 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid    0 Jan 26 12:06 debugprod.log
disk usage
17M	.
justIN time: 2026-02-04 20:42:12 UTC       justIN version: 01.06.00