justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 293047.45@dunegpschedd02.fnal.gov

Jobsub ID293047.45@dunegpschedd02.fnal.gov
Workflow ID12206
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 21:56:16
SiteUS_UChicago
EntryEngage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat2026-01-22 02:06:07
From worker nodeHostnamemwt2-c177.campuscluster.illinois.edu
cpuinfoIntel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 21:57:31
Input fileshd-protodune:np04hd_raw_run027308_0007_dataflow1_datawriter_0_20240619T182639.hdf5
JobscriptExit code0
Real time4h (14900s)
CPU time3h (14035s = 94%)
Max RSS bytes2103132160 (2005 MiB)
Outputting started2026-01-22 02:05:51
Output files
Finished2026-01-22 02:06:07
Saved logsjustin-logs:293047.45-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

00471797 0.00522973 0.00245336 0.000503853 0.000142351 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 2.83848e-05 2.83848e-05 2.83853e-05 2.83848e-05 2.83848e-05 0.000229002 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 
----view 1: 0.00522973 0.00245336 0.000503853 0.000142351 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 0.00205039 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 2.83848e-05 2.83853e-05 2.83848e-05 2.83848e-05 0.000229002 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 
----view 1: 0.00245336 0.000503853 0.000142351 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 0.00205039 0.000181697 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 2.83853e-05 2.83848e-05 2.83848e-05 0.000229002 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 
----view 1: 0.000503853 0.000142351 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 0.00205039 0.000181697 0.00018169 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 2.83848e-05 2.83848e-05 0.000229002 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41548e-05 
----view 1: 0.000142351 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 0.00205039 0.000181697 0.00018169 0.000181697 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 2.83848e-05 0.000229002 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41545e-05 2.4155e-05 2.41548e-05 2.41548e-05 2.41548e-05 
----view 1: 0.00299492 0.00141515 0.000342436 0.00667718 0.00775 0.00973716 0.00694764 0.00632513 0.00205039 0.000181697 0.00018169 0.000181697 0.000181697 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28934e-05 3.28934e-05 3.28925e-05 3.28934e-05 3.28934e-05 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 
----view 1: 0.00371386 0.00543624 0.00475116 0.00536491 0.00277561 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28934e-05 3.28925e-05 3.28934e-05 3.28934e-05 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 
----view 1: 0.00543624 0.00475116 0.00536491 0.00277561 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28925e-05 3.28934e-05 3.28934e-05 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79904e-05 
----view 1: 0.00475116 0.00536491 0.00277561 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 0.00307325 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28934e-05 3.28934e-05 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79904e-05 2.79914e-05 
----view 1: 0.00536491 0.00277561 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 0.00307325 0.000594787 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28934e-05 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79904e-05 2.79914e-05 2.79914e-05 
----view 1: 0.00277561 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 0.00307325 0.000594787 0.000190325 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28929e-05 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79904e-05 2.79914e-05 2.79914e-05 2.79909e-05 
----view 1: 0.000149118 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 0.00307325 0.000594787 0.000190325 0.000190333 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 3.28934e-05 0.000260763 2.79904e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79909e-05 2.79914e-05 2.79904e-05 2.79914e-05 2.79914e-05 2.79909e-05 2.79909e-05 
----view 1: 0.000149116 0.00383677 0.00070766 0.00175726 0.00921983 0.00617069 0.00910884 0.0103764 0.00307325 0.000594787 0.000190325 0.000190333 0.000190325 
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
21-Jan-2026 20:05:49 CST  Closed output file "np04hd_raw_run027308_0007_dataflow1_datawriter_0_20240619T182639_20260122T020549_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          178.338       422.191       1596.4        312.601       282.611        35     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.00010101    0.000143294   0.000336769   0.000126971   4.32172e-05      35     
produce:tpcrawdecoder:PDHDTPCReader                 12.8713       16.7972       19.8833       17.1565       1.52592        35     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0496767     0.0512501     0.0707756     0.0501591    0.00386697       35     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.024982      0.0260406     0.0399668     0.0252858    0.00276216       35     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.123428      0.125229      0.135781      0.124522     0.00261798       35     
produce:beamevent:BeamEvent                        0.0221132     0.0803918      1.00266      0.0378234     0.175615        35     
produce:fembfilter:PDHDFEMBFilter                 0.000172296   0.000261571   0.00105102    0.000201066   0.000176527      35     
produce:wclsdatahd:WireCellToolkit                  127.895       140.066       155.814       139.386       7.49879        35     
produce:gaushit:GausHitFinder                       0.98614       1.86219       2.84815       1.86345       0.46138        35     
produce:nhitsfilter:NumberOfHitsFilter            0.000540366   0.000893732   0.00165006    0.000850315   0.000251411      35     
produce:reco3d:SpacePointSolver                    0.457204       5.42181       17.8336       4.60133       3.45354        34     
produce:hitpdune:DisambigFromSpacePoints           0.642185       3.47663       7.80778       3.58189       1.84376        34     
produce:pandora:StandardPandora                     21.1898       232.139       1408.15       137.146       275.322        34     
produce:pandoraWriter:StandardPandora              0.262602       0.44445      0.657766      0.443074      0.0934332       34     
produce:pandoraTrack:LArPandoraTrackCreation        4.63873       13.7033       33.2847       12.2885       6.4289         34     
produce:pandoraShower:LArPandoraShowerCreation      3.99442       13.4809       28.5573       13.9397       6.09942        34     
[art]:TriggerResults:TriggerResultInserter        7.6429e-05    0.000104529   0.000313158   9.1419e-05    4.38283e-05      35     
end_path:out1:RootOutput                          1.4938e-05    2.29113e-05   6.8211e-05    2.1994e-05    8.10258e-06      35     
end_path:out1:RootOutput(write)                     1.04703       1.82432       2.97749       1.81067      0.397482        35     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14617.1 MB
  Peak resident set size usage (VmHWM): 2103.13 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 718M
-rw-r--r-- 1 dune osgvo 515M Jan 21 20:05 np04hd_raw_run027308_0007_dataflow1_datawriter_0_20240619T182639_20260122T020549_keepup.root
-rw-r--r-- 1 dune osgvo 196M Jan 21 20:05 Pandora_Events.pndr
-rw-r--r-- 1 dune osgvo 7.8M Jan 21 20:05 jobscript.log
-rw-r--r-- 1 dune osgvo 272K Jan 21 20:05 mem.db
-rw-r--r-- 1 dune osgvo  56K Jan 21 20:05 time.db
-rw-r--r-- 1 dune osgvo  18K Jan 21 20:05 np04hd_raw_run027308_0007_dataflow1_datawriter_0_20240619T182639.hdf5_calibana.root
-rw-r--r-- 1 dune osgvo 3.5K Jan 21 20:05 Pandora_Geometry.xml
-rw-r--r-- 1 dune osgvo   83 Jan 21 15:57 all-input-dids.txt
-rw-r--r-- 1 dune osgvo    0 Jan 21 15:58 debugprod.log
disk usage
718M	.
justIN time: 2026-02-05 11:17:10 UTC       justIN version: 01.06.00