justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 298724.20@dunegpschedd01.fnal.gov

Jobsub ID298724.20@dunegpschedd01.fnal.gov
Workflow ID12195
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:54:08
SiteUS_UChicago
EntryEngage_US_MWT2_iut2_gk02_condce_mcore
Last heartbeat2026-01-21 21:09:34
From worker nodeHostnameuct2-c547.mwt2.org
cpuinfoAMD EPYC 7402 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:54:48
Input fileshd-protodune:np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415.hdf5
JobscriptExit code0
Real time1h (4469s)
CPU time1h (3893s = 87%)
Max RSS bytes2040213504 (1945 MiB)
Outputting started2026-01-21 21:09:18
Output files
Finished2026-01-21 21:09:34
Saved logsjustin-logs:298724.20-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

28
wclsFrameSaver: no samples within desired window for channel 9428
wclsFrameSaver: no samples within desired window for channel 9429
wclsFrameSaver: no samples within desired window for channel 9430
wclsFrameSaver: no samples within desired window for channel 9431
wclsFrameSaver: no samples within desired window for channel 9431
wclsFrameSaver: no samples within desired window for channel 9432
wclsFrameSaver: no samples within desired window for channel 9433
wclsFrameSaver: no samples within desired window for channel 9434
wclsFrameSaver: no samples within desired window for channel 9435
wclsFrameSaver: no samples within desired window for channel 9436
wclsFrameSaver: no samples within desired window for channel 9497
wclsFrameSaver: no samples within desired window for channel 9511
wclsFrameSaver: no samples within desired window for channel 9522
wclsFrameSaver: no samples within desired window for channel 9590
wclsFrameSaver: no samples within desired window for channel 9592
wclsFrameSaver: no samples within desired window for channel 9593
wclsFrameSaver: no samples within desired window for channel 9597
wclsFrameSaver: no samples within desired window for channel 9598
wclsFrameSaver: no samples within desired window for channel 9599
wclsFrameSaver: no samples within desired window for channel 9599
wclsFrameSaver: no samples within desired window for channel 9600
wclsFrameSaver: no samples within desired window for channel 9600
wclsFrameSaver: no samples within desired window for channel 9600
wclsFrameSaver: no samples within desired window for channel 9601
wclsFrameSaver: no samples within desired window for channel 9602
wclsFrameSaver: no samples within desired window for channel 9603
wclsFrameSaver: no samples within desired window for channel 9603
wclsFrameSaver: no samples within desired window for channel 9604
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9620
wclsFrameSaver: no samples within desired window for channel 9621
wclsFrameSaver: no samples within desired window for channel 9622
wclsFrameSaver: no samples within desired window for channel 9623
wclsFrameSaver: no samples within desired window for channel 9623
wclsFrameSaver: no samples within desired window for channel 9624
wclsFrameSaver: no samples within desired window for channel 9625
wclsFrameSaver: no samples within desired window for channel 9625
wclsFrameSaver: no samples within desired window for channel 9626
wclsFrameSaver: no samples within desired window for channel 9692
wclsFrameSaver: no samples within desired window for channel 9693
wclsFrameSaver: no samples within desired window for channel 9693
wclsFrameSaver: no samples within desired window for channel 9694
wclsFrameSaver: no samples within desired window for channel 9695
wclsFrameSaver: no samples within desired window for channel 9695
wclsFrameSaver: no samples within desired window for channel 9696
wclsFrameSaver: no samples within desired window for channel 9697
wclsFrameSaver: no samples within desired window for channel 9698
wclsFrameSaver: no samples within desired window for channel 9699
wclsFrameSaver: no samples within desired window for channel 9700
wclsFrameSaver: no samples within desired window for channel 9701
wclsFrameSaver: no samples within desired window for channel 9702
wclsFrameSaver: no samples within desired window for channel 9702
wclsFrameSaver: no samples within desired window for channel 9703
wclsFrameSaver: no samples within desired window for channel 9704
wclsFrameSaver: no samples within desired window for channel 9705
wclsFrameSaver: no samples within desired window for channel 9705
wclsFrameSaver: no samples within desired window for channel 9706
wclsFrameSaver: no samples within desired window for channel 9707
wclsFrameSaver: no samples within desired window for channel 9709
wclsFrameSaver: no samples within desired window for channel 9722
wclsFrameSaver: no samples within desired window for channel 9723
wclsFrameSaver: no samples within desired window for channel 9728
wclsFrameSaver: no samples within desired window for channel 9738
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9740
wclsFrameSaver: no samples within desired window for channel 9740
wclsFrameSaver: no samples within desired window for channel 9740
wclsFrameSaver: no samples within desired window for channel 9740
wclsFrameSaver: no samples within desired window for channel 9741
FrameSaver: q=3.04095e+06 n=1326286 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 23388 XUs and 13071 XVs -> 578 XUVs
C:0 T:1 263474 XUs and 202317 XVs -> 14391 XUVs
C:0 T:2 12445 XUs and 9829 XVs -> 241 XUVs
C:0 T:3 366 XUs and 385 XVs -> 0 XUVs
C:0 T:4 37 XUs and 65 XVs -> 0 XUVs
C:0 T:5 36982 XUs and 54263 XVs -> 892 XUVs
C:0 T:6 56587 XUs and 32565 XVs -> 868 XUVs
C:0 T:7 286 XUs and 281 XVs -> 11 XUVs
16981 XUVs total
4602 collection wire objects
16981 potential space points
Neighbour search...
1699307 tests to find 427214 neighbours
Iterating with no regularization...
Begin: 8.41332e+07
0 7.18394e+07
1 7.07338e+07
2 7.06303e+07
3 7.06138e+07
Now with regularization...
Begin: 6.81261e+07
0 6.81164e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
21-Jan-2026 15:09:15 CST  Closed output file "np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415_20260121T210915_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          10.6591       121.11        294.071       143.644       89.6309        36     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        8.6313e-05    0.000159027   0.000335311   0.000153794   5.54691e-05      36     
produce:tpcrawdecoder:PDHDTPCReader                 10.4395       14.2435       59.4118       13.0143       8.21077        36     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0451348     0.0522623      0.18424      0.0458059     0.0260306       36     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.022652      0.026391      0.0961002     0.0230878     0.0139299       36     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.112879      0.130294      0.491843      0.113871      0.0692356       36     
produce:beamevent:BeamEvent                       0.00123085     0.0231638      0.55979     0.00338035     0.0938038       36     
produce:fembfilter:PDHDFEMBFilter                 3.9725e-05    0.000166468   0.000570182   0.000153328   0.000106892      36     
produce:wclsdatahd:WireCellToolkit                  81.3352       88.7539       97.2026       86.3486       5.25872        23     
produce:gaushit:GausHitFinder                       0.69347       1.11973       1.46092       1.09159      0.202356        23     
produce:nhitsfilter:NumberOfHitsFilter            0.000346331   0.00060696    0.00113484    0.000553019   0.000191676      23     
produce:reco3d:SpacePointSolver                    0.366007       2.16278       6.14964       1.70195       1.41748        23     
produce:hitpdune:DisambigFromSpacePoints           0.435423       1.19871       2.29935       1.13375      0.466765        23     
produce:pandora:StandardPandora                     19.0576       62.7659       169.244       45.5192       38.2948        23     
produce:pandoraWriter:StandardPandora              0.195262      0.312831      0.496843      0.300169      0.0716913       23     
produce:pandoraTrack:LArPandoraTrackCreation        1.54501       4.20036       8.30907       3.44787       1.92701        23     
produce:pandoraShower:LArPandoraShowerCreation      1.19899       4.87667       10.1389       4.08821       2.48043        23     
[art]:TriggerResults:TriggerResultInserter        2.0439e-05    8.34156e-05   0.000165952   8.8922e-05    3.14312e-05      36     
end_path:out1:RootOutput                           4.037e-06    2.04529e-05   3.9605e-05    2.2167e-05    1.02842e-05      36     
end_path:out1:RootOutput(write)                    0.0111072     0.810068       1.76758       1.05575      0.625599        36     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 16695.3 MB
  Peak resident set size usage (VmHWM): 2040.21 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 437M
-rw-r--r--. 1 dune osgvo 313M Jan 21 15:09 np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415_20260121T210915_keepup.root
-rw-r--r--. 1 dune osgvo 120M Jan 21 15:08 Pandora_Events.pndr
-rw-r--r--. 1 dune osgvo 4.1M Jan 21 15:09 jobscript.log
-rw-r--r--. 1 dune osgvo 236K Jan 21 15:09 mem.db
-rw-r--r--. 1 dune osgvo  48K Jan 21 15:09 time.db
-rw-r--r--. 1 dune osgvo  18K Jan 21 15:09 np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415.hdf5_calibana.root
-rw-r--r--. 1 dune osgvo 3.5K Jan 21 15:09 Pandora_Geometry.xml
-rw-r--r--. 1 dune osgvo   83 Jan 21 13:54 all-input-dids.txt
-rw-r--r--. 1 dune osgvo    0 Jan 21 13:55 debugprod.log
disk usage
437M	.
justIN time: 2026-02-05 11:20:38 UTC       justIN version: 01.06.00