justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 298724.7@dunegpschedd01.fnal.gov

Jobsub ID298724.7@dunegpschedd01.fnal.gov
Workflow ID12195
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:54:08
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2026-01-21 21:30:37
From worker nodeHostnamesdsc-49.t2.ucsd.edu
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:54:58
Input fileshd-protodune:np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408.hdf5
JobscriptExit code0
Real time1h (5723s)
CPU time1h (4723s = 82%)
Max RSS bytes2021117952 (1927 MiB)
Outputting started2026-01-21 21:30:21
Output files
Finished2026-01-21 21:30:37
Saved logsjustin-logs:298724.7-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

er: no samples within desired window for channel 9285
wclsFrameSaver: no samples within desired window for channel 9286
wclsFrameSaver: no samples within desired window for channel 9287
wclsFrameSaver: no samples within desired window for channel 9601
wclsFrameSaver: no samples within desired window for channel 9602
wclsFrameSaver: no samples within desired window for channel 9603
wclsFrameSaver: no samples within desired window for channel 9603
wclsFrameSaver: no samples within desired window for channel 9604
wclsFrameSaver: no samples within desired window for channel 9604
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9605
wclsFrameSaver: no samples within desired window for channel 9606
wclsFrameSaver: no samples within desired window for channel 9606
wclsFrameSaver: no samples within desired window for channel 9607
wclsFrameSaver: no samples within desired window for channel 9607
wclsFrameSaver: no samples within desired window for channel 9608
wclsFrameSaver: no samples within desired window for channel 9608
wclsFrameSaver: no samples within desired window for channel 9609
wclsFrameSaver: no samples within desired window for channel 9609
wclsFrameSaver: no samples within desired window for channel 9610
wclsFrameSaver: no samples within desired window for channel 9610
wclsFrameSaver: no samples within desired window for channel 9611
wclsFrameSaver: no samples within desired window for channel 9611
wclsFrameSaver: no samples within desired window for channel 9612
wclsFrameSaver: no samples within desired window for channel 9613
wclsFrameSaver: no samples within desired window for channel 9613
wclsFrameSaver: no samples within desired window for channel 9613
wclsFrameSaver: no samples within desired window for channel 9614
wclsFrameSaver: no samples within desired window for channel 9615
wclsFrameSaver: no samples within desired window for channel 9615
wclsFrameSaver: no samples within desired window for channel 9616
wclsFrameSaver: no samples within desired window for channel 9617
wclsFrameSaver: no samples within desired window for channel 9618
wclsFrameSaver: no samples within desired window for channel 9618
wclsFrameSaver: no samples within desired window for channel 9619
wclsFrameSaver: no samples within desired window for channel 9619
wclsFrameSaver: no samples within desired window for channel 9620
wclsFrameSaver: no samples within desired window for channel 9620
wclsFrameSaver: no samples within desired window for channel 9621
wclsFrameSaver: no samples within desired window for channel 9621
wclsFrameSaver: no samples within desired window for channel 9622
wclsFrameSaver: no samples within desired window for channel 9622
wclsFrameSaver: no samples within desired window for channel 9623
wclsFrameSaver: no samples within desired window for channel 9623
wclsFrameSaver: no samples within desired window for channel 9624
wclsFrameSaver: no samples within desired window for channel 9624
wclsFrameSaver: no samples within desired window for channel 9625
wclsFrameSaver: no samples within desired window for channel 9625
wclsFrameSaver: no samples within desired window for channel 9626
wclsFrameSaver: no samples within desired window for channel 9627
wclsFrameSaver: no samples within desired window for channel 9628
wclsFrameSaver: no samples within desired window for channel 9628
wclsFrameSaver: no samples within desired window for channel 9628
wclsFrameSaver: no samples within desired window for channel 9629
wclsFrameSaver: no samples within desired window for channel 9630
wclsFrameSaver: no samples within desired window for channel 9630
wclsFrameSaver: no samples within desired window for channel 9630
wclsFrameSaver: no samples within desired window for channel 9631
wclsFrameSaver: no samples within desired window for channel 9632
wclsFrameSaver: no samples within desired window for channel 9633
wclsFrameSaver: no samples within desired window for channel 9633
wclsFrameSaver: no samples within desired window for channel 9634
wclsFrameSaver: no samples within desired window for channel 9635
wclsFrameSaver: no samples within desired window for channel 9636
wclsFrameSaver: no samples within desired window for channel 9661
wclsFrameSaver: no samples within desired window for channel 9741
wclsFrameSaver: no samples within desired window for channel 9742
wclsFrameSaver: no samples within desired window for channel 9981
wclsFrameSaver: no samples within desired window for channel 9989
FrameSaver: q=2.71596e+06 n=1140537 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 32157 XUs and 13436 XVs -> 400 XUVs
C:0 T:1 255424 XUs and 182303 XVs -> 15033 XUVs
C:0 T:2 20065 XUs and 18530 XVs -> 689 XUVs
C:0 T:3 6205 XUs and 6340 XVs -> 343 XUVs
C:0 T:4 203 XUs and 218 XVs -> 10 XUVs
C:0 T:5 30184 XUs and 91028 XVs -> 2599 XUVs
C:0 T:6 5678 XUs and 7098 XVs -> 617 XUVs
C:0 T:7 121 XUs and 204 XVs -> 21 XUVs
19712 XUVs total
4796 collection wire objects
19712 potential space points
Neighbour search...
2290298 tests to find 671306 neighbours
Iterating with no regularization...
Begin: 1.09833e+08
0 8.76253e+07
1 8.54844e+07
2 8.53041e+07
3 8.52745e+07
Now with regularization...
Begin: 8.2887e+07
0 8.28643e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
21-Jan-2026 13:30:15 PST  Closed output file "np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408_20260121T213015_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          26.8158       155.286       553.807       168.492       135.579        36     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        8.4414e-05    0.000119922   0.000266634   0.000110796   3.30502e-05      36     
produce:tpcrawdecoder:PDHDTPCReader                 26.2482       27.1434       29.497        27.1605      0.676295        36     
produce:triggerrawdecoder:PDHDTriggerReader3        0.13272      0.135218       0.15612      0.133071     0.00585004       36     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0665076     0.0674679     0.077691      0.0666733    0.00230867       36     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.331243      0.335851      0.394513      0.331994      0.0122889       36     
produce:beamevent:BeamEvent                       0.00176725     0.0579763      1.88013     0.00363482      0.30835        36     
produce:fembfilter:PDHDFEMBFilter                 3.6099e-05    0.000125791   0.00068699    0.000134708   0.000107752      36     
produce:wclsdatahd:WireCellToolkit                  108.288       130.204       149.95        129.608       11.3691        19     
produce:gaushit:GausHitFinder                      0.788722       1.33383       2.0403        1.24932      0.345854        19     
produce:nhitsfilter:NumberOfHitsFilter            0.000357056   0.000553857   0.000724029   0.000565288   0.000112658      19     
produce:reco3d:SpacePointSolver                    0.358095       1.93676       5.09349       1.76706       1.22682        19     
produce:hitpdune:DisambigFromSpacePoints            0.50329       1.88346       3.46329       1.65404      0.941617        19     
produce:pandora:StandardPandora                     13.2446       93.1932       370.968       73.7605       79.791         19     
produce:pandoraWriter:StandardPandora               0.15351      0.290717      0.404129      0.285408      0.0694855       19     
produce:pandoraTrack:LArPandoraTrackCreation        1.71065       5.53239       10.0253       4.99579       2.35984        19     
produce:pandoraShower:LArPandoraShowerCreation      2.06029       5.67725       11.3253       4.74529       2.93607        19     
[art]:TriggerResults:TriggerResultInserter        2.4527e-05    5.57191e-05   0.000107809   5.31685e-05   1.80428e-05      36     
end_path:out1:RootOutput                           4.421e-06    1.16492e-05   3.5327e-05    1.0646e-05    6.06603e-06      36     
end_path:out1:RootOutput(write)                    0.0131681     0.756043       2.01005      0.876701      0.739373        36     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11295.6 MB
  Peak resident set size usage (VmHWM): 2021.12 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 345M
-rw-r--r--. 1 cuser cuser 248M Jan 21 13:30 np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408_20260121T213015_keepup.root
-rw-r--r--. 1 cuser cuser  94M Jan 21 13:30 Pandora_Events.pndr
-rw-r--r--. 1 cuser cuser 3.0M Jan 21 13:30 jobscript.log
-rw-r--r--. 1 cuser cuser 220K Jan 21 13:30 mem.db
-rw-r--r--. 1 cuser cuser  48K Jan 21 13:30 time.db
-rw-r--r--. 1 cuser cuser  18K Jan 21 13:30 np04hd_raw_run027306_0000_dataflow1_datawriter_0_20240619T175408.hdf5_calibana.root
-rw-r--r--. 1 cuser cuser 3.5K Jan 21 13:30 Pandora_Geometry.xml
-rw-r--r--. 1 cuser cuser   83 Jan 21 11:55 all-input-dids.txt
-rw-r--r--. 1 cuser cuser    0 Jan 21 11:55 debugprod.log
disk usage
345M	.
justIN time: 2026-02-05 11:20:38 UTC       justIN version: 01.06.00