justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 292996.7@dunegpschedd02.fnal.gov

Jobsub ID292996.7@dunegpschedd02.fnal.gov
Workflow ID12192
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:35:07
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2026-01-21 19:41:18
From worker nodeHostnamesdsc-49.t2.ucsd.edu
cpuinfoIntel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:36:37
Input fileshd-protodune:np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415.hdf5
JobscriptExit code0
Real time4m (260s)
CPU time3m (182s = 69%)
Max RSS bytes1617711104 (1542 MiB)
Outputting started2026-01-21 19:40:57
Output files
Finished2026-01-21 19:41:18
Saved logsjustin-logs:292996.7-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

w for channel 9214
wclsFrameSaver: no samples within desired window for channel 9214
wclsFrameSaver: no samples within desired window for channel 9218
wclsFrameSaver: no samples within desired window for channel 9227
wclsFrameSaver: no samples within desired window for channel 9253
wclsFrameSaver: no samples within desired window for channel 9279
wclsFrameSaver: no samples within desired window for channel 9294
wclsFrameSaver: no samples within desired window for channel 9295
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9458
wclsFrameSaver: no samples within desired window for channel 9460
wclsFrameSaver: no samples within desired window for channel 9461
wclsFrameSaver: no samples within desired window for channel 9462
wclsFrameSaver: no samples within desired window for channel 9463
wclsFrameSaver: no samples within desired window for channel 9464
wclsFrameSaver: no samples within desired window for channel 9464
wclsFrameSaver: no samples within desired window for channel 9465
wclsFrameSaver: no samples within desired window for channel 9465
wclsFrameSaver: no samples within desired window for channel 9466
wclsFrameSaver: no samples within desired window for channel 9467
wclsFrameSaver: no samples within desired window for channel 9467
wclsFrameSaver: no samples within desired window for channel 9468
wclsFrameSaver: no samples within desired window for channel 9468
wclsFrameSaver: no samples within desired window for channel 9469
wclsFrameSaver: no samples within desired window for channel 9469
wclsFrameSaver: no samples within desired window for channel 9470
wclsFrameSaver: no samples within desired window for channel 9471
wclsFrameSaver: no samples within desired window for channel 9472
wclsFrameSaver: no samples within desired window for channel 9472
wclsFrameSaver: no samples within desired window for channel 9473
wclsFrameSaver: no samples within desired window for channel 9474
wclsFrameSaver: no samples within desired window for channel 9475
wclsFrameSaver: no samples within desired window for channel 9475
wclsFrameSaver: no samples within desired window for channel 9476
wclsFrameSaver: no samples within desired window for channel 9477
wclsFrameSaver: no samples within desired window for channel 9478
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9479
wclsFrameSaver: no samples within desired window for channel 9480
wclsFrameSaver: no samples within desired window for channel 9481
wclsFrameSaver: no samples within desired window for channel 9559
wclsFrameSaver: no samples within desired window for channel 9561
wclsFrameSaver: no samples within desired window for channel 9562
wclsFrameSaver: no samples within desired window for channel 9563
wclsFrameSaver: no samples within desired window for channel 9563
wclsFrameSaver: no samples within desired window for channel 9564
wclsFrameSaver: no samples within desired window for channel 9564
wclsFrameSaver: no samples within desired window for channel 9565
wclsFrameSaver: no samples within desired window for channel 9566
wclsFrameSaver: no samples within desired window for channel 9566
wclsFrameSaver: no samples within desired window for channel 9567
wclsFrameSaver: no samples within desired window for channel 9567
wclsFrameSaver: no samples within desired window for channel 9568
wclsFrameSaver: no samples within desired window for channel 9569
wclsFrameSaver: no samples within desired window for channel 9570
wclsFrameSaver: no samples within desired window for channel 9571
wclsFrameSaver: no samples within desired window for channel 9572
wclsFrameSaver: no samples within desired window for channel 9573
wclsFrameSaver: no samples within desired window for channel 9573
wclsFrameSaver: no samples within desired window for channel 9573
wclsFrameSaver: no samples within desired window for channel 9574
wclsFrameSaver: no samples within desired window for channel 9574
wclsFrameSaver: no samples within desired window for channel 9574
wclsFrameSaver: no samples within desired window for channel 9575
wclsFrameSaver: no samples within desired window for channel 9576
wclsFrameSaver: no samples within desired window for channel 9577
wclsFrameSaver: no samples within desired window for channel 9578
wclsFrameSaver: no samples within desired window for channel 9706
wclsFrameSaver: no samples within desired window for channel 9744
wclsFrameSaver: no samples within desired window for channel 9745
wclsFrameSaver: no samples within desired window for channel 9765
FrameSaver: q=1.74551e+06 n=834850 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 3712 XUs and 3345 XVs -> 22 XUVs
C:0 T:1 30018 XUs and 28316 XVs -> 1135 XUVs
C:0 T:2 4646 XUs and 5776 XVs -> 356 XUVs
C:0 T:3 107 XUs and 113 XVs -> 3 XUVs
C:0 T:4 290 XUs and 315 XVs -> 11 XUVs
C:0 T:5 20983 XUs and 20305 XVs -> 1088 XUVs
C:0 T:6 1225 XUs and 1614 XVs -> 132 XUVs
C:0 T:7 145 XUs and 226 XVs -> 16 XUVs
2763 XUVs total
1598 collection wire objects
2763 potential space points
Neighbour search...
74867 tests to find 30944 neighbours
Iterating with no regularization...
Begin: 3.29317e+07
0 3.10309e+07
1 3.07953e+07
2 3.07854e+07
Now with regularization...
Begin: 2.91268e+07
0 2.91239e+07
21-Jan-2026 11:40:30 PST  Opened output file with pattern "%ifb_%tc_keepup.root"
21-Jan-2026 11:40:33 PST  Closed output file "np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415_20260121T194033_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          167.189       167.189       167.189       167.189          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000216759   0.000216759   0.000216759   0.000216759        0            1     
produce:tpcrawdecoder:PDHDTPCReader                 26.2685       26.2685       26.2685       26.2685          0            1     
produce:triggerrawdecoder:PDHDTriggerReader3        0.13504       0.13504       0.13504       0.13504          0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0674163     0.0674163     0.0674163     0.0674163         0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.333643      0.333643      0.333643      0.333643          0            1     
produce:beamevent:BeamEvent                         2.22181       2.22181       2.22181       2.22181          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.000649659   0.000649659   0.000649659   0.000649659        0            1     
produce:wclsdatahd:WireCellToolkit                  105.048       105.048       105.048       105.048          0            1     
produce:gaushit:GausHitFinder                      0.596422      0.596422      0.596422      0.596422          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000569319   0.000569319   0.000569319   0.000569319        0            1     
produce:reco3d:SpacePointSolver                    0.311452      0.311452      0.311452      0.311452          0            1     
produce:hitpdune:DisambigFromSpacePoints           0.494844      0.494844      0.494844      0.494844          0            1     
produce:pandora:StandardPandora                     28.4786       28.4786       28.4786       28.4786          0            1     
produce:pandoraWriter:StandardPandora              0.126014      0.126014      0.126014      0.126014          0            1     
produce:pandoraTrack:LArPandoraTrackCreation        1.50203       1.50203       1.50203       1.50203          0            1     
produce:pandoraShower:LArPandoraShowerCreation     0.941742      0.941742      0.941742      0.941742          0            1     
[art]:TriggerResults:TriggerResultInserter        6.3043e-05    6.3043e-05    6.3043e-05    6.3043e-05         0            1     
end_path:out1:RootOutput                          2.4059e-05    2.4059e-05    2.4059e-05    2.4059e-05         0            1     
end_path:out1:RootOutput(write)                    0.641315      0.641315      0.641315      0.641315          0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11024.5 MB
  Peak resident set size usage (VmHWM): 1617.71 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 14M
-rw-r--r--. 1 cuser cuser 9.7M Jan 21 11:40 np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415_20260121T194033_keepup.root
-rw-r--r--. 1 cuser cuser 3.4M Jan 21 11:40 Pandora_Events.pndr
-rw-r--r--. 1 cuser cuser 228K Jan 21 11:40 jobscript.log
-rw-r--r--. 1 cuser cuser  44K Jan 21 11:40 mem.db
-rw-r--r--. 1 cuser cuser  16K Jan 21 11:40 time.db
-rw-r--r--. 1 cuser cuser 7.2K Jan 21 11:40 np04hd_raw_run027306_0000_dataflow3_datawriter_0_20240619T175415.hdf5_calibana.root
-rw-r--r--. 1 cuser cuser 3.5K Jan 21 11:40 Pandora_Geometry.xml
-rw-r--r--. 1 cuser cuser   83 Jan 21 11:36 all-input-dids.txt
-rw-r--r--. 1 cuser cuser    0 Jan 21 11:37 debugprod.log
disk usage
14M	.
justIN time: 2026-02-05 11:16:33 UTC       justIN version: 01.06.00