justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295531.8@dunegpschedd02.fnal.gov

Jobsub ID295531.8@dunegpschedd02.fnal.gov
Workflow ID12400
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-26 12:06:05
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2026-01-26 12:08:42
From worker nodeHostnamedunegli-8436071-0-fnpc22027.fnal.gov
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-26 12:06:19
Input fileshd-protodune:np04hd_raw_run027308_0000_dataflow0_datawriter_0_20240619T182038.hdf5
JobscriptExit code0
Real time2m (133s)
CPU time1m (116s = 87%)
Max RSS bytes1618259968 (1543 MiB)
Outputting started2026-01-26 12:08:33
Output files
Finished2026-01-26 12:08:42
Saved logsjustin-logs:295531.8-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ired window for channel 9356
wclsFrameSaver: no samples within desired window for channel 9357
wclsFrameSaver: no samples within desired window for channel 9358
wclsFrameSaver: no samples within desired window for channel 9359
wclsFrameSaver: no samples within desired window for channel 9360
wclsFrameSaver: no samples within desired window for channel 9361
wclsFrameSaver: no samples within desired window for channel 9362
wclsFrameSaver: no samples within desired window for channel 9363
wclsFrameSaver: no samples within desired window for channel 9364
wclsFrameSaver: no samples within desired window for channel 9364
wclsFrameSaver: no samples within desired window for channel 9365
wclsFrameSaver: no samples within desired window for channel 9366
wclsFrameSaver: no samples within desired window for channel 9367
wclsFrameSaver: no samples within desired window for channel 9368
wclsFrameSaver: no samples within desired window for channel 9369
wclsFrameSaver: no samples within desired window for channel 9370
wclsFrameSaver: no samples within desired window for channel 9371
wclsFrameSaver: no samples within desired window for channel 9372
wclsFrameSaver: no samples within desired window for channel 9373
wclsFrameSaver: no samples within desired window for channel 9374
wclsFrameSaver: no samples within desired window for channel 9375
wclsFrameSaver: no samples within desired window for channel 9376
wclsFrameSaver: no samples within desired window for channel 9377
wclsFrameSaver: no samples within desired window for channel 9378
wclsFrameSaver: no samples within desired window for channel 9379
wclsFrameSaver: no samples within desired window for channel 9380
wclsFrameSaver: no samples within desired window for channel 9381
wclsFrameSaver: no samples within desired window for channel 9382
wclsFrameSaver: no samples within desired window for channel 9383
wclsFrameSaver: no samples within desired window for channel 9384
wclsFrameSaver: no samples within desired window for channel 9385
wclsFrameSaver: no samples within desired window for channel 9386
wclsFrameSaver: no samples within desired window for channel 9387
wclsFrameSaver: no samples within desired window for channel 9388
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9390
wclsFrameSaver: no samples within desired window for channel 9391
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9394
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9396
wclsFrameSaver: no samples within desired window for channel 9397
wclsFrameSaver: no samples within desired window for channel 9398
wclsFrameSaver: no samples within desired window for channel 9399
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9400
wclsFrameSaver: no samples within desired window for channel 9401
wclsFrameSaver: no samples within desired window for channel 9402
wclsFrameSaver: no samples within desired window for channel 9403
wclsFrameSaver: no samples within desired window for channel 9404
wclsFrameSaver: no samples within desired window for channel 9405
wclsFrameSaver: no samples within desired window for channel 9406
wclsFrameSaver: no samples within desired window for channel 9407
wclsFrameSaver: no samples within desired window for channel 9408
wclsFrameSaver: no samples within desired window for channel 9409
wclsFrameSaver: no samples within desired window for channel 9410
wclsFrameSaver: no samples within desired window for channel 9411
wclsFrameSaver: no samples within desired window for channel 9412
wclsFrameSaver: no samples within desired window for channel 9431
wclsFrameSaver: no samples within desired window for channel 9499
wclsFrameSaver: no samples within desired window for channel 9574
wclsFrameSaver: no samples within desired window for channel 9576
wclsFrameSaver: no samples within desired window for channel 9582
wclsFrameSaver: no samples within desired window for channel 9591
wclsFrameSaver: no samples within desired window for channel 9737
wclsFrameSaver: no samples within desired window for channel 9739
wclsFrameSaver: no samples within desired window for channel 9746
wclsFrameSaver: no samples within desired window for channel 9874
wclsFrameSaver: no samples within desired window for channel 9938
FrameSaver: q=2.70566e+06 n=1179768 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 13728 XUs and 8904 XVs -> 273 XUVs
C:0 T:1 160888 XUs and 132461 XVs -> 8621 XUVs
C:0 T:2 8357 XUs and 7869 XVs -> 320 XUVs
C:0 T:3 159 XUs and 163 XVs -> 5 XUVs
C:0 T:4 291 XUs and 287 XVs -> 3 XUVs
C:0 T:5 30512 XUs and 32425 XVs -> 1007 XUVs
C:0 T:6 26237 XUs and 32932 XVs -> 1259 XUVs
C:0 T:7 128 XUs and 172 XVs -> 3 XUVs
11491 XUVs total
3799 collection wire objects
11491 potential space points
Neighbour search...
935935 tests to find 267346 neighbours
Iterating with no regularization...
Begin: 5.94221e+07
0 5.17817e+07
1 5.10676e+07
2 5.10141e+07
3 5.10054e+07
Now with regularization...
Begin: 4.85421e+07
0 4.85371e+07
26-Jan-2026 12:08:30 UTC  Opened output file with pattern "%ifb_%tc_keepup.root"
26-Jan-2026 12:08:32 UTC  Closed output file "np04hd_raw_run027308_0000_dataflow0_datawriter_0_20240619T182038_20260126T120832_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          96.0581       96.0581       96.0581       96.0581          0            1     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000180602   0.000180602   0.000180602   0.000180602        0            1     
produce:tpcrawdecoder:PDHDTPCReader                  10.77         10.77         10.77         10.77           0            1     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0536792     0.0536792     0.0536792     0.0536792         0            1     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0269436     0.0269436     0.0269436     0.0269436         0            1     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.132993      0.132993      0.132993      0.132993          0            1     
produce:beamevent:BeamEvent                        0.298333      0.298333      0.298333      0.298333          0            1     
produce:fembfilter:PDHDFEMBFilter                 0.000465953   0.000465953   0.000465953   0.000465953        0            1     
produce:wclsdatahd:WireCellToolkit                  48.4615       48.4615       48.4615       48.4615          0            1     
produce:gaushit:GausHitFinder                      0.550379      0.550379      0.550379      0.550379          0            1     
produce:nhitsfilter:NumberOfHitsFilter            0.000704315   0.000704315   0.000704315   0.000704315        0            1     
produce:reco3d:SpacePointSolver                     1.14482       1.14482       1.14482       1.14482          0            1     
produce:hitpdune:DisambigFromSpacePoints           0.644934      0.644934      0.644934      0.644934          0            1     
produce:pandora:StandardPandora                     29.5079       29.5079       29.5079       29.5079          0            1     
produce:pandoraWriter:StandardPandora              0.171575      0.171575      0.171575      0.171575          0            1     
produce:pandoraTrack:LArPandoraTrackCreation        1.6884        1.6884        1.6884        1.6884           0            1     
produce:pandoraShower:LArPandoraShowerCreation      1.8206        1.8206        1.8206        1.8206           0            1     
[art]:TriggerResults:TriggerResultInserter        9.1784e-05    9.1784e-05    9.1784e-05    9.1784e-05         0            1     
end_path:out1:RootOutput                          2.8244e-05    2.8244e-05    2.8244e-05    2.8244e-05         0            1     
end_path:out1:RootOutput(write)                    0.692015      0.692015      0.692015      0.692015          0            1     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20602.9 MB
  Peak resident set size usage (VmHWM): 1618.26 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 21M
-rw-r--r-- 1 dunegli fnalgrid  15M Jan 26 12:08 np04hd_raw_run027308_0000_dataflow0_datawriter_0_20240619T182038_20260126T120832_keepup.root
-rw-r--r-- 1 dunegli fnalgrid 5.5M Jan 26 12:08 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid 206K Jan 26 12:08 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid  44K Jan 26 12:08 mem.db
-rw-r--r-- 1 dunegli fnalgrid  16K Jan 26 12:08 time.db
-rw-r--r-- 1 dunegli fnalgrid 7.2K Jan 26 12:08 calibana.root
-rw-r--r-- 1 dunegli fnalgrid 3.5K Jan 26 12:08 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid   83 Jan 26 12:06 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid    0 Jan 26 12:06 debugprod.log
disk usage
21M	.
justIN time: 2026-02-04 20:42:17 UTC       justIN version: 01.06.00