justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 298724.21@dunegpschedd01.fnal.gov

Jobsub ID298724.21@dunegpschedd01.fnal.gov
Workflow ID12195
Stage ID1
User namemsiden@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes2096103424 (1999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2026-01-21 19:54:08
SiteUS_UChicago
EntryEngage_US_MWT2_iut2_gk02_condce_mcore
Last heartbeat2026-01-21 21:07:02
From worker nodeHostnameuct2-c539.mwt2.org
cpuinfoAMD EPYC 7402 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-21 19:54:47
Input fileshd-protodune:np04hd_raw_run027306_0000_dataflow2_datawriter_0_20240619T175409.hdf5
JobscriptExit code0
Real time1h (4318s)
CPU time1h (3769s = 87%)
Max RSS bytes2015318016 (1921 MiB)
Outputting started2026-01-21 21:06:45
Output files
Finished2026-01-21 21:07:02
Saved logsjustin-logs:298724.21-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

FrameSaver: no samples within desired window for channel 9204
wclsFrameSaver: no samples within desired window for channel 9208
wclsFrameSaver: no samples within desired window for channel 9221
wclsFrameSaver: no samples within desired window for channel 9227
wclsFrameSaver: no samples within desired window for channel 9252
wclsFrameSaver: no samples within desired window for channel 9262
wclsFrameSaver: no samples within desired window for channel 9361
wclsFrameSaver: no samples within desired window for channel 9362
wclsFrameSaver: no samples within desired window for channel 9365
wclsFrameSaver: no samples within desired window for channel 9365
wclsFrameSaver: no samples within desired window for channel 9366
wclsFrameSaver: no samples within desired window for channel 9367
wclsFrameSaver: no samples within desired window for channel 9367
wclsFrameSaver: no samples within desired window for channel 9368
wclsFrameSaver: no samples within desired window for channel 9369
wclsFrameSaver: no samples within desired window for channel 9369
wclsFrameSaver: no samples within desired window for channel 9370
wclsFrameSaver: no samples within desired window for channel 9371
wclsFrameSaver: no samples within desired window for channel 9372
wclsFrameSaver: no samples within desired window for channel 9373
wclsFrameSaver: no samples within desired window for channel 9373
wclsFrameSaver: no samples within desired window for channel 9374
wclsFrameSaver: no samples within desired window for channel 9374
wclsFrameSaver: no samples within desired window for channel 9374
wclsFrameSaver: no samples within desired window for channel 9375
wclsFrameSaver: no samples within desired window for channel 9375
wclsFrameSaver: no samples within desired window for channel 9376
wclsFrameSaver: no samples within desired window for channel 9376
wclsFrameSaver: no samples within desired window for channel 9377
wclsFrameSaver: no samples within desired window for channel 9377
wclsFrameSaver: no samples within desired window for channel 9378
wclsFrameSaver: no samples within desired window for channel 9378
wclsFrameSaver: no samples within desired window for channel 9379
wclsFrameSaver: no samples within desired window for channel 9380
wclsFrameSaver: no samples within desired window for channel 9381
wclsFrameSaver: no samples within desired window for channel 9382
wclsFrameSaver: no samples within desired window for channel 9382
wclsFrameSaver: no samples within desired window for channel 9383
wclsFrameSaver: no samples within desired window for channel 9383
wclsFrameSaver: no samples within desired window for channel 9383
wclsFrameSaver: no samples within desired window for channel 9384
wclsFrameSaver: no samples within desired window for channel 9385
wclsFrameSaver: no samples within desired window for channel 9385
wclsFrameSaver: no samples within desired window for channel 9386
wclsFrameSaver: no samples within desired window for channel 9386
wclsFrameSaver: no samples within desired window for channel 9387
wclsFrameSaver: no samples within desired window for channel 9388
wclsFrameSaver: no samples within desired window for channel 9388
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9389
wclsFrameSaver: no samples within desired window for channel 9390
wclsFrameSaver: no samples within desired window for channel 9391
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9392
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9393
wclsFrameSaver: no samples within desired window for channel 9394
wclsFrameSaver: no samples within desired window for channel 9395
wclsFrameSaver: no samples within desired window for channel 9396
wclsFrameSaver: no samples within desired window for channel 9396
wclsFrameSaver: no samples within desired window for channel 9397
wclsFrameSaver: no samples within desired window for channel 9398
wclsFrameSaver: no samples within desired window for channel 9399
wclsFrameSaver: no samples within desired window for channel 9506
wclsFrameSaver: no samples within desired window for channel 9507
wclsFrameSaver: no samples within desired window for channel 9551
wclsFrameSaver: no samples within desired window for channel 9600
wclsFrameSaver: no samples within desired window for channel 9733
FrameSaver: q=1.74185e+06 n=911951 tag=wiener
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 15358 XUs and 6439 XVs -> 305 XUVs
C:0 T:1 161212 XUs and 78579 XVs -> 4054 XUVs
C:0 T:2 2787 XUs and 3527 XVs -> 198 XUVs
C:0 T:3 46 XUs and 123 XVs -> 4 XUVs
C:0 T:4 117 XUs and 62 XVs -> 11 XUVs
C:0 T:5 47492 XUs and 19180 XVs -> 388 XUVs
C:0 T:6 2374 XUs and 1186 XVs -> 38 XUVs
C:0 T:7 1220 XUs and 468 XVs -> 73 XUVs
5071 XUVs total
1995 collection wire objects
5071 potential space points
Neighbour search...
149079 tests to find 67914 neighbours
Iterating with no regularization...
Begin: 3.85016e+07
0 3.4832e+07
1 3.45459e+07
2 3.45292e+07
Now with regularization...
Begin: 3.33206e+07
0 3.33151e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
21-Jan-2026 15:06:41 CST  Closed output file "np04hd_raw_run027306_0000_dataflow2_datawriter_0_20240619T175409_20260121T210641_keepup.root"

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          10.6544       116.669       317.597       150.608       89.3573        36     
------------------------------------------------------------------------------------------------------------------------------------
source:HDF5RawInput3(read)                        0.000104677   0.000149461   0.000374074   0.000141145   4.47155e-05      36     
produce:tpcrawdecoder:PDHDTPCReader                 10.1763       13.3065       39.9581       11.8424       5.5611         36     
produce:triggerrawdecoder:PDHDTriggerReader3       0.0454606     0.0514992     0.216845      0.0457893     0.0283093       36     
produce:timingrawdecoder:PDHDTimingRawDecoder      0.0227998     0.0262272      0.12007      0.0231561     0.0160354       36     
produce:ctbrawdecoder:PDHDCTBRawDecoder            0.112749       0.1289       0.588344      0.113738      0.0785158       36     
produce:beamevent:BeamEvent                       0.00159173     0.0297078      0.77342     0.00412173     0.127723        36     
produce:fembfilter:PDHDFEMBFilter                 6.6435e-05    0.000198423   0.00103532    0.000159019   0.000180557      36     
produce:wclsdatahd:WireCellToolkit                  88.5588       96.1375       101.817       96.3031       3.21775        22     
produce:gaushit:GausHitFinder                      0.503876       1.01868       1.36179       1.01532      0.180491        22     
produce:nhitsfilter:NumberOfHitsFilter            0.000431282   0.000769089   0.00120859    0.000749903   0.000206695      22     
produce:reco3d:SpacePointSolver                    0.621514       2.25159       4.12444       1.9811        1.0939         22     
produce:hitpdune:DisambigFromSpacePoints           0.382328       1.09182       2.11811       1.01596      0.379739        22     
produce:pandora:StandardPandora                     21.9606       57.1074       154.801       46.3048       31.6644        22     
produce:pandoraWriter:StandardPandora              0.182532      0.346219       0.5118       0.350613      0.0806461       22     
produce:pandoraTrack:LArPandoraTrackCreation        1.02809       4.49614       10.3692       4.33776       2.2363         22     
produce:pandoraShower:LArPandoraShowerCreation     0.852368       4.80775       9.25992       4.87354       2.40465        22     
[art]:TriggerResults:TriggerResultInserter        4.8241e-05    8.9387e-05    0.000182143   8.67085e-05   2.6684e-05       36     
end_path:out1:RootOutput                           5.32e-06     1.86259e-05   4.9744e-05    1.8229e-05    9.79799e-06      36     
end_path:out1:RootOutput(write)                    0.011464      0.721172       1.72324      0.959305      0.585539        36     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 16690.2 MB
  Peak resident set size usage (VmHWM): 2015.32 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
lsing
total 407M
-rw-r--r--. 1 dune osgvo 291M Jan 21 15:06 np04hd_raw_run027306_0000_dataflow2_datawriter_0_20240619T175409_20260121T210641_keepup.root
-rw-r--r--. 1 dune osgvo 112M Jan 21 15:06 Pandora_Events.pndr
-rw-r--r--. 1 dune osgvo 4.1M Jan 21 15:06 jobscript.log
-rw-r--r--. 1 dune osgvo 232K Jan 21 15:06 mem.db
-rw-r--r--. 1 dune osgvo  48K Jan 21 15:06 time.db
-rw-r--r--. 1 dune osgvo  18K Jan 21 15:06 np04hd_raw_run027306_0000_dataflow2_datawriter_0_20240619T175409.hdf5_calibana.root
-rw-r--r--. 1 dune osgvo 3.5K Jan 21 15:06 Pandora_Geometry.xml
-rw-r--r--. 1 dune osgvo   83 Jan 21 13:54 all-input-dids.txt
-rw-r--r--. 1 dune osgvo    0 Jan 21 13:55 debugprod.log
disk usage
407M	.
justIN time: 2026-02-05 11:20:38 UTC       justIN version: 01.06.00