justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297627.16@dunegpschedd02.fnal.gov

Jobsub ID297627.16@dunegpschedd02.fnal.gov
Workflow ID12664
Stage ID4
User namenivedpm@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit7200 (2 hours)
Submitted time2026-02-03 10:26:43
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2026-02-03 10:38:57
From worker nodeHostnamecn017.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-02-03 10:27:27
Input filesusertests:prod_monte-carlo-012664-000012_gen_g4_detsim.root
JobscriptExit code0
Real time3m (180s)
CPU time2m (136s = 75%)
Max RSS bytes927309824 (884 MiB)
Outputting started2026-02-03 10:30:28
Output filesusertests:prod_monte-carlo-012664-000012_gen_g4_detsim_reco.root
usertests:prod_monte-carlo-012664-000012_gen_g4_detsim_reco_larcv.root
Finished2026-02-03 10:38:57
Saved logsjustin-logs:297627.16-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ORMAL]  <SuperaMCParticleCluster::process> Saved interaction count: 1
    [NORMAL]  <SuperaMCParticleCluster::process> 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 48th record. run: 210012 subRun: 0 event: 48 at 03-Feb-2026 10:30:18 GMT
++++>>>> total num hits: 575, num free: 575

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1601!
Filling voxel information...
   [WARNING]  <SuperaMCParticleCluster::AnalyzeSimChannel> Total IDE ctr 28347
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 0
 ... due to invalid 2d index 0
 ... due to invalid 3d index 1161

    [NORMAL]  <SuperaMCParticleCluster::process> Saved interaction count: 1
    [NORMAL]  <SuperaMCParticleCluster::process> 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 49th record. run: 210012 subRun: 0 event: 49 at 03-Feb-2026 10:30:20 GMT
++++>>>> total num hits: 26879, num free: 26879

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1259!
Filling voxel information...
   [WARNING]  <SuperaMCParticleCluster::AnalyzeSimChannel> Total IDE ctr 25159
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 0
 ... due to invalid 2d index 0
 ... due to invalid 3d index 4285

    [NORMAL]  <SuperaMCParticleCluster::process> Saved interaction count: 1
    [NORMAL]  <SuperaMCParticleCluster::process> 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 50th record. run: 210012 subRun: 0 event: 50 at 03-Feb-2026 10:30:23 GMT
++++>>>> total num hits: 299, num free: 299

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1444!
Filling voxel information...
   [WARNING]  <SuperaMCParticleCluster::AnalyzeSimChannel> Total IDE ctr 27215
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 0
 ... due to invalid 2d index 0
 ... due to invalid 3d index 6710

    [NORMAL]  <SuperaMCParticleCluster::process> Saved interaction count: 1
    [NORMAL]  <SuperaMCParticleCluster::process> 0 particles have no ancestor, summed deposit energy = 0
03-Feb-2026 10:30:26 GMT  Closed output file "prod_monte-carlo-012664-000012_gen_g4_detsim_reco.root"
03-Feb-2026 10:30:26 GMT  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/2a/f6/prod_monte-carlo-012664-000012_gen_g4_detsim.root"
    [NORMAL]  <IOManager::finalize> Writing particle_mpv_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing neutrino_mpv_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_mcst_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_pcluster_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_sed_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_masked_true2reco_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_pcluster_index_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_masked_true_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_pcluster_highE_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_pcluster_lowE_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_pcluster_leftover_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_pcluster_semantics_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing particle_pcluster_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing cluster3d_sed_at_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_chi2_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_occupancy_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_nhits_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_cryo_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_tpc_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge0_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge1_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_charge2_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_time0_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_time1_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_time2_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key0_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key1_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_hit_key2_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_pcluster_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_sed_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_pcluster_semantics_ghost_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing particle_corrected_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Writing sparse3d_reco_rescaled_tree with 50 entries
    [NORMAL]  <IOManager::finalize> Closing output file

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.473829       1.95153       6.10518       1.58323       1.25899        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000311558   0.00710833      0.12061     0.000635773     0.02164        50     
reco:gaushit:GausHitFinder                     0.0410226     0.101491      0.241521      0.0874199     0.0470266       50     
reco:cluster3d:Cluster3D                      0.00336287     0.145168       1.43854      0.0444628     0.273425        50     
reco:opdec:Deconvolution                       0.0922281     0.204622      0.328543      0.210578      0.0588318       50     
reco:ophitspe:OpHitFinderDeco                  0.0293966     0.042924      0.257612      0.0371875     0.0312611       50     
reco:opflash:OpFlashFinder                    0.000534223   0.00162351    0.00258586    0.00165574    0.000533604      50     
reco:opslicer:OpSlicer                         0.0154188     0.252512      0.809653      0.207363      0.175352        50     
reco:mcreco:MCReco                             0.0346143     0.218221      0.858815      0.137624       0.20054        50     
[art]:TriggerResults:TriggerResultInserter    1.6237e-05    2.26145e-05   7.8527e-05    2.15725e-05   8.71799e-06      50     
end_path:superaMC:LArSoftSuperaDriver          0.183545      0.875327       3.16974      0.695088      0.626923        50     
end_path:out1:RootOutput                       3.534e-06    5.43464e-06    1.263e-05     5.105e-06    1.68834e-06      50     
end_path:out1:RootOutput(write)                0.0296101      0.10182      0.299587      0.0899106     0.0516559       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14621 MB
  Peak resident set size usage (VmHWM): 927.31 MB
====================================================================================================
Art has completed and will exit with status 0.
'larcv.root' -> 'prod_monte-carlo-012664-000012_gen_g4_detsim_reco_larcv.root'
Sandbox folder contains:
total 297M
-rw-r--r-- 1 pildune22 pildune   60 Feb  3 10:27 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune    0 Feb  3 10:27 debugprod.log
drwxr-xr-x 2 pildune22 pildune 4.0K Jan 17 01:52 fhicl
-rw-r--r-- 1 pildune22 pildune  12K Feb  3 10:27 fhicl_bundle.tgz
-rw-r--r-- 1 pildune22 pildune  90K Feb  3 10:30 jobscript.log
-rw-r--r-- 1 pildune22 pildune 271M Feb  3 10:30 prod_monte-carlo-012664-000012_gen_g4_detsim_reco.root
-rw-r--r-- 1 pildune22 pildune  26M Feb  3 10:30 prod_monte-carlo-012664-000012_gen_g4_detsim_reco_larcv.root
-rw-r--r-- 1 pildune22 pildune  519 Feb  3 10:30 supera_hist.root
Wrote processed DID: usertests:prod_monte-carlo-012664-000012_gen_g4_detsim.root
=== RECOjob completed successfully ===
justIN time: 2026-02-04 04:46:53 UTC       justIN version: 01.06.00