Jobsub ID 297627.6@dunegpschedd02.fnal.gov

Jobsub ID	297627.6@dunegpschedd02.fnal.gov
Workflow ID	12664
Stage ID	4
User name	nivedpm@fnal.gov
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	7200 (2 hours)
Submitted time	2026-02-03 10:26:43
Site	UK_QMUL
Entry	DUNE_UK_London_QMUL_arcce03
Last heartbeat	2026-02-03 10:39:34
From worker node	Hostname	cn053.htc.esc.qmul
	cpuinfo	Intel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2026-02-03 10:27:48
Input files	usertests:prod_monte-carlo-012664-000015_gen_g4_detsim.root
Jobscript	Exit code	0
	Real time	3m (188s)
	CPU time	2m (136s = 72%)
	Max RSS bytes	929624064 (886 MiB)
Outputting started	2026-02-03 10:30:56
Output files	usertests:prod_monte-carlo-012664-000015_gen_g4_detsim_reco.root usertests:prod_monte-carlo-012664-000015_gen_g4_detsim_reco_larcv.root
Finished	2026-02-03 10:39:34
Saved logs	justin-logs:297627.6-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m Saved interaction count: 1
    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 48th record. run: 210015 subRun: 0 event: 48 at 03-Feb-2026 10:30:46 GMT
++++>>>> total num hits: 454, num free: 454

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1644!
Filling voxel information...
   [93m[WARNING][00m [0m [94m<SuperaMCParticleCluster::AnalyzeSimChannel>[00m Total IDE ctr 7076
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 19
 ... due to invalid 2d index 0
 ... due to invalid 3d index 863

    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m Saved interaction count: 1
    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 49th record. run: 210015 subRun: 0 event: 49 at 03-Feb-2026 10:30:48 GMT
++++>>>> total num hits: 164, num free: 164

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 219!
Filling voxel information...
   [93m[WARNING][00m [0m [94m<SuperaMCParticleCluster::AnalyzeSimChannel>[00m Total IDE ctr 9436
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 0
 ... due to invalid 2d index 0
 ... due to invalid 3d index 384

    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m Saved interaction count: 1
    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m 0 particles have no ancestor, summed deposit energy = 0
Begin processing the 50th record. run: 210015 subRun: 0 event: 50 at 03-Feb-2026 10:30:48 GMT
++++>>>> total num hits: 6127, num free: 6127

Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2361!
Filling voxel information...
   [93m[WARNING][00m [0m [94m<SuperaMCParticleCluster::AnalyzeSimChannel>[00m Total IDE ctr 89606
 ... due to missing track ID 0 from 0 track IDs
 ... due to invalid particle 14
 ... due to invalid 2d index 0
 ... due to invalid 3d index 6846

    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m Saved interaction count: 1
    [95m[NORMAL][00m [0m [94m<SuperaMCParticleCluster::process>[00m 0 particles have no ancestor, summed deposit energy = 0
03-Feb-2026 10:30:55 GMT  Closed output file "prod_monte-carlo-012664-000015_gen_g4_detsim_reco.root"
03-Feb-2026 10:30:55 GMT  Closed input file "root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/usertests/c3/f5/prod_monte-carlo-012664-000015_gen_g4_detsim.root"
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing particle_mpv_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing neutrino_mpv_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_mcst_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_pcluster_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_sed_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_masked_true2reco_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_pcluster_index_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_masked_true_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_pcluster_highE_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_pcluster_lowE_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_pcluster_leftover_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_pcluster_semantics_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing particle_pcluster_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing cluster3d_sed_at_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_chi2_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_occupancy_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_nhits_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_cryo_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_tpc_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_charge0_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_charge1_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_charge2_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_time0_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_time1_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_time2_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_key0_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_key1_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_hit_key2_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_pcluster_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_sed_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_pcluster_semantics_ghost_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing particle_corrected_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Writing sparse3d_reco_rescaled_tree with 50 entries
    [95m[NORMAL][00m [0m [94m<IOManager::finalize>[00m Closing output file

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.458412       2.09982       6.58863       1.60793       1.41266        50     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000408959    0.0030596     0.0607814    0.000751237    0.0088337       50     
reco:gaushit:GausHitFinder                     0.0483787     0.106336      0.293248      0.0881065     0.0585495       50     
reco:cluster3d:Cluster3D                      0.00305992     0.121246       1.13046      0.0453484     0.208734        50     
reco:opdec:Deconvolution                       0.0708057     0.199852      0.333055      0.194022      0.0655223       50     
reco:ophitspe:OpHitFinderDeco                  0.0315287     0.040172      0.0543981     0.0390364    0.00574003       50     
reco:opflash:OpFlashFinder                    0.000506953   0.00175967    0.00318594    0.00170248    0.000579824      50     
reco:opslicer:OpSlicer                        0.00472187     0.277205      0.748449       0.22103      0.216428        50     
reco:mcreco:MCReco                             0.0353416     0.277957      0.979722      0.141329      0.268837        50     
[art]:TriggerResults:TriggerResultInserter    1.6147e-05    2.46142e-05   9.8363e-05    2.2749e-05    1.12916e-05      50     
end_path:superaMC:LArSoftSuperaDriver          0.172824      0.956946       3.36225      0.747846      0.719478        50     
end_path:out1:RootOutput                       3.753e-06    5.51804e-06    1.54e-05      5.287e-06    2.10478e-06      50     
end_path:out1:RootOutput(write)                0.022062      0.114545      0.342673      0.105877      0.0642578       50     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 14622.6 MB
  Peak resident set size usage (VmHWM): 929.624 MB
====================================================================================================
Art has completed and will exit with status 0.
'larcv.root' -> 'prod_monte-carlo-012664-000015_gen_g4_detsim_reco_larcv.root'
Sandbox folder contains:
total 309M
-rw-r--r-- 1 pildune32 pildune   60 Feb  3 10:27 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune    0 Feb  3 10:28 debugprod.log
drwxr-xr-x 2 pildune32 pildune 4.0K Jan 17 01:52 fhicl
-rw-r--r-- 1 pildune32 pildune  12K Feb  3 10:27 fhicl_bundle.tgz
-rw-r--r-- 1 pildune32 pildune  90K Feb  3 10:30 jobscript.log
-rw-r--r-- 1 pildune32 pildune 281M Feb  3 10:30 prod_monte-carlo-012664-000015_gen_g4_detsim_reco.root
-rw-r--r-- 1 pildune32 pildune  28M Feb  3 10:30 prod_monte-carlo-012664-000015_gen_g4_detsim_reco_larcv.root
-rw-r--r-- 1 pildune32 pildune  519 Feb  3 10:30 supera_hist.root
Wrote processed DID: usertests:prod_monte-carlo-012664-000015_gen_g4_detsim.root
=== RECOjob completed successfully ===