Jobsub ID 22071.162@dunegpschedd01.fnal.gov
| Jobsub ID | 22071.162@dunegpschedd01.fnal.gov |
| Workflow ID | 684 |
| Stage ID | 1 |
| User name | ichong@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-08-08 13:14:59 |
| Site | US_UChicago |
| Entry | Engage_US_MWT2_iut2_condce_mcore |
| Last heartbeat | 2025-08-08 15:51:15 |
| From worker node | Hostname | cit2-c030.mwt2.org |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-08-08 13:20:27 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9029s) |
| CPU time | 33m (1990s = 22%) |
| Max RSS bytes | 2086260736 (1989 MiB) |
| Outputting started | 2025-08-08 15:50:57 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00684/1/001/caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_154722Z_caf.root
|
| Finished | 2025-08-08 15:51:15 |
| Saved logs | justin-logs:22071.162-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
t/localProducts_larsoft_v10_04_06d00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Running LArSoft with FCL: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/aaa05dc551a4535ad0eeaa35b47a4bc39c7f55ae/atm-reco_truth_vtx.fcl
Running CAF step with FCL_SECONDARY: cafmaker_atmos_dune10kt_1x2x6_runreco-nuenergy-nuangular_geov5.fcl using reco output: atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_132042Z.root
Renamed caf.root -> caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_154722Z_caf.root
=== Start last 100 lines of lar log file ===
Now with regularization...
Begin: 6.67277e+07
0 6.67193e+07
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
BeamNeutrinos:
--Primary 0, MCPDG 321, Energy 0.711265, Dist. 39.3488, nMCHits 373 (82, 162, 129)
MCPDG 321, Energy 0.711265, Dist. 39.3488, nMCHits 70 (4, 43, 23)
\_ MCPDG -13, Energy 0.25454, Dist. 49.1127, nMCHits 225 (57, 85, 83)
\_ MCPDG -11, Energy 0.0475989, Dist. 10.092, nMCHits 78 (21, 34, 23)
--Primary 1, MCPDG 211, Energy 0.257844, Dist. 31.5375, nMCHits 116 (11, 61, 44)
MCPDG 211, Energy 0.257844, Dist. 31.5375, nMCHits 105 (7, 58, 40)
\_ MCPDG -11, Energy 0.0438454, Dist. 4.06887, nMCHits 11 (4, 3, 4)
--Primary 2, MCPDG -321, Energy 1.08721, Dist. 24.1004, nMCHits 62 (25, 29, 8)
MCPDG -321, Energy 1.08721, Dist. 24.1004, nMCHits 48 (20, 26, 2)
\_ MCPDG 22, Energy 0.178193, Dist. 0, nMCHits 14 (5, 3, 6)
--Primary 3, MCPDG 2212, Energy 1.15313, Dist. 19.6691, nMCHits 52 (7, 31, 14)
MCPDG 2212, Energy 1.15313, Dist. 19.6691, nMCHits 47 (5, 29, 13)
\_ MCPDG 1000180380, Energy 35.3533, Dist. 7.47525e-05, nMCHits 5 (2, 2, 1)
--Primary 4, MCPDG 13, Energy 0.269355, Dist. 16.9306, nMCHits 40 (19, 13, 8)
MCPDG 13, Energy 0.269355, Dist. 16.9306, nMCHits 40 (19, 13, 8)
--Primary 5, MCPDG 2212, Energy 1.0682, Dist. 12.3439, nMCHits 35 (11, 20, 4)
MCPDG 2212, Energy 1.0682, Dist. 12.3439, nMCHits 35 (11, 20, 4)
------------------------------------------------------------------------------------------------
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -353.7, -156.278, 317.228
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -353.7, -156.278, 317.228
Boundary wire vector sizes: 382, 217, 264
minwire 0: 858
minwire 1: 1569
minwire 2: 321
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.00912787,
Output 1: 0.168301, 0.00178174, 0.00510764, 0.82481,
Output 2: 0.0198049, 0.756838, 0.22109, 0.00226714,
Output 3: 0.0580348, 0.0920136, 0.379157, 0.470795,
Output 4: 0.00973563, 0.056228, 0.616891, 0.317146,
Output 5: 0.994417, 0.00535674, 0.000201914, 2.42463e-05,
Output 6: 0.0731148, 0.342364, 0.270432, 0.314089,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 279!
08-Aug-2025 11:47:21 EDT Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_132042Z.root"
08-Aug-2025 11:47:21 EDT Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/63/8d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 2.17426 10.7305 292.618 8.33519 26.1728 200
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.108732 0.17443 0.25487 0.14841 0.0562457 200
reco:gaushit:GausHitFinder 0.344648 0.467436 3.08128 0.377041 0.284781 200
reco:spsolve:SpacePointSolver 0.000184141 0.266306 19.5356 0.00585794 1.53421 200
reco:hitfd:DisambigFromSpacePoints 0.000322793 0.035034 1.97076 0.00238433 0.163262 200
reco:rns:RandomNumberSaver 3.0281e-05 5.47429e-05 0.000488336 4.9329e-05 3.42459e-05 200
reco:pandora:StandardPandora 0.340063 3.27888 211.03 0.411288 20.5669 200
reco:pandoraTrack:LArPandoraTrackCreation 0.000265572 0.014908 0.668662 0.0028204 0.0548522 200
reco:pandoraShower:LArPandoraModularShowerCreation 0.000292532 0.0301402 0.645459 0.00579769 0.0732885 200
reco:pandoracalo:Calorimetry 0.000212669 0.048961 3.21856 0.00213884 0.264507 200
reco:pandorapid:Chi2ParticleID 8.1978e-05 0.000335149 0.00289614 0.000262946 0.000301782 200
reco:cvnmap:CVNMapper 4.2816e-05 0.0823402 0.326981 0.0917657 0.0643458 200
reco:cvneva:CVNEvaluator 3.111e-05 3.2635 8.76615 4.32471 2.1277 200
reco:energyrecnumu:EnergyReco 0.000524997 0.224581 25.0967 0.0108602 1.79877 200
reco:energyrecnue:EnergyReco 0.00034903 0.00625304 0.273363 0.00130475 0.0259739 200
reco:energyrecnc:EnergyReco 0.00263321 0.00910606 0.273254 0.00424436 0.0259163 200
reco:energyrecnumurange:EnergyReco 0.000508825 0.226687 26.387 0.00394951 1.88895 200
reco:energyrecnumumcs:EnergyReco 0.00277213 0.227956 26.0375 0.00544872 1.86604 200
reco:opdec:Deconvolution 0.32639 0.686646 1.20349 0.748633 0.186393 200
reco:ophitspe:OpHitFinderDeco 0.970068 0.982927 1.09558 0.981053 0.0124772 200
reco:opflash:OpFlashFinder 0.000214678 0.000761865 0.00505692 0.000720898 0.000403295 200
reco:opslicer:OpSlicer 5.8028e-05 0.0200223 0.145602 0.00943463 0.0266457 200
[art]:TriggerResults:TriggerResultInserter 1.7446e-05 3.25036e-05 0.000118439 3.04955e-05 1.04692e-05 200
end_path:out1:RootOutput 3.908e-06 8.62664e-06 3.9953e-05 8.535e-06 4.36215e-06 200
end_path:out1:RootOutput(write) 0.0094577 0.681046 1.95223 0.83893 0.412986 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 7944.3 MB
Peak resident set size usage (VmHWM): 2086.26 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
22071.162_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_132042Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_132042Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_154722Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_154722Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50258586_434_20231122T080945Z_gen_g4_detsim_hitreco__20240503T060005Z_reco2_truthvtx_2025-08-08T_154722Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco_hist.root 