Jobsub ID 22071.161@dunegpschedd01.fnal.gov
| Jobsub ID | 22071.161@dunegpschedd01.fnal.gov |
| Workflow ID | 684 |
| Stage ID | 1 |
| User name | ichong@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-08-08 13:14:59 |
| Site | US_UChicago |
| Entry | Engage_US_MWT2_iut2_condce_mcore |
| Last heartbeat | 2025-08-08 15:45:51 |
| From worker node | Hostname | cit2-c030.mwt2.org |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 86400 (24 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-08-08 13:20:23 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 2h (8705s) |
| CPU time | 27m (1675s = 19%) |
| Max RSS bytes | 1999990784 (1907 MiB) |
| Outputting started | 2025-08-08 15:45:28 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00684/1/001/caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_154234Z_caf.root
|
| Finished | 2025-08-08 15:45:51 |
| Saved logs | justin-logs:22071.161-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Running LArSoft with FCL: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/aaa05dc551a4535ad0eeaa35b47a4bc39c7f55ae/atm-reco_truth_vtx.fcl
Running CAF step with FCL_SECONDARY: cafmaker_atmos_dune10kt_1x2x6_runreco-nuenergy-nuangular_geov5.fcl using reco output: atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_132042Z.root
Renamed caf.root -> caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_154234Z_caf.root
=== Start last 100 lines of lar log file ===
0 1.53664e+07
1 1.52953e+07
2 1.52489e+07
3 1.52153e+07
4 1.51902e+07
5 1.51714e+07
6 1.51552e+07
7 1.51412e+07
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
BeamNeutrinos:
--Primary 0, MCPDG 13, Energy 0.6096, Dist. 219.914, nMCHits 1197 (446, 304, 447)
MCPDG 13, Energy 0.6096, Dist. 219.914, nMCHits 1118 (420, 279, 419)
\_ MCPDG 11, Energy 0.0481817, Dist. 13.47, nMCHits 79 (26, 25, 28)
--Primary 1, MCPDG 22, Energy 0.538921, Dist. 16.8444, nMCHits 728 (214, 275, 239)
MCPDG 22, Energy 0.538921, Dist. 16.8444, nMCHits 728 (214, 275, 239)
--Primary 2, MCPDG 211, Energy 1.07173, Dist. 42.5867, nMCHits 267 (49, 130, 88)
MCPDG 211, Energy 1.07173, Dist. 42.5867, nMCHits 165 (22, 84, 59)
\_ MCPDG 2212, Energy 1.10306, Dist. 18.5274, nMCHits 53 (10, 31, 12)
\_ MCPDG 2212, Energy 0.975715, Dist. 1.38479, nMCHits 4 (1, 2, 1)
\_ MCPDG 211, Energy 0.44896, Dist. 10.7674, nMCHits 22 (9, 6, 7)
\_ MCPDG 2212, Energy 1.03504, Dist. 5.44281, nMCHits 12 (3, 4, 5)
\_ MCPDG 2212, Energy 0.982771, Dist. 1.87095, nMCHits 11 (4, 3, 4)
--Primary 3, MCPDG 22, Energy 0.139332, Dist. 4.36778, nMCHits 226 (54, 94, 78)
MCPDG 22, Energy 0.139332, Dist. 4.36778, nMCHits 226 (54, 94, 78)
------------------------------------------------------------------------------------------------
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -35.4526, 146.511, 923.973
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -35.4526, 146.511, 923.973
Boundary wire vector sizes: 1094, 1041, 1106
minwire 0: 1590
minwire 1: 1104
minwire 2: 1358
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.047832,
Output 1: 0.401837, 0.0109104, 0.0705451, 0.516708,
Output 2: 0.0275785, 0.311664, 0.658735, 0.0020221,
Output 3: 0.600591, 0.368588, 0.0243917, 0.00642898,
Output 4: 0.0572307, 0.255697, 0.56064, 0.126432,
Output 5: 0.914044, 0.0781073, 0.00713757, 0.000711312,
Output 6: 0.0779383, 0.595499, 0.228751, 0.0978117,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 430!
08-Aug-2025 11:42:32 EDT Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_132042Z.root"
08-Aug-2025 11:42:33 EDT Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/9e/4e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 2.16384 9.35916 189.463 8.23593 16.818 200
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.1074 0.177883 0.301788 0.217074 0.056573 200
reco:gaushit:GausHitFinder 0.341982 0.446204 2.05534 0.375861 0.215827 200
reco:spsolve:SpacePointSolver 0.000190022 0.140278 12.2305 0.00534944 0.893808 200
reco:hitfd:DisambigFromSpacePoints 0.000353125 0.0364065 2.13078 0.0021402 0.193606 200
reco:rns:RandomNumberSaver 2.7391e-05 5.62395e-05 0.000587499 4.94135e-05 4.10153e-05 200
reco:pandora:StandardPandora 0.337414 2.57506 133.351 0.412207 12.4397 200
reco:pandoraTrack:LArPandoraTrackCreation 0.000249754 0.0113453 0.211177 0.00276106 0.0282685 200
reco:pandoraShower:LArPandoraModularShowerCreation 0.00029544 0.0190392 0.441448 0.00630375 0.0424918 200
reco:pandoracalo:Calorimetry 0.000217475 0.0367524 1.86366 0.00188077 0.163907 200
reco:pandorapid:Chi2ParticleID 7.3004e-05 0.00034013 0.00333521 0.000276463 0.000319835 200
reco:cvnmap:CVNMapper 5.1953e-05 0.0641334 0.215661 0.0770884 0.052132 200
reco:cvneva:CVNEvaluator 3.2553e-05 3.09587 11.5915 4.34592 2.31673 200
reco:energyrecnumu:EnergyReco 0.000571274 0.15314 15.3596 0.0101833 1.12249 200
reco:energyrecnue:EnergyReco 0.000360751 0.00463241 0.0892339 0.00151558 0.0110425 200
reco:energyrecnc:EnergyReco 0.00280169 0.0076725 0.0802823 0.00461143 0.0107322 200
reco:energyrecnumurange:EnergyReco 0.000538398 0.132829 14.2108 0.00417516 1.03022 200
reco:energyrecnumumcs:EnergyReco 0.00179249 0.146701 15.5251 0.00550278 1.12964 200
reco:opdec:Deconvolution 0.340002 0.676604 1.07245 0.745471 0.182348 200
reco:ophitspe:OpHitFinderDeco 0.947021 0.981714 1.05625 0.982382 0.0103089 200
reco:opflash:OpFlashFinder 0.000283475 0.000788392 0.00572316 0.00072855 0.000443593 200
reco:opslicer:OpSlicer 7.7775e-05 0.020267 0.124813 0.00973001 0.0256097 200
[art]:TriggerResults:TriggerResultInserter 2.2981e-05 3.77179e-05 0.000149688 3.44005e-05 1.37459e-05 200
end_path:out1:RootOutput 4.032e-06 1.0913e-05 5.0577e-05 1.00605e-05 7.16326e-06 200
end_path:out1:RootOutput(write) 0.0108057 0.629127 1.67699 0.808391 0.42074 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 7852.36 MB
Peak resident set size usage (VmHWM): 1999.99 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
22071.161_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_132042Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_132042Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_154234Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_154234Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50253389_78_20231120T133454Z_gen_g4_detsim_hitreco__20240503T063201Z_reco2_truthvtx_2025-08-08T_154234Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco_hist.root
