Jobsub ID 22072.4@dunegpschedd01.fnal.gov
| Jobsub ID | 22072.4@dunegpschedd01.fnal.gov |
| Workflow ID | 684 |
| Stage ID | 1 |
| User name | ichong@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-08-08 13:18:59 |
| Site | US_UCSD |
| Entry | CMSHTPC_T2_US_UCSD_gw7 |
| Last heartbeat | 2025-08-08 16:59:32 |
| From worker node | Hostname | mh-7763-6.t2.ucsd.edu |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-08-08 13:21:58 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 3h (13031s) |
| CPU time | 14m (869s = 6%) |
| Max RSS bytes | 2026483712 (1932 MiB) |
| Outputting started | 2025-08-08 16:59:10 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00684/1/001/caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_165752Z_caf.root
|
| Finished | 2025-08-08 16:59:32 |
| Saved logs | justin-logs:22072.4-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Running LArSoft with FCL: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/aaa05dc551a4535ad0eeaa35b47a4bc39c7f55ae/atm-reco_truth_vtx.fcl
Running CAF step with FCL_SECONDARY: cafmaker_atmos_dune10kt_1x2x6_runreco-nuenergy-nuangular_geov5.fcl using reco output: atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_132204Z.root
Renamed caf.root -> caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_165752Z_caf.root
=== Start last 100 lines of lar log file ===
Output 6: 0.353392, 0.432709, 0.13696, 0.0769394,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 210!
Begin processing the 199th record. run: 1793 subRun: 1 event: 52999 at 08-Aug-2025 09:57:37 PDT
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
BeamNeutrinos:
--Primary 0, MCPDG 2212, Energy 0.963188, Dist. 0.660113, nMCHits 27 (16, 8, 3)
MCPDG 2212, Energy 0.963188, Dist. 0.660113, nMCHits 27 (16, 8, 3)
------------------------------------------------------------------------------------------------
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -286.357, -183.682, 1176.27
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: -286.357, -183.682, 1176.27
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 08-Aug-2025 09:57:40 PDT run: 1793 subRun: 1 event: 52999
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 08-Aug-2025 09:57:40 PDT run: 1793 subRun: 1 event: 52999
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 24!
Begin processing the 200th record. run: 1793 subRun: 1 event: 53000 at 08-Aug-2025 09:57:43 PDT
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
BeamNeutrinos:
--Primary 0, MCPDG 2212, Energy 0.962612, Dist. 0.642542, nMCHits 17 (8, 7, 2)
MCPDG 2212, Energy 0.962612, Dist. 0.642542, nMCHits 17 (8, 7, 2)
------------------------------------------------------------------------------------------------
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: 74.9195, 391.846, 828.491
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: 74.9195, 391.846, 828.491
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 08-Aug-2025 09:57:47 PDT run: 1793 subRun: 1 event: 53000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 08-Aug-2025 09:57:47 PDT run: 1793 subRun: 1 event: 53000
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 55!
08-Aug-2025 09:57:50 PDT Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_132204Z.root"
08-Aug-2025 09:57:51 PDT Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a7/73/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 3.01732 7.14388 93.3236 5.8262 9.1294 200
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.151791 0.251437 0.69236 0.22483 0.0985929 200
reco:gaushit:GausHitFinder 0.470579 0.786325 5.96053 0.643031 0.567577 200
reco:spsolve:SpacePointSolver 0.00012845 0.0885393 2.73541 0.00492287 0.331384 200
reco:hitfd:DisambigFromSpacePoints 0.000267678 0.022151 0.66709 0.00154684 0.0839234 200
reco:rns:RandomNumberSaver 2.007e-05 3.66766e-05 0.000320998 3.2445e-05 2.26352e-05 200
reco:pandora:StandardPandora 0.463218 1.97794 66.5049 0.527556 7.1048 200
reco:pandoraTrack:LArPandoraTrackCreation 0.000155989 0.00757236 0.223757 0.00201487 0.0214809 200
reco:pandoraShower:LArPandoraModularShowerCreation 0.000232518 0.0125074 0.173378 0.00421547 0.0236891 200
reco:pandoracalo:Calorimetry 0.000132909 0.0254961 1.16229 0.00154816 0.108357 200
reco:pandorapid:Chi2ParticleID 5.036e-05 0.000205955 0.00215982 0.000164884 0.000197639 200
reco:cvnmap:CVNMapper 2.794e-05 0.0248236 0.0742106 0.0262825 0.0205512 200
reco:cvneva:CVNEvaluator 2.176e-05 0.913127 3.03907 1.302 0.644153 200
reco:energyrecnumu:EnergyReco 0.000356708 0.0545583 3.1954 0.00824367 0.290632 200
reco:energyrecnue:EnergyReco 0.000190649 0.00322403 0.113536 0.000933915 0.0102651 200
reco:energyrecnc:EnergyReco 0.00182805 0.00579605 0.124841 0.00341329 0.0111651 200
reco:energyrecnumurange:EnergyReco 0.000212259 0.0514044 3.24004 0.00315563 0.302025 200
reco:energyrecnumumcs:EnergyReco 0.00159143 0.0503512 3.32823 0.00397498 0.279448 200
reco:opdec:Deconvolution 0.491479 1.00449 2.56282 0.955205 0.352541 200
reco:ophitspe:OpHitFinderDeco 1.35592 1.37912 2.08011 1.36763 0.071012 200
reco:opflash:OpFlashFinder 0.000196769 0.000575429 0.00559456 0.000513438 0.000409983 200
reco:opslicer:OpSlicer 4.9469e-05 0.0166384 0.0935082 0.00863458 0.0205032 200
[art]:TriggerResults:TriggerResultInserter 1.494e-05 2.51596e-05 0.000102309 2.241e-05 1.13464e-05 200
end_path:out1:RootOutput 3.929e-06 5.47935e-06 3.3519e-05 5.1505e-06 2.21065e-06 200
end_path:out1:RootOutput(write) 0.00552888 0.465981 1.45279 0.587447 0.30485 200
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 38029.4 MB
Peak resident set size usage (VmHWM): 2026.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
22072.4_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_132204Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_132204Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_165752Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_165752Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_264_20231027T175135Z_gen_g4_detsim_hitreco__20240503T042820Z_reco2_truthvtx_2025-08-08T_165752Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco_hist.root
