Jobsub ID 22681.1@dunegpschedd01.fnal.gov
| Jobsub ID | 22681.1@dunegpschedd01.fnal.gov |
| Workflow ID | 682 |
| Stage ID | 1 |
| User name | ichong@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| GPU | No |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2025-08-10 01:17:43 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce02 |
| Last heartbeat | 2025-08-10 01:29:13 |
| From worker node | Hostname | wn1904250.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| GPU | |
| Inner Apptainer? | True |
| Job state | finished |
| Started | 2025-08-10 01:19:02 |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2.root
|
| Jobscript | Exit code | 0 |
| Real time | 9m (578s) |
| CPU time | 2m (127s = 21%) |
| Max RSS bytes | 1948119040 (1857 MiB) |
| Outputting started | 2025-08-10 01:28:42 |
| Output files | https://fndcadoor.fnal.gov:2880/dune/scratch/users/ichong/fnal/00682/1/001/caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_012804Z_caf.root
|
| Finished | 2025-08-10 01:29:13 |
| Saved logs | justin-logs:22681.1-dunegpschedd01.fnal.gov.logs.tgz |
| List job events Cached HTCondor job logs |
Jobscript log (last 10,000 characters)
_vtx_larsoft/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/1f8f8aaded68cc7c3e873196b083ca84b933f518/truth_vtx_larsoft/localProducts_larsoft_v10_04_06d00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Running LArSoft with FCL: /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/7e4ea945688a9906bc9086d3744735f9a7bd40fa/atm-reco_truth_vtx.fcl
Running CAF step with FCL_SECONDARY: cafmaker_atmos_dune10kt_1x2x6_runreco-nuenergy-nuangular_geov5.fcl using reco output: atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_011911Z.root
Renamed caf.root -> caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_012804Z_caf.root
=== Start last 100 lines of lar log file ===
minwire 0: 437
minwire 1: 1560
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary:
Output 0: 0.0564674,
Output 1: 0.0118564, 0.000830009, 0.0101169, 0.977197,
Output 2: 0.947517, 0.0178284, 0.00171756, 0.0329372,
Output 3: 0.973812, 0.0256541, 0.000321686, 0.000212014,
Output 4: 0.0852546, 0.82778, 0.0802336, 0.00673215,
Output 5: 0.999866, 0.000110314, 2.66352e-06, 2.10036e-05,
Output 6: 0.208642, 0.705574, 0.0717362, 0.014048,
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 114!
Begin processing the 10th record. run: 66006611 subRun: 1 event: 49510 at 10-Aug-2025 02:27:51 BST
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:23 56 XUs and 54 XVs -> 26 XUVs
26 XUVs total
13 collection wire objects
26 potential space points
Neighbour search...
186 tests to find 158 neighbours
Iterating with no regularization...
Begin: 552239
0 95803.7
1 65992.4
2 65196.7
3 65048.7
4 65012
Now with regularization...
Begin: -67769.2
0 -67946.4
1 -67978.5
---MC-PARTICLE-MONITORING-----------------------------------------------------------------------
------------------------------------------------------------------------------------------------
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: 280.053, 50.5933, 1310.38
DlVertexingAlgorithm::Run() called. m_trainingMode = 0
DlVertexingAlgorithm::Run() calling Infer()
DlVertexingAlgorithm::Infer() called
DlVertexingAlgorithm: Outputting the truth vertex.
Truth vertex position: 280.053, 50.5933, 1310.38
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 10-Aug-2025 02:27:52 BST run: 66006611 subRun: 1 event: 49510
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 10-Aug-2025 02:27:52 BST run: 66006611 subRun: 1 event: 49510
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 60!
10-Aug-2025 02:28:03 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_011911Z.root"
10-Aug-2025 02:28:03 BST Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/98/ca/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 9.55211 16.2477 61.8654 12.1629 15.2612 10
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.0280811 0.0394838 0.0568934 0.0291319 0.0133537 10
reco:gaushit:GausHitFinder 0.104074 0.124617 0.144287 0.12554 0.0119723 10
reco:spsolve:SpacePointSolver 0.000168767 0.00440142 0.0184035 0.00263507 0.0052621 10
reco:hitfd:DisambigFromSpacePoints 0.000288543 0.00139639 0.00545009 0.000948303 0.00144968 10
reco:rns:RandomNumberSaver 2.5638e-05 9.82818e-05 0.000442272 5.26535e-05 0.000119221 10
reco:pandora:StandardPandora 0.0909704 4.83282 47.1961 0.130784 14.1211 10
reco:pandoraTrack:LArPandoraTrackCreation 0.000133701 0.00152204 0.00801862 0.000968015 0.00222665 10
reco:pandoraShower:LArPandoraModularShowerCreation 0.000168016 0.00530273 0.0161029 0.00280337 0.00534535 10
reco:pandoracalo:Calorimetry 0.000141586 0.00108873 0.00496492 0.000724362 0.0013714 10
reco:pandorapid:Chi2ParticleID 4.6487e-05 0.000367282 0.00240726 0.000158824 0.000683006 10
reco:cvnmap:CVNMapper 3.0116e-05 0.0502054 0.18892 0.0373378 0.0603971 10
reco:cvneva:CVNEvaluator 1.6932e-05 1.02116 3.69751 1.27467 1.08327 10
reco:energyrecnumu:EnergyReco 0.000382419 0.00553861 0.0177969 0.00306053 0.00499407 10
reco:energyrecnue:EnergyReco 0.000188204 0.000879474 0.0026744 0.000535482 0.000904064 10
reco:energyrecnc:EnergyReco 0.00267171 0.00310576 0.00382702 0.00312823 0.000300481 10
reco:energyrecnumurange:EnergyReco 0.000337564 0.00250217 0.00640358 0.00218573 0.0018727 10
reco:energyrecnumumcs:EnergyReco 0.00267581 0.00558851 0.0118903 0.00418696 0.00288076 10
reco:opdec:Deconvolution 0.0997369 0.257104 0.409541 0.2719 0.0952926 10
reco:ophitspe:OpHitFinderDeco 0.247807 0.258016 0.28686 0.252991 0.0122252 10
reco:opflash:OpFlashFinder 0.00021283 0.000873965 0.00466223 0.000427369 0.00127625 10
reco:opslicer:OpSlicer 7.5341e-05 0.0037633 0.0167254 0.00181718 0.00487819 10
[art]:TriggerResults:TriggerResultInserter 1.4277e-05 2.93674e-05 8.1734e-05 2.16405e-05 1.95377e-05 10
end_path:out1:RootOutput 4.488e-06 8.9208e-06 2.6389e-05 7.1485e-06 6.15517e-06 10
end_path:out1:RootOutput(write) 8.87239 9.62638 10.2047 9.90561 0.544 10
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 20728.1 MB
Peak resident set size usage (VmHWM): 1948.12 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
=== Generated output files ===
22681.1_dunegpschedd01.fnal.gov.logs.tgz
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_011911Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_011911Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_012804Z.log
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_012804Z.root
caf_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_495_20231203T041432Z_gen_g4_detsim_hitreco__20240508T071847Z_reco2_truthvtx_2025-08-10T_012804Z_caf.root
debugprod.log
flatcaf.root
jobscript.log
justin-processed-pfns.txt
reco_hist.root
