justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 17439.0@dunegpschedd01.fnal.gov

Jobsub ID17439.0@dunegpschedd01.fnal.gov
Workflow ID54
Stage ID1
User nameepennacc@fnal.gov
HTCondor Groupgroup_dune.prod.mcsim
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-07-23 17:12:58
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2025-07-24 06:03:59
From worker nodeHostnametdm004.pic.es
cpuinfoAMD EPYC 7662 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-07-23 17:14:38
Input filesfardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129.root
JobscriptExit code0
Real time12h (46042s)
CPU time2h (10415s = 22%)
Max RSS bytes1443868672 (1376 MiB)
Outputting started2025-07-24 06:02:01
Output filesfardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root
Finished2025-07-24 06:03:59
Saved logsjustin-logs:17439.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

entified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 17
Printing V view candidate length: 16
Printing W view candidate length: 14
Printing 2D view candidate length: 47
Printing background CaloHitList length: 17
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
Printing U view candidate length: 26
Printing V view candidate length: 23
Printing W view candidate length: 15
Printing 2D view candidate length: 64
Printing background CaloHitList length: 0
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 08:01:40 CEST run: 3786 subRun: 0 event: 49900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 08:01:40 CEST run: 3786 subRun: 0 event: 49900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 08:01:40 CEST run: 3786 subRun: 0 event: 49900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 08:01:40 CEST run: 3786 subRun: 0 event: 49900
Comparing two wires in the same plane: return failure
%MSG
108
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 257!
Genrating OpFlash 0 with 30 hits
Genrating OpFlash 1 with 6 hits
Genrating OpFlash 2 with 4 hits
Genrating OpFlash 3 with 6 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 4 hits
Genrating OpFlash 6 with 5 hits
Genrating OpFlash 7 with 4 hits
Genrating OpFlash 8 with 5 hits
Genrating OpFlash 9 with 13 hits
...
Generated 13 OpFlashes
24-Jul-2025 08:01:53 CEST  Closed output file "fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root"
24-Jul-2025 08:01:53 CEST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/ff/c7/fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              11.8659       16.7132       20.278        16.7563       1.22461       2500    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000372387    0.0182215     0.829091     0.000684343    0.0527553      2500    
reco:linecluster:LineCluster                           0.808985       1.23456       2.13139       1.23042      0.0578149      2500    
reco:trajcluster:TrajCluster                           0.404703      0.407627      0.773748      0.406836      0.010083       2500    
reco:pandora:StandardPandora                            0.41411       3.05719       6.08669       3.05824      0.599312       2500    
reco:pandoraTrack:LArPandoraTrackCreation             0.000124163   0.000458616   0.00476043    0.000451353   0.000191954     2500    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000171383   0.000971461    0.0100306    0.000937698   0.000462486     2500    
reco:pandoracalo:Calorimetry                          9.2041e-05    0.000320679    0.0033595    0.000311471   0.000137754     2500    
reco:pandorapid:Chi2ParticleID                         3.216e-05    8.8265e-05    0.00115552    8.11265e-05   3.82219e-05     2500    
reco:pmtracktc:PMAlgTrackMaker                        0.00133843     0.0420318      1.0775       0.0193415     0.0711914      2500    
reco:pmtracktccalo:Calorimetry                        6.2672e-05    0.00033043    0.00102607    0.000320126   0.000157883     2500    
reco:pmtracktcpid:Chi2ParticleID                       2.438e-05    8.46544e-05   0.000312906   7.9961e-05    3.41518e-05     2500    
reco:emtrkmichelid:EmTrackMichelId                    0.000112572    0.578004       1.77988      0.547697      0.286147       2500    
reco:opdec:Deconvolution                               0.408045      0.843586       1.7574       0.902782      0.136936       2500    
reco:ophitspe:OpHitFinderDeco                         0.000270585     0.76759       2.74009      0.718089      0.383675       2500    
reco:opflash:OpFlashFinder                            0.000136233   0.000647418    0.0138766    0.000610022   0.00035052      2500    
reco:solarflash:SolarOpFlash                          5.5411e-05    0.00605706     0.0276177    0.00520661     0.0042199      2500    
reco:rns:RandomNumberSaver                             1.849e-05    3.28823e-05   0.000308106   2.92005e-05   1.54052e-05     2500    
[art]:TriggerResults:TriggerResultInserter            1.0761e-05    1.63185e-05   0.000103722   1.52155e-05   8.20056e-06     2500    
end_path:out1:RootOutput                                 3e-06      4.54556e-06   0.000100602    4.11e-06     3.98677e-06     2500    
end_path:out1:RootOutput(write)                         9.38641       9.7543        11.9149       9.73403      0.144591       2500    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2972.3 MB
  Peak resident set size usage (VmHWM): 1443.87 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root
Extracted metadata
checking file====== fardet-hd:fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root.ext.json
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root.json
fardet-hd_mc_le_mc_2024a_workflow0000003786_out1_reco1_supernova_dune10kt_1x2x6_hit-reconstructed_merged_skip000050_lim000025_mergeLar_20241217T144129_20250723T171446Z_reco2.root.temp.json
jobscript.log
justin-processed-pfns.txt
old_md.json
reco2_hist.root
justIN time: 2025-08-14 20:16:55 UTC       justIN version: 01.04.00