Jobsub ID 17624.45@dunegpschedd01.fnal.gov

Jobsub ID	17624.45@dunegpschedd01.fnal.gov
Workflow ID	55
Stage ID	1
User name	higuera@fnal.gov
HTCondor Group	group_dune.prod.mcsim
Requested	Processors	1
	GPU	No
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-07-24 09:29:52
Site	US_FNAL-T1
Entry	CMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat	2025-07-24 22:37:47
From worker node	Hostname	dunegli-44332-0-cmswn3007.fnal.gov
	cpuinfo	AMD EPYC 7542 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-07-24 13:04:15
Input files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco.root
Jobscript	Exit code	0
	Real time	9h (34299s)
	CPU time	9h (33897s = 98%)
	Max RSS bytes	1734426624 (1654 MiB)
Outputting started	2025-07-24 22:35:54
Output files	fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root
Finished	2025-07-24 22:37:47
Saved logs	justin-logs:17624.45-dunegpschedd01.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
*Signal Pixel identified*
Printing U view candidate length: 33
Printing V view candidate length: 34
Printing W view candidate length: 17
Printing 2D view candidate length: 84
Printing background CaloHitList length: 564
Printing New CaloHitList Names: 
SCaloHitListU | SCaloHitListV | SCaloHitListW | SCaloHitList2D | BBCaloHitList
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Electron identified*
*Photon identified*
*Photon identified*
Printing U view candidate length: 32
Printing V view candidate length: 34
Printing W view candidate length: 14
Printing 2D view candidate length: 80
Printing background CaloHitList length: 4
Printing New CaloHitList Names: 
CaloHitListU | CaloHitListV | CaloHitListW | CaloHitList2D | BCaloHitList
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  24-Jul-2025 22:31:12 UTC run: 5089 subRun: 0 event: 222950
Comparing two wires in the same plane: return failure
%MSG
54702
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 70966!
Genrating OpFlash 0 with 5 hits
Genrating OpFlash 1 with 4 hits
Genrating OpFlash 2 with 4 hits
Genrating OpFlash 3 with 5 hits
Genrating OpFlash 4 with 3 hits
Genrating OpFlash 5 with 6 hits
Genrating OpFlash 6 with 3 hits
Genrating OpFlash 7 with 14 hits
Genrating OpFlash 8 with 6 hits
Genrating OpFlash 9 with 3 hits
...
Generated 2142 OpFlashes
24-Jul-2025 22:35:46 UTC  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root"
24-Jul-2025 22:35:46 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/76/40/prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              502.66        681.349       769.291       691.465       56.5814        50     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00880428     0.0199064     0.0480576     0.0181597    0.00758835       50     
reco:linecluster:LineCluster                           0.0427023     0.0633317     0.224838      0.0580254     0.0265131       50     
reco:trajcluster:TrajCluster                           0.0111238     0.0199796     0.0696619     0.0173545     0.010696        50     
reco:pandora:StandardPandora                            221.484       396.232       473.081       407.638       56.1074        50     
reco:pandoraTrack:LArPandoraTrackCreation             0.00054387     0.0194067     0.0453212     0.0191781     0.0103279       50     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000438809   0.00344249     0.0569356    0.00208569    0.00771812       50     
reco:pandoracalo:Calorimetry                          0.000330161   0.00118827     0.018633     0.000678388    0.0025278       50     
reco:pandorapid:Chi2ParticleID                        8.5694e-05    0.000334173   0.00604888    0.000189603   0.000824072      50     
reco:pmtracktc:PMAlgTrackMaker                        0.00411867     0.0444314     0.159011      0.0321591     0.0397385       50     
reco:pmtracktccalo:Calorimetry                        0.000459068   0.00108859    0.00242272    0.000935559   0.000510834      50     
reco:pmtracktcpid:Chi2ParticleID                      0.00010475    0.000239099   0.000806963   0.000179077   0.000135799      50     
reco:emtrkmichelid:EmTrackMichelId                      16.5341       27.6708       38.7219       27.0309       5.76101        50     
reco:opdec:Deconvolution                                19.0132       26.093        36.1925       25.5271       4.39717        50     
reco:ophitspe:OpHitFinderDeco                           155.403       176.857       202.227       177.572       10.3679        50     
reco:opflash:OpFlashFinder                              24.6059       53.4449       71.4313       54.0464       11.3468        50     
reco:solarflash:SolarOpFlash                           0.294574       0.58969       1.03119      0.607578      0.163995        50     
reco:rns:RandomNumberSaver                            7.9642e-05    0.000215735   0.00224705    0.000133425   0.000305472      50     
[art]:TriggerResults:TriggerResultInserter            3.2963e-05    6.34969e-05   0.000280286   4.6759e-05    4.35458e-05      50     
end_path:out1:RootOutput                              1.3967e-05    2.48849e-05   7.8901e-05    1.9312e-05    1.46607e-05      50     
end_path:out1:RootOutput(write)                        0.171285      0.283479      0.440833      0.281607      0.0615434       50     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3255.35 MB
  Peak resident set size usage (VmHWM): 1734.43 MB
====================================================================================================
Art has completed and will exit with status 0.
Moving on to metadata extractor 
============OUTPUT FILE:  prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root
Extracted metadata
checking file====== fardet-hd:prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco.root
MD conversion to MetaCat OK
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
old_md.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root.ext.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root.json
prodmarley_nue_cc_flat_radiological_decay0_dune10kt_1x2x6_centralAPA_20250207T123421Z_gen_004459_supernova_g4_detsim_reco_20250724T130426Z_reco2.root.temp.json
reco2_hist.root