Jobsub ID 207247.8@dunegpschedd02.fnal.gov

Jobsub ID	207247.8@dunegpschedd02.fnal.gov
Workflow ID	8090
Stage ID	1
User name	calcuttj@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	8387559424 (7999 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-09-26 14:09:37
Site	IT_CNAF
Entry	CMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat	2025-09-26 17:24:21
From worker node	Hostname	wn-s04-06-01-03.cr.cnaf.infn.it
	cpuinfo	AMD EPYC 7351 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	10485760000 (10000 MiB)
	Wall seconds limit	257400 (71 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Started	2025-09-26 15:08:39
Input files	usertests:prod_beam_p5GeV_protodunehd_1_212254_24_1758820594_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6.root
Jobscript	Exit code	0
	Real time	2h (8015s)
	CPU time	2h (7783s = 97%)
	Max RSS bytes	7511547904 (7163 MiB)
Outputting started	2025-09-26 17:22:14
Output files	usertests:prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.root
Finished	2025-09-26 17:24:21
Saved logs	justin-logs:207247.8-dunegpschedd02.fnal.gov.logs.tgz
List job events Cached HTCondor job logs

Jobscript log (last 10,000 characters)

cooked to 6000 ticks
wclsFrameSaver: saving 77824 traces tagged "gauss"
FrameSaver: q=2.32245e+06 n=1266878 tag=gauss
wclsFrameSaver: saving 97549 traces tagged "wiener"
FrameSaver: q=2.85299e+06 n=1223912 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 92224 traces, 92224 summary
Retagger: tagging trace set: gauss with 74096 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 74096 traces tagged "gauss"
FrameSaver: q=2.32421e+06 n=1181393 tag=gauss
wclsFrameSaver: saving 92224 traces tagged "wiener"
FrameSaver: q=3.03373e+06 n=1141293 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 92138 traces, 92138 summary
Retagger: tagging trace set: gauss with 76326 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 76326 traces tagged "gauss"
FrameSaver: q=2.24872e+06 n=1114478 tag=gauss
wclsFrameSaver: saving 92138 traces tagged "wiener"
FrameSaver: q=3.16614e+06 n=1101730 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 127288 traces, 127288 summary
Retagger: tagging trace set: gauss with 96251 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96251 traces tagged "gauss"
FrameSaver: q=3.22068e+06 n=1555454 tag=gauss
wclsFrameSaver: saving 127288 traces tagged "wiener"
FrameSaver: q=4.07372e+06 n=1455687 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 97806 traces, 97806 summary
Retagger: tagging trace set: gauss with 79157 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 79157 traces tagged "gauss"
FrameSaver: q=2.34438e+06 n=1187267 tag=gauss
wclsFrameSaver: saving 97806 traces tagged "wiener"
FrameSaver: q=3.18884e+06 n=1153371 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 105842 traces, 105842 summary
Retagger: tagging trace set: gauss with 83093 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 83093 traces tagged "gauss"
FrameSaver: q=2.70803e+06 n=1343712 tag=gauss
wclsFrameSaver: saving 105842 traces tagged "wiener"
FrameSaver: q=3.3212e+06 n=1275365 tag=wiener
26-Sep-2025 19:22:12 CEST  Closed output file "prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.root"
26-Sep-2025 19:22:12 CEST  Closed input file "prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            164.113       216.49        281.123       204.378       37.3917        10     
--------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                              0.00120819    0.00892503     0.0230523    0.00766402     0.0059453       10     
reco:opdec:Deconvolution                             0.968414       1.17618       1.42909       1.18363      0.141547        10     
reco:ophitspe:OpHitFinderDeco                        0.180204      0.233322      0.329752      0.211741      0.0516852       10     
reco:opflash:OpFlashFinder                           0.0875184     0.121868      0.169158      0.121317      0.0283369       10     
reco:opslicer:OpSlicer                               0.213598      0.317707       0.46428      0.316261      0.0725531       10     
reco:wclsdatahd:WireCellToolkit                       47.0832       55.6432       65.7893       54.7003       5.61024        10     
reco:gaushit:GausHitFinder                           0.833525       1.1587        1.7276        1.13773      0.255247        10     
reco:nhitsfilter:NumberOfHitsFilter                 0.000465641   0.000673194   0.000822852   0.000679282   0.000110812      10     
reco:reco3d:SpacePointSolver                          0.80446       1.2802        2.11379       1.05771      0.433776        10     
reco:hitpdune:DisambigFromSpacePoints                0.660042      0.984154       1.26062      0.993821      0.181774        10     
reco:pandora:StandardPandora                          18.7502       36.3851       98.4806       30.2801       21.4775        10     
reco:pandoraWriter:StandardPandora                   0.288814      0.382107      0.493882      0.374482      0.0675578       10     
reco:pandoraTrack:LArPandoraTrackCreation             1.92749       3.68304       5.26031       3.66655       1.09488        10     
reco:pandoraShower:LArPandoraShowerCreation           2.51254       4.04168       7.15765       3.81058       1.36964        10     
reco:pandoracalo:Calorimetry                          0.99763       1.62249       2.40982       1.51234      0.462942        10     
reco:pandoracalonosce:Calorimetry                    0.932361       1.57135       2.47598       1.48404      0.550343        10     
reco:pandoraShowercalo:ShowerCalorimetry              1.72986       3.4838        6.66648       2.85871       1.66734        10     
reco:pandoraShowercalonosce:ShowerCalorimetry         1.54721       3.25036       6.16806       2.7576        1.30127        10     
reco:pandoraGnocchiCalo:GnocchiCalorimetry           0.0559483     0.0897364     0.134026      0.090209      0.023896        10     
reco:pandoraGnocchiCalonosce:GnocchiCalorimetry      0.0517696     0.0880486     0.132137      0.0901289     0.0244688       10     
reco:emtrkmichelid:EmTrackMichelId                    49.6903       67.6452       96.5155       67.2775       12.0766        10     
reco:crttag:SingleCRTMatchingProducer                0.0062457     0.0111118     0.0163701     0.0106207    0.00336093       10     
reco:crtreco:TwoCRTMatchingProducer                  0.0063036    0.00958865     0.0147071    0.00843584    0.00295322       10     
reco:pandora2Track:LArPandoraTrackCreation            5.78182       10.6759       18.9783       8.11379       4.38785        10     
reco:pandora2calo:ShowerCalorimetry                   1.30907       3.41865       7.1714        2.45406       1.98396        10     
reco:pandora2calonosce:ShowerCalorimetry              1.4086        3.09282       5.76625       2.43622       1.50996        10     
reco:pandora2GnocchiCalo:GnocchiCalorimetry          0.0501281     0.090534      0.129035      0.0886639     0.0243973       10     
reco:pandora2GnocchiCalonosce:GnocchiCalorimetry     0.0504469     0.0887155     0.120764      0.0898291     0.0227629       10     
reco:pandora2Shower:LArPandoraShowerCreation          2.72586       4.62078       9.06598       3.73218       1.88065        10     
reco:pandora2Showercalo:ShowerCalorimetry             2.00825       3.98795       9.45566       3.45587       2.07739        10     
reco:pandora2Showercalonosce:ShowerCalorimetry        1.93426       3.98879       8.69318       3.23367       2.10597        10     
[art]:TriggerResults:TriggerResultInserter           2.788e-05    5.0714e-05     0.0001075    4.0595e-05    2.49685e-05      10     
end_path:out1:RootOutput                             7.56e-06     1.2931e-05     3.282e-05     9.95e-06     7.31707e-06      10     
end_path:out1:RootOutput(write)                       2.43807       3.17232       4.94772       2.88384      0.706703        10     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6351.2 MB
  Peak resident set size usage (VmHWM): 4147.42 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.root', 'start_time': 1758905036.3959904, 'end_time': 1758907334.10322}
lsing
total 13G
-rw-r--r-- 1 neutrino002 neutrino 6.4G Sep 26 19:22 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.root
-rw-r--r-- 1 neutrino002 neutrino 6.1G Sep 26 18:43 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim.root
-rw-r--r-- 1 neutrino002 neutrino  56M Sep 26 19:19 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino 196K Sep 26 19:22 mem.db
-rw-r--r-- 1 neutrino002 neutrino 178K Sep 26 19:22 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino  36K Sep 26 19:22 time.db
-rw-r--r-- 1 neutrino002 neutrino  13K Sep 26 19:22 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.err
-rw-r--r-- 1 neutrino002 neutrino  12K Sep 26 18:43 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim.log
-rw-r--r-- 1 neutrino002 neutrino 9.6K Sep 26 19:22 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim_reco1.log
-rw-r--r-- 1 neutrino002 neutrino 4.4K Sep 26 18:43 prod_beam_p5GeV_protodunehd_1_207247_8_1758899329_detsim.err
-rw-r--r-- 1 neutrino002 neutrino 3.5K Sep 26 19:22 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino  519 Sep 26 18:43 detsim_single_protodunehd_hist.root
-rw-r--r-- 1 neutrino002 neutrino  519 Sep 26 19:22 reco_protodunehd_hist.root
-rw-r--r-- 1 neutrino002 neutrino  110 Sep 26 17:08 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino    0 Sep 26 18:47 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino    0 Sep 26 18:44 debugprod.log
disk usage
13G	.