justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 47663.2@dunegpschedd01.fnal.gov

Jobsub ID47663.2@dunegpschedd01.fnal.gov
Workflow ID2675
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-09-17 17:17:34
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw7
Last heartbeat2025-09-17 20:39:51
From worker nodeHostnamemh-7763-12.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-17 17:18:18
Input filesusertests:prod_beam_p5GeV_protodunehd_1_39008_9_1757695965_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6.root
JobscriptExit code0
Real time3h (11858s)
CPU time1h (6674s = 56%)
Max RSS bytes7323222016 (6983 MiB)
Outputting started2025-09-17 20:35:56
Output filesusertests:prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02675/1/001/prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02675/1/001/prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.log
Finished2025-09-17 20:39:51
Saved logsjustin-logs:47663.2-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

is
Retagger: tagging trace set: wiener with 81893 traces, 81893 summary
Retagger: tagging trace set: gauss with 65965 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 65965 traces tagged "gauss"
FrameSaver: q=1.75787e+06 n=969359 tag=gauss
wclsFrameSaver: saving 81893 traces tagged "wiener"
FrameSaver: q=2.33266e+06 n=963076 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 95526 traces, 95526 summary
Retagger: tagging trace set: gauss with 75067 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 75067 traces tagged "gauss"
FrameSaver: q=2.20414e+06 n=1174278 tag=gauss
wclsFrameSaver: saving 95526 traces tagged "wiener"
FrameSaver: q=2.76558e+06 n=1135121 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 95280 traces, 95280 summary
Retagger: tagging trace set: gauss with 76838 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 76838 traces tagged "gauss"
FrameSaver: q=2.26884e+06 n=1184887 tag=gauss
wclsFrameSaver: saving 95280 traces tagged "wiener"
FrameSaver: q=2.95343e+06 n=1138428 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 91016 traces, 91016 summary
Retagger: tagging trace set: gauss with 72611 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 72611 traces tagged "gauss"
FrameSaver: q=2.0553e+06 n=1083833 tag=gauss
wclsFrameSaver: saving 91016 traces tagged "wiener"
FrameSaver: q=2.78313e+06 n=1053741 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 80002 traces, 80002 summary
Retagger: tagging trace set: gauss with 64620 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 64620 traces tagged "gauss"
FrameSaver: q=1.78517e+06 n=985580 tag=gauss
wclsFrameSaver: saving 80002 traces tagged "wiener"
FrameSaver: q=2.18081e+06 n=961833 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 120042 traces, 120042 summary
Retagger: tagging trace set: gauss with 95125 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95125 traces tagged "gauss"
FrameSaver: q=3.15629e+06 n=1578062 tag=gauss
wclsFrameSaver: saving 120042 traces tagged "wiener"
FrameSaver: q=4.03144e+06 n=1507721 tag=wiener
17-Sep-2025 13:35:50 PDT  Closed output file "prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.root"
17-Sep-2025 13:35:50 PDT  Closed input file "prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            94.2082       145.969       219.538       145.927       33.3612        10     
--------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                              0.00125534    0.00568938    0.00970172    0.00631221    0.00286467       10     
reco:opdec:Deconvolution                             0.675266      0.947311       1.37077      0.933416      0.203035        10     
reco:ophitspe:OpHitFinderDeco                        0.163056      0.251005      0.515865      0.233377      0.093683        10     
reco:opflash:OpFlashFinder                           0.0400996     0.0952589     0.162489      0.0962972     0.0342337       10     
reco:opslicer:OpSlicer                                0.16712      0.282779      0.344664      0.295577      0.0595996       10     
reco:wclsdatahd:WireCellToolkit                       49.856        51.4206       53.1006       51.1834       1.08312        10     
reco:gaushit:GausHitFinder                           0.399834      0.993012       1.82513      0.960314      0.362233        10     
reco:nhitsfilter:NumberOfHitsFilter                 0.000230147   0.000447668   0.000795561   0.000417695   0.000141326      10     
reco:reco3d:SpacePointSolver                         0.598024       1.17266       2.10991       1.31224      0.460591        10     
reco:hitpdune:DisambigFromSpacePoints                0.195864      0.892167       1.97021      0.804283      0.490845        10     
reco:pandora:StandardPandora                          9.47729       24.8629       51.1093       21.9963       11.7293        10     
reco:pandoraWriter:StandardPandora                   0.161886      0.306216      0.491878      0.322796      0.0842126       10     
reco:pandoraTrack:LArPandoraTrackCreation            0.592601       1.96201       3.69494       1.96663      0.801474        10     
reco:pandoraShower:LArPandoraShowerCreation          0.431733       1.95497       3.93016       2.05593      0.923655        10     
reco:pandoracalo:Calorimetry                         0.242966       1.06508       2.12062       1.06155      0.472454        10     
reco:pandoracalonosce:Calorimetry                    0.234877       1.04439       1.74553       1.07783      0.388524        10     
reco:pandoraShowercalo:ShowerCalorimetry              0.2971        1.91681       4.05439       1.83859      0.985084        10     
reco:pandoraShowercalonosce:ShowerCalorimetry         0.36549       1.82124       3.80714       1.6954       0.936534        10     
reco:pandoraGnocchiCalo:GnocchiCalorimetry           0.0363343     0.0616174     0.0920669     0.0577615     0.0185897       10     
reco:pandoraGnocchiCalonosce:GnocchiCalorimetry      0.0339752     0.0632981     0.0941369     0.0639861     0.0186968       10     
reco:emtrkmichelid:EmTrackMichelId                    22.4153       36.2053       51.4207       36.0973       7.60871        10     
reco:crttag:SingleCRTMatchingProducer               0.00429762    0.00828108     0.0122081    0.00812771    0.00227439       10     
reco:crtreco:TwoCRTMatchingProducer                  0.0035794    0.00587591    0.00763445    0.00625082    0.00144131       10     
reco:pandora2Track:LArPandoraTrackCreation            1.56964       5.42067       10.9175       5.64408       2.49907        10     
reco:pandora2calo:ShowerCalorimetry                  0.472357       1.75171       3.55566       1.86124      0.812229        10     
reco:pandora2calonosce:ShowerCalorimetry             0.398705       1.66382       3.76249       1.67438      0.872655        10     
reco:pandora2GnocchiCalo:GnocchiCalorimetry          0.0401247     0.0618819     0.101353      0.0541062     0.0172951       10     
reco:pandora2GnocchiCalonosce:GnocchiCalorimetry     0.0468368     0.0589177     0.0812904     0.0539779     0.0116243       10     
reco:pandora2Shower:LArPandoraShowerCreation         0.609971       2.25252       4.23023       2.23469      0.989328        10     
reco:pandora2Showercalo:ShowerCalorimetry            0.441521       2.39632       4.88699       2.40347       1.18723        10     
reco:pandora2Showercalonosce:ShowerCalorimetry       0.412392       2.28537       4.84887       2.26522       1.13253        10     
[art]:TriggerResults:TriggerResultInserter           2.663e-05    4.10845e-05   0.000105569   3.22245e-05   2.26713e-05      10     
end_path:out1:RootOutput                             7.37e-06     1.1678e-05     3.226e-05     9.42e-06     6.97678e-06      10     
end_path:out1:RootOutput(write)                       1.94329       2.61679       3.39937       2.69836      0.450386        10     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6408.91 MB
  Peak resident set size usage (VmHWM): 4120.78 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.root', 'start_time': 1758139771.2480137, 'end_time': 1758141351.6612628}
lsing
total 12G
-rw-r--r-- 1 cuser cuser 5.9G Sep 17 13:35 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.root
-rw-r--r-- 1 cuser cuser 5.6G Sep 17 13:09 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim.root
-rw-r--r-- 1 cuser cuser  50M Sep 17 13:34 Pandora_Events.pndr
-rw-r--r-- 1 cuser cuser 196K Sep 17 13:35 mem.db
-rw-r--r-- 1 cuser cuser 186K Sep 17 13:35 jobscript.log
-rw-r--r-- 1 cuser cuser  36K Sep 17 13:35 time.db
-rw-r--r-- 1 cuser cuser  13K Sep 17 13:35 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.err
-rw-r--r-- 1 cuser cuser  13K Sep 17 13:09 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim.log
-rw-r--r-- 1 cuser cuser 9.1K Sep 17 13:35 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim_reco1.log
-rw-r--r-- 1 cuser cuser 4.5K Sep 17 13:09 prod_beam_p5GeV_protodunehd_1_47663_2_1758129548_detsim.err
-rw-r--r-- 1 cuser cuser 3.5K Sep 17 13:35 Pandora_Geometry.xml
-rw-r--r-- 1 cuser cuser  519 Sep 17 13:09 detsim_single_protodunehd_hist.root
-rw-r--r-- 1 cuser cuser  519 Sep 17 13:35 reco_protodunehd_hist.root
-rw-r--r-- 1 cuser cuser  108 Sep 17 10:18 all-input-dids.txt
-rw-r--r-- 1 cuser cuser    0 Sep 17 13:12 ProtoDUNE.log
-rw-r--r-- 1 cuser cuser    0 Sep 17 13:09 debugprod.log
disk usage
12G	.
justIN time: 2025-09-19 11:35:22 UTC       justIN version: 01.05.00