justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 40546.0@dunegpschedd02.fnal.gov

Jobsub ID40546.0@dunegpschedd02.fnal.gov
Workflow ID2568
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-09-16 17:44:10
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-09-16 20:44:02
From worker nodeHostnamewn-lb-14.gina.surf.nl
cpuinfoAMD EPYC 9754 128-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-09-16 17:45:36
Input filesusertests:prod_beam_p5GeV_protodunehd_1_39241_22_1757750243_pdint_TPC1_pdint_TPC2_pdint_TPC5_pdint_TPC6.root
JobscriptExit code0
Real time2h (10554s)
CPU time1h (7013s = 66%)
Max RSS bytes6659997696 (6351 MiB)
Outputting started2025-09-16 20:41:31
Output filesusertests:prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02568/1/001/prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02568/1/001/prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02568/1/001/prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/PDHD_MC_Prod_Test_stage2/fnal/02568/1/001/prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2_tfile.root
Finished2025-09-16 20:44:02
Saved logsjustin-logs:40546.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

8 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 77388 traces tagged "gauss"
FrameSaver: q=2.32565e+06 n=1083011 tag=gauss
wclsFrameSaver: saving 93494 traces tagged "wiener"
FrameSaver: q=2.52647e+06 n=1074397 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 99098 traces, 99098 summary
Retagger: tagging trace set: gauss with 83839 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 83839 traces tagged "gauss"
FrameSaver: q=2.37675e+06 n=1286371 tag=gauss
wclsFrameSaver: saving 99098 traces tagged "wiener"
FrameSaver: q=2.48901e+06 n=1246348 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 81646 traces, 81646 summary
Retagger: tagging trace set: gauss with 69231 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 69231 traces tagged "gauss"
FrameSaver: q=1.75748e+06 n=925088 tag=gauss
wclsFrameSaver: saving 81646 traces tagged "wiener"
FrameSaver: q=1.97007e+06 n=919506 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 110598 traces, 110598 summary
Retagger: tagging trace set: gauss with 93652 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 93652 traces tagged "gauss"
FrameSaver: q=2.63664e+06 n=1378987 tag=gauss
wclsFrameSaver: saving 110598 traces tagged "wiener"
FrameSaver: q=2.832e+06 n=1350801 tag=wiener
16-Sep-2025 22:41:29 CEST  Closed output file "prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.root"
16-Sep-2025 22:41:29 CEST  Closed input file "prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1.root"

======================================================================================================================================
TimeTracker printout (sec)                              Min           Avg           Max         Median          RMS         nEvts   
======================================================================================================================================
Full event                                            118.843       173.875       344.077       153.958       61.6842        10     
--------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                              0.00113811    0.00957282     0.0173243    0.00947506    0.00463917       10     
reco:wclsdatahd:WireCellToolkit                       35.7513       48.3916       58.2046       50.2665       7.2004         10     
reco:gaushit:GausHitFinder                            0.59962      0.774238      0.996532      0.761676      0.113343        10     
reco:nhitsfilter:NumberOfHitsFilter                 0.000582074   0.00157688    0.00310306    0.00124574    0.000922188      10     
reco:reco3d:SpacePointSolver                         0.393289      0.951788       1.80354      0.894388      0.446088        10     
reco:hitpdune:DisambigFromSpacePoints                0.343839      0.681296       1.13362      0.624522      0.255687        10     
reco:pandora:StandardPandora                          10.7951       30.4217       125.062       20.0436       31.8735        10     
reco:pandoraWriter:StandardPandora                   0.217865      0.302157      0.432588      0.266316      0.0697521       10     
reco:pandoraTrack:LArPandoraTrackCreation             1.40308       2.92426       5.46722       2.72576       1.38628        10     
reco:pandoraShower:LArPandoraShowerCreation           1.44071       3.60435       7.00531       3.01214       1.80644        10     
reco:pandoracalo:Calorimetry                         0.712102       1.32107       2.12738       1.18171      0.522614        10     
reco:pandoracalonosce:Calorimetry                    0.688549       1.31234       2.17567       1.20156      0.520573        10     
reco:pandoraShowercalo:ShowerCalorimetry              1.40292       3.1093        5.21332       2.56426       1.36816        10     
reco:pandoraShowercalonosce:ShowerCalorimetry         1.33493       2.81086       4.86752       2.2139        1.3239         10     
reco:pandoraGnocchiCalo:GnocchiCalorimetry           0.0398682     0.0545896     0.0747279     0.0535813     0.0105578       10     
reco:pandoraGnocchiCalonosce:GnocchiCalorimetry      0.0403509     0.0547948     0.0751259     0.0539251     0.0109869       10     
reco:emtrkmichelid:EmTrackMichelId                    17.6414       22.5077       33.2131       21.9825       4.21254        10     
reco:opdec:Deconvolution                             0.614787      0.858626       1.19475      0.826433      0.147651        10     
reco:ophitspe:OpHitFinderDeco                        0.134451      0.157127      0.202583      0.149375      0.0219738       10     
reco:opflash:OpFlashFinder                           0.0461698     0.101086      0.196326      0.0864056     0.0386642       10     
reco:opslicer:OpSlicer                               0.176447      0.244808      0.315271      0.250867      0.0373099       10     
reco:crtreco:TwoCRTMatchingProducer                 0.00669029     0.0112248     0.0231238    0.00827958    0.00503952       10     
reco:crttag:SingleCRTMatchingProducer               0.00460281    0.00837279     0.0132898    0.00812674    0.00283364       10     
reco:pandora2Track:LArPandoraTrackCreation            3.9804        9.11777       17.0008       7.62956       3.86409        10     
reco:pandora2calo:ShowerCalorimetry                   1.21316       2.70852       5.85054       2.14495       1.36571        10     
reco:pandora2calonosce:ShowerCalorimetry              1.27649       2.56021       5.66324       2.20287       1.25954        10     
reco:pandora2GnocchiCalo:GnocchiCalorimetry          0.0393835     0.053325      0.0674474     0.0524173    0.00974888       10     
reco:pandora2GnocchiCalonosce:GnocchiCalorimetry     0.0409757     0.053893      0.0688311     0.0525767    0.00962416       10     
reco:pandora2Shower:LArPandoraShowerCreation          1.57479       3.64889       6.6711        2.96792       1.61448        10     
reco:pandora2Showercalo:ShowerCalorimetry             1.39987       3.20713       7.09008       2.56267       1.69343        10     
reco:pandora2Showercalonosce:ShowerCalorimetry        1.40869       2.95182       6.68964       2.26988       1.5323         10     
[art]:TriggerResults:TriggerResultInserter           2.684e-05    4.43156e-05   0.000101673   3.78265e-05   2.03233e-05      10     
end_path:out1:RootOutput                             5.478e-06    1.03637e-05   2.7682e-05    6.3395e-06    8.43063e-06      10     
end_path:michelremoving:michelremoving               0.0272274     0.216867      0.655415       0.18018      0.183842        10     
end_path:michelremovinggnocchi:michelremoving        0.0259579     0.210058      0.649464       0.17503      0.182125        10     
end_path:calibanagnocchi:CalibAnaTree                 15.5741       26.2422       49.7566       24.6898       9.10179        10     
end_path:out1:RootOutput(write)                       1.55409       2.05537       3.05324       1.98897      0.382399        10     
======================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7787.74 MB
  Peak resident set size usage (VmHWM): 5102.09 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.root', 'tfile_out': 'prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2_tfile.root', 'start_time': 1758053460.6454103, 'end_time': 1758055290.9981036}
lsing
total 16G
-rw-r--r--. 1 dune003 dune 5.8G Sep 16 22:10 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1.root
-rw-r--r--. 1 dune003 dune 5.2G Sep 16 22:07 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim.root
-rw-r--r--. 1 dune003 dune 5.0G Sep 16 22:41 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.root
-rw-r--r--. 1 dune003 dune  53M Sep 16 22:41 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2_tfile.root
-rw-r--r--. 1 dune003 dune  51M Sep 16 22:38 Pandora_Events.pndr
-rw-r--r--. 1 dune003 dune 360K Sep 16 22:41 jobscript.log
-rw-r--r--. 1 dune003 dune 220K Sep 16 22:41 mem.db
-rw-r--r--. 1 dune003 dune 174K Sep 16 22:41 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.log
-rw-r--r--. 1 dune003 dune  40K Sep 16 22:41 time.db
-rw-r--r--. 1 dune003 dune  13K Sep 16 22:41 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1_reco2.err
-rw-r--r--. 1 dune003 dune  13K Sep 16 22:07 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim.log
-rw-r--r--. 1 dune003 dune 6.8K Sep 16 22:11 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1.err
-rw-r--r--. 1 dune003 dune 4.5K Sep 16 22:07 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim.err
-rw-r--r--. 1 dune003 dune 3.5K Sep 16 22:41 Pandora_Geometry.xml
-rw-r--r--. 1 dune003 dune 1.4K Sep 16 22:11 prod_beam_p5GeV_protodunehd_1_40546_0_1758044749_detsim_reco1.log
-rw-r--r--. 1 dune003 dune  519 Sep 16 22:07 detsim_single_protodunehd_hist.root
-rw-r--r--. 1 dune003 dune  519 Sep 16 22:10 reco_protodunehd_hist.root
-rw-r--r--. 1 dune003 dune  109 Sep 16 19:45 all-input-dids.txt
-rw-r--r--. 1 dune003 dune    0 Sep 16 22:13 ProtoDUNE.log
-rw-r--r--. 1 dune003 dune    0 Sep 16 22:11 debugprod.log
disk usage
16G	.
justIN time: 2025-09-19 14:58:50 UTC       justIN version: 01.05.00