justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233332.1@dunegpschedd02.fnal.gov

Jobsub ID233332.1@dunegpschedd02.fnal.gov
Workflow ID9392
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 21:19:21
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-10-28 01:02:19
From worker nodeHostnamewn-db-02.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 21:20:10
Input filesmonte-carlo-009392-000001
JobscriptExit code0
Real time3h (13241s)
CPU time3h (13085s = 98%)
Max RSS bytes4193566720 (3999 MiB)
Outputting started2025-10-28 01:00:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09392/1/001/reco_stage1.root
Finished2025-10-28 01:02:19
Saved logsjustin-logs:233332.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122646 traces, 0 summary
Retagger: tagging trace set: gauss with 94891 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94891 traces tagged "gauss"
FrameSaver: q=3.26823e+06 n=1690152 tag=gauss
wclsFrameSaver: saving 122646 traces tagged "wiener"
FrameSaver: q=3.44298e+06 n=1504970 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 42229 XUs and 26259 XVs -> 289 XUVs
C:0 T:1 85414 XUs and 61300 XVs -> 5485 XUVs
C:0 T:2 87615 XUs and 150583 XVs -> 6637 XUVs
C:0 T:3 44598 XUs and 57877 XVs -> 723 XUVs
C:0 T:4 6746 XUs and 10742 XVs -> 151 XUVs
C:0 T:5 29608 XUs and 28804 XVs -> 2781 XUVs
C:0 T:6 106095 XUs and 118331 XVs -> 4471 XUVs
C:0 T:7 43830 XUs and 47019 XVs -> 445 XUVs
C:0 T:8 9667 XUs and 12006 XVs -> 225 XUVs
C:0 T:9 60620 XUs and 87280 XVs -> 6470 XUVs
C:0 T:10 10161 XUs and 7499 XVs -> 1390 XUVs
C:0 T:11 350 XUs and 79 XVs -> 57 XUVs
29124 XUVs total
10750 collection wire objects
25966 potential space points
Neighbour search...
1542706 tests to find 802324 neighbours
Iterating with no regularization...
Begin: 7.52684e+07
0 6.05224e+07
1 5.97324e+07
2 5.96533e+07
3 5.96321e+07
Now with regularization...
Begin: 5.10858e+07
0 5.10373e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410154375
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410154375
fLifetime: 30000 [us]
28-Oct-2025 02:00:50 CET  Closed output file "reco_stage1.root"
28-Oct-2025 02:00:50 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          327.099       414.527       504.727       399.834       57.4259        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000879755   0.00516982    0.00834029    0.00559533    0.00255979       10     
reco:ophit:OpHitFinder                             0.264109      0.294603      0.346953      0.287575      0.0253728       10     
reco:opflash:OpFlashFinder                         0.164641      0.188645      0.240605      0.183384      0.0224527       10     
reco:opslicer:OpSlicer                             0.592199      0.713621       0.8233        0.72185      0.0893444       10     
reco:caldata:DataPrepModule                         5.05251       5.19466       5.54666       5.15451      0.133278        10     
reco:wclsdatasp:WireCellToolkit                     37.1975       55.9231       74.0158       59.2368       13.149         10     
reco:gaushit:GausHitFinder                          1.02665       1.29468       1.60239       1.26766       0.16773        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000597979   0.000754077   0.000986127   0.000712161   0.000112886      10     
reco:reco3d:SpacePointSolver                        12.0961       16.2423       26.0291       15.2664       3.92819        10     
reco:hitpdune:DisambigFromSpacePoints               1.10799       1.61369       2.2668        1.54624      0.336197        10     
reco:pandora:StandardPandora                        20.2158       34.1284       56.0511       28.8999       10.6747        10     
reco:pandoraWriter:StandardPandora                 0.348062      0.442938      0.555637      0.441394      0.0629646       10     
reco:pandoraTrack:LArPandoraTrackCreation           3.23825       4.60436       6.21554       4.55498      0.955736        10     
reco:pandoraShower:LArPandoraShowerCreation         3.28719       5.5105        8.55955       5.02164       1.70471        10     
reco:pandoracalo:Calorimetry                        1.71187       2.52722       3.27193       2.4935        0.50383        10     
reco:pandoracalonosce:Calorimetry                   1.52303       2.34493       3.03409       2.3012       0.499628        10     
reco:pandorapid:Chi2ParticleID                    0.00489425    0.00601502    0.00759988    0.00606899    0.000761656      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0443321     0.0537157     0.0707747     0.0507587    0.00766959       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00490684    0.00576637     0.0068341    0.00580544    0.000598397      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.98804       5.45319       7.92216       5.01263       1.59766        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.76005       5.17981       7.83894       4.62986       1.57291        10     
reco:emtrkmichelid:EmTrackMichelId                  184.493       221.395       267.731       218.018       27.4611        10     
reco:crttag:SingleCRTMatchingProducer             0.00901361     0.0132995     0.0214602     0.0122228    0.00333051       10     
reco:crtreco:TwoCRTMatchingProducer               0.00775807     0.0107926     0.013674      0.0103793    0.00165542       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.743163      0.906578       1.04765      0.906473      0.096289        10     
reco:pandora2Track:LArPandoraTrackCreation          9.39936       15.5863       22.9297       14.5933       4.39825        10     
reco:pandora2calo:Calorimetry                       4.28881       7.34836       10.5665       6.7395        1.96059        10     
reco:pandora2calonosce:Calorimetry                  4.18789       6.89504       9.20029       6.1636        1.80861        10     
reco:pandora2pid:Chi2ParticleID                    0.011829      0.0149384     0.0187711      0.01457     0.00224508       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0546284     0.0683725     0.0845011     0.0662969      0.0103         10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0118053     0.0150004     0.0189367     0.0145211    0.00226525       10     
reco:pandora2Shower:LArPandoraShowerCreation        3.71387       5.58481       7.65057       5.20205       1.45063        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.79661       6.22597       8.44441       5.52201       1.56546        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      3.35892       5.4449        7.51836       4.98573       1.26579        10     
[art]:TriggerResults:TriggerResultInserter        3.2682e-05    4.21362e-05   0.000106132   3.5513e-05    2.13839e-05      10     
end_path:out1:RootOutput                           6.382e-06    7.4221e-06    1.1833e-05     6.898e-06    1.53379e-06      10     
end_path:out1:RootOutput(write)                     2.62849       3.14513       3.76463       3.01179      0.402455        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4996.94 MB
  Peak resident set size usage (VmHWM): 4193.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.26 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10127300
-rw-r--r--. 1 dune009 dune 2375990731 Oct 28 00:49 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2253114540 Oct 28 00:47 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2071562889 Oct 28 02:00 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1843318405 Oct 27 22:39 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1748910551 Oct 27 22:26 g4_stage1.root
-rw-r--r--. 1 dune009 dune   76846900 Oct 28 01:56 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242876 Oct 28 02:00 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 28 02:00 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 28 02:00 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 28 02:00 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 28 00:49 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 27 22:39 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 28 02:00 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 27 22:20 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 28 02:00 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 28 00:56 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 28 00:49 debugprod.log
justIN time: 2025-11-05 19:15:34 UTC       justIN version: 01.05.01