justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 255112.1@dunegpschedd01.fnal.gov

Jobsub ID255112.1@dunegpschedd01.fnal.gov
Workflow ID10492
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-21 21:05:48
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc02
Last heartbeat2025-11-22 00:25:59
From worker nodeHostnamewn-da-02.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-21 21:06:35
Input filesmonte-carlo-010492-000001
JobscriptExit code0
Real time3h (11882s)
CPU time3h (11744s = 98%)
Max RSS bytes4124778496 (3933 MiB)
Outputting started2025-11-22 00:24:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10492/1/001/reco_stage1.root
Finished2025-11-22 00:25:59
Saved logsjustin-logs:255112.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ing tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 121185 traces, 0 summary
Retagger: tagging trace set: gauss with 90454 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90454 traces tagged "gauss"
FrameSaver: q=3.32644e+06 n=1572906 tag=gauss
wclsFrameSaver: saving 121185 traces tagged "wiener"
FrameSaver: q=3.50957e+06 n=1382272 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38708 XUs and 24958 XVs -> 396 XUVs
C:0 T:1 58178 XUs and 48001 XVs -> 3922 XUVs
C:0 T:2 42943 XUs and 66736 XVs -> 2453 XUVs
C:0 T:3 23939 XUs and 28414 XVs -> 108 XUVs
C:0 T:4 11925 XUs and 12150 XVs -> 95 XUVs
C:0 T:5 42730 XUs and 37830 XVs -> 3913 XUVs
C:0 T:6 76993 XUs and 82801 XVs -> 3642 XUVs
C:0 T:7 35520 XUs and 33390 XVs -> 462 XUVs
C:0 T:8 7551 XUs and 8196 XVs -> 77 XUVs
C:0 T:9 56838 XUs and 80067 XVs -> 8532 XUVs
C:0 T:10 26606 XUs and 18178 XVs -> 3703 XUVs
C:0 T:11 56 XUs and 34 XVs -> 8 XUVs
27311 XUVs total
9109 collection wire objects
25165 potential space points
Neighbour search...
1708749 tests to find 877862 neighbours
Iterating with no regularization...
Begin: 7.14837e+07
0 5.86641e+07
1 5.77485e+07
2 5.76166e+07
3 5.75796e+07
Now with regularization...
Begin: 4.70081e+07
0 4.69776e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410657092
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410657092
fLifetime: 30000 [us]
22-Nov-2025 01:24:36 CET  Closed output file "reco_stage1.root"
22-Nov-2025 01:24:36 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          262.629       359.506       461.712       356.466       63.4002        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000973226   0.00409869    0.00824789    0.00330874    0.00253749       10     
reco:ophit:OpHitFinder                             0.234831      0.263724      0.315989      0.266995      0.0260676       10     
reco:opflash:OpFlashFinder                         0.109925      0.159966      0.220223      0.161119      0.0291937       10     
reco:opslicer:OpSlicer                             0.521021       0.65489      0.849234      0.642828      0.0863918       10     
reco:caldata:DataPrepModule                         4.5811        4.82398       5.20655       4.78921      0.170186        10     
reco:wclsdatasp:WireCellToolkit                     35.9153       51.7917       64.1389       54.4834       10.7935        10     
reco:gaushit:GausHitFinder                         0.909748       1.17962       1.53651       1.1418       0.209084        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000489063   0.000708912   0.000893344   0.000733234   0.000110415      10     
reco:reco3d:SpacePointSolver                        6.89465       10.7888       17.8039       9.91373       3.19233        10     
reco:hitpdune:DisambigFromSpacePoints              0.956885       1.38964       1.95992       1.24375      0.351246        10     
reco:pandora:StandardPandora                        16.3545       30.4153       42.7463       30.7968       6.85909        10     
reco:pandoraWriter:StandardPandora                 0.290407      0.394627      0.516645       0.37822      0.0666414       10     
reco:pandoraTrack:LArPandoraTrackCreation           2.46279       3.83562       5.17529       3.83796      0.986131        10     
reco:pandoraShower:LArPandoraShowerCreation         2.28793       4.38578       6.7048        3.8993        1.47064        10     
reco:pandoracalo:Calorimetry                        1.35834       2.1055        2.90365       2.08668       0.55316        10     
reco:pandoracalonosce:Calorimetry                   1.23114       1.94424       2.73736       1.93192      0.529017        10     
reco:pandorapid:Chi2ParticleID                    0.00465864    0.00546313    0.00629548    0.00551882    0.000530629      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.032991      0.0459215     0.0570565     0.0441274    0.00773755       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00429063    0.00512482    0.00596478     0.0051762    0.000524466      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.18607       4.40267       6.78788       4.02394       1.47225        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       2.03324       4.25257       6.75221       3.88558       1.47779        10     
reco:emtrkmichelid:EmTrackMichelId                  155.796       194.935       243.129       190.968       27.1753        10     
reco:crttag:SingleCRTMatchingProducer             0.00778438     0.010455      0.0146628     0.0103662    0.00187319       10     
reco:crtreco:TwoCRTMatchingProducer               0.00654688    0.00857398     0.0103273    0.00905156    0.00127465       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.654193      0.785875      0.962649      0.783713      0.100497        10     
reco:pandora2Track:LArPandoraTrackCreation          6.9097        12.3904       19.4012       12.0252       3.72083        10     
reco:pandora2calo:Calorimetry                       3.19259       5.93138       9.48709       6.04502       1.90389        10     
reco:pandora2calonosce:Calorimetry                  3.02063       5.59772       8.98637       5.64322       1.82941        10     
reco:pandora2pid:Chi2ParticleID                    0.0103736     0.013131      0.0153192     0.0131101    0.00158065       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0424432     0.0585659     0.073477      0.0564442     0.0102024       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0106453     0.0131499     0.0155057     0.013056     0.00152866       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.66192       4.60601       7.05211       4.45944       1.44099        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.78839       5.0072         7.705        5.16378       1.54031        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.55734       4.45221       6.61633       4.62739       1.30367        10     
[art]:TriggerResults:TriggerResultInserter        2.8475e-05    4.20573e-05   0.000113066   3.23265e-05   2.48277e-05      10     
end_path:out1:RootOutput                           6.302e-06     7.303e-06    1.1803e-05    6.8085e-06    1.53987e-06      10     
end_path:out1:RootOutput(write)                     2.18458       2.72013       3.45844       2.62792      0.412557        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4927.41 MB
  Peak resident set size usage (VmHWM): 4124.78 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.98 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9767852
-rw-r--r--. 1 dune009 dune 2308720045 Nov 22 00:22 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2192222698 Nov 22 00:20 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 1987575126 Nov 22 01:24 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1753318771 Nov 21 22:24 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1686704728 Nov 21 22:12 g4_stage1.root
-rw-r--r--. 1 dune009 dune   73122190 Nov 22 01:20 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     242904 Nov 22 01:24 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Nov 22 01:24 mem.db
-rw-r--r--. 1 dune009 dune      62376 Nov 22 01:24 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Nov 22 01:24 time.db
-rw-r--r--. 1 dune009 dune        519 Nov 22 00:22 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 21 22:24 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Nov 22 01:24 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Nov 21 22:06 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Nov 22 01:24 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:27 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Nov 22 00:22 debugprod.log
justIN time: 2026-02-08 22:56:46 UTC       justIN version: 01.06.00