justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 234883.1@dunegpschedd02.fnal.gov

Jobsub ID234883.1@dunegpschedd02.fnal.gov
Workflow ID9448
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-29 17:10:41
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc01
Last heartbeat2025-10-29 21:34:27
From worker nodeHostnamewn-db-09.gina.surf.nl
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit129600 (36 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-29 17:12:08
Input filesmonte-carlo-009448-000001
JobscriptExit code0
Real time4h (15632s)
CPU time4h (15462s = 98%)
Max RSS bytes4409384960 (4205 MiB)
Outputting started2025-10-29 21:32:41
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09448/1/001/reco_stage1.root
Finished2025-10-29 21:34:27
Saved logsjustin-logs:234883.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 137568 traces, 0 summary
Retagger: tagging trace set: gauss with 106895 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106895 traces tagged "gauss"
FrameSaver: q=3.64495e+06 n=1899875 tag=gauss
wclsFrameSaver: saving 137568 traces tagged "wiener"
FrameSaver: q=3.84665e+06 n=1686140 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36460 XUs and 54689 XVs -> 554 XUVs
C:0 T:1 74365 XUs and 92755 XVs -> 4411 XUVs
C:0 T:2 223739 XUs and 217984 XVs -> 13681 XUVs
C:0 T:3 79721 XUs and 73964 XVs -> 848 XUVs
C:0 T:4 9454 XUs and 12150 XVs -> 57 XUVs
C:0 T:5 33837 XUs and 26537 XVs -> 3061 XUVs
C:0 T:6 131193 XUs and 110768 XVs -> 7817 XUVs
C:0 T:7 45566 XUs and 35640 XVs -> 339 XUVs
C:0 T:8 6620 XUs and 5650 XVs -> 95 XUVs
C:0 T:9 54256 XUs and 51248 XVs -> 4537 XUVs
C:0 T:10 15947 XUs and 7564 XVs -> 1789 XUVs
C:0 T:11 336 XUs and 272 XVs -> 41 XUVs
37230 XUVs total
13142 collection wire objects
33259 potential space points
Neighbour search...
3277221 tests to find 1525002 neighbours
Iterating with no regularization...
Begin: 9.0415e+07
0 7.77116e+07
1 7.68904e+07
2 7.67767e+07
3 7.6741e+07
Now with regularization...
Begin: 6.56549e+07
0 6.56289e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410190999
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410190999
fLifetime: 30000 [us]
29-Oct-2025 22:32:39 CET  Closed output file "reco_stage1.root"
29-Oct-2025 22:32:39 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          375.684       523.817       643.406       515.735       75.3359        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00113191    0.00527743    0.00951853    0.00567328    0.00273606       10     
reco:ophit:OpHitFinder                             0.292341      0.329016      0.376186      0.337266      0.0269908       10     
reco:opflash:OpFlashFinder                         0.179096      0.236132      0.315855      0.228338      0.0419161       10     
reco:opslicer:OpSlicer                             0.715068      0.827834      0.998703      0.812897      0.0856796       10     
reco:caldata:DataPrepModule                         4.98892       5.46596       6.11062       5.37227      0.371378        10     
reco:wclsdatasp:WireCellToolkit                     41.4285       67.9783       78.6313       74.1025       13.3206        10     
reco:gaushit:GausHitFinder                          1.13698       1.50382       1.78247       1.45997      0.178867        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000596697   0.000821652   0.00107058    0.000807535   0.000131199      10     
reco:reco3d:SpacePointSolver                        10.669        19.624        26.8196       20.8476       4.66618        10     
reco:hitpdune:DisambigFromSpacePoints               1.19879       2.05387       2.59968       1.99428      0.365266        10     
reco:pandora:StandardPandora                        22.749        49.1999       81.2341       42.614        16.0707        10     
reco:pandoraWriter:StandardPandora                 0.438509      0.535411      0.594222      0.537952      0.042607        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.45589       8.01829       11.2323       8.18566       1.58898        10     
reco:pandoraShower:LArPandoraShowerCreation         5.0474        9.65396       14.1445       9.02735       2.29133        10     
reco:pandoracalo:Calorimetry                        2.9953        4.27264       5.57333       4.47133      0.752352        10     
reco:pandoracalonosce:Calorimetry                   2.8169        4.02079       5.27778       4.22202      0.764328        10     
reco:pandorapid:Chi2ParticleID                    0.00634287    0.00730243    0.00830185    0.00736781    0.000623507      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0485284     0.0616383     0.0701186     0.0626097    0.00658441       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00602531    0.00706238    0.00853237    0.00710166    0.000708618      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.85781       9.14559       13.5691       8.64405       2.23987        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.7306        8.81849       12.5709       8.38312       2.03941        10     
reco:emtrkmichelid:EmTrackMichelId                  189.211       254.273       297.125       251.933       29.8517        10     
reco:crttag:SingleCRTMatchingProducer              0.0156403     0.0220785     0.0309858     0.019362     0.00543716       10     
reco:crtreco:TwoCRTMatchingProducer                0.0115437     0.0155382     0.0226016     0.0144598    0.00338135       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.748179       1.04625       1.36945       1.01352      0.162739        10     
reco:pandora2Track:LArPandoraTrackCreation          15.1801       24.3929       37.7151       24.3422       6.3792         10     
reco:pandora2calo:Calorimetry                       7.22372       11.7933        17.37        11.5902       3.14927        10     
reco:pandora2calonosce:Calorimetry                  6.7566        10.877        16.3593       10.3761       2.94124        10     
reco:pandora2pid:Chi2ParticleID                    0.014069      0.0182702     0.0249527     0.0178182    0.00276217       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0603224     0.0794459     0.0954829     0.0789392    0.00938843       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.012762      0.0178726     0.0237129     0.0178339    0.00273646       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.11673       8.59229       12.2948       8.65078       2.09758        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.43245       9.24209       13.4914       9.33538       2.29759        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.71005       7.88356       11.5531       7.69855       1.96151        10     
[art]:TriggerResults:TriggerResultInserter        3.0017e-05    4.73191e-05   0.000132061   3.80325e-05   2.85895e-05      10     
end_path:out1:RootOutput                           5.74e-06     8.4532e-06    2.2533e-05    6.2465e-06    5.02438e-06      10     
end_path:out1:RootOutput(write)                     2.54741       3.61694       4.41127       3.59566      0.504738        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5214.23 MB
  Peak resident set size usage (VmHWM): 4409.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.43 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10978476
-rw-r--r--. 1 dune009 dune 2552962756 Oct 29 21:03 detsim_stage2.root
-rw-r--r--. 1 dune009 dune 2416268607 Oct 29 21:00 detsim_stage1.root
-rw-r--r--. 1 dune009 dune 2294243548 Oct 29 22:32 reco_stage1.root
-rw-r--r--. 1 dune009 dune 1990340803 Oct 29 18:34 g4_stage2.root
-rw-r--r--. 1 dune009 dune 1899894242 Oct 29 18:19 g4_stage1.root
-rw-r--r--. 1 dune009 dune   87631850 Oct 29 22:26 Pandora_Events.pndr
-rw-r--r--. 1 dune009 dune     243140 Oct 29 22:32 jobscript.log
-rw-r--r--. 1 dune009 dune     217088 Oct 29 22:32 mem.db
-rw-r--r--. 1 dune009 dune      62376 Oct 29 22:32 Pandora_Geometry.xml
-rw-r--r--. 1 dune009 dune      36864 Oct 29 22:32 time.db
-rw-r--r--. 1 dune009 dune        519 Oct 29 21:03 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 18:34 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune009 dune        519 Oct 29 22:32 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune009 dune         52 Oct 29 18:12 all-input-dids.txt
-rw-r--r--. 1 dune009 dune          7 Oct 29 22:32 justin-processed-pfns.txt
-rw-r--r--. 1 dune009 dune          0 Oct 29 21:10 ProtoDUNE.log
-rw-r--r--. 1 dune009 dune          0 Oct 29 21:03 debugprod.log
justIN time: 2025-11-03 18:41:08 UTC       justIN version: 01.05.01