justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241996.0@dunegpschedd01.fnal.gov

Jobsub ID241996.0@dunegpschedd01.fnal.gov
Workflow ID9528
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 14:19:37
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-02 21:28:25
From worker nodeHostnamewn1205250.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 14:20:51
Input filesmonte-carlo-009528-000001
JobscriptExit code0
Real time7h (25528s)
CPU time6h (24451s = 95%)
Max RSS bytes4265574400 (4067 MiB)
Outputting started2025-11-02 21:26:20
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09528/1/001/reco_stage1.root
Finished2025-11-02 21:28:25
Saved logsjustin-logs:241996.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

olBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 144988 traces, 0 summary
Retagger: tagging trace set: gauss with 109071 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 109071 traces tagged "gauss"
FrameSaver: q=3.83865e+06 n=1980545 tag=gauss
wclsFrameSaver: saving 144988 traces tagged "wiener"
FrameSaver: q=4.0614e+06 n=1746703 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36553 XUs and 28136 XVs -> 410 XUVs
C:0 T:1 64933 XUs and 50634 XVs -> 4364 XUVs
C:0 T:2 123370 XUs and 132132 XVs -> 5828 XUVs
C:0 T:3 58197 XUs and 49476 XVs -> 569 XUVs
C:0 T:4 11585 XUs and 10540 XVs -> 190 XUVs
C:0 T:5 57362 XUs and 31224 XVs -> 5999 XUVs
C:0 T:6 137353 XUs and 132956 XVs -> 12092 XUVs
C:0 T:7 47453 XUs and 42739 XVs -> 564 XUVs
C:0 T:8 11673 XUs and 13178 XVs -> 408 XUVs
C:0 T:9 92923 XUs and 96711 XVs -> 6846 XUVs
C:0 T:10 70260 XUs and 49718 XVs -> 8407 XUVs
C:0 T:11 36 XUs and 22 XVs -> 1 XUVs
45678 XUVs total
14462 collection wire objects
42029 potential space points
Neighbour search...
3688243 tests to find 1894432 neighbours
Iterating with no regularization...
Begin: 6.83393e+07
0 5.62807e+07
1 5.52101e+07
2 5.50105e+07
3 5.49478e+07
4 5.49199e+07
Now with regularization...
Begin: 4.43241e+07
0 4.42926e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410269032
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410269032
fLifetime: 30000 [us]
02-Nov-2025 21:26:17 GMT  Closed output file "reco_stage1.root"
02-Nov-2025 21:26:17 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          640.65        960.15        1145.3        970.722       134.093        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00224885     0.0142122     0.0287487     0.0116177    0.00986909       10     
reco:ophit:OpHitFinder                             0.453116      0.519326      0.582655      0.527999      0.0389369       10     
reco:opflash:OpFlashFinder                         0.286463      0.350099      0.453213      0.340029      0.0566447       10     
reco:opslicer:OpSlicer                              1.20076       1.47199       1.62791       1.4721       0.120333        10     
reco:caldata:DataPrepModule                         9.43311       9.71326       10.0817       9.64116      0.239075        10     
reco:wclsdatasp:WireCellToolkit                     70.3146       73.9843       82.1453       73.3603       3.00222        10     
reco:gaushit:GausHitFinder                          2.08621       2.64806       3.74994       2.53783      0.435233        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00110211    0.00165924    0.00206993    0.00168225    0.000287916      10     
reco:reco3d:SpacePointSolver                        19.4563       35.5485       66.9345       31.6756       12.5771        10     
reco:hitpdune:DisambigFromSpacePoints               2.51884       3.82458       5.19089       3.72286      0.734678        10     
reco:pandora:StandardPandora                        71.0481       100.905       170.538       92.0141       27.3487        10     
reco:pandoraWriter:StandardPandora                 0.865755       1.12858       1.29766       1.13251      0.130155        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.90306       14.7068       21.5453       14.5697        3.129         10     
reco:pandoraShower:LArPandoraShowerCreation         8.42625       18.9211       27.9919       19.2286       5.02401        10     
reco:pandoracalo:Calorimetry                        4.03896       8.74932       11.8983       8.89498       1.95654        10     
reco:pandoracalonosce:Calorimetry                   4.34982       8.35949       11.0328       8.45107       1.69119        10     
reco:pandorapid:Chi2ParticleID                     0.0124408     0.0133619     0.0147128     0.0129191    0.000888009      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0945463     0.139679      0.172572      0.140207      0.0232202       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0114654     0.0134246     0.0175053     0.0127596    0.00177511       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.64719       22.4053       32.6896       23.4756       6.43224        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.12621       21.7894       30.8919       22.927        5.94807        10     
reco:emtrkmichelid:EmTrackMichelId                  347.925       458.854       520.643       472.633       48.0379        10     
reco:crttag:SingleCRTMatchingProducer              0.0373665     0.0592166     0.080352      0.0612912     0.0144149       10     
reco:crtreco:TwoCRTMatchingProducer                0.0188571     0.0358005     0.056501      0.0350327    0.00901835       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.10351       1.70736       2.02611       1.69708      0.243342        10     
reco:pandora2Track:LArPandoraTrackCreation          22.5145       49.171        70.6448       50.6756       11.5273        10     
reco:pandora2calo:Calorimetry                       11.5799       28.2323       37.4078       29.0319       6.73678        10     
reco:pandora2calonosce:Calorimetry                  10.0864       26.9822       36.3307       27.4966       6.53727        10     
reco:pandora2pid:Chi2ParticleID                    0.0230786     0.0325306     0.0401445     0.032121     0.00450076       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP         0.12149      0.181228      0.221681      0.181153      0.0263355       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0234863     0.0324664     0.0433424     0.0319526    0.00562746       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.29762       16.8887       22.5272       17.3413       3.69336        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.63453       25.0002       33.8714       26.7438       6.25014        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.1792        21.5036       27.9652       23.254        5.19883        10     
[art]:TriggerResults:TriggerResultInserter        4.7609e-05    7.03157e-05   0.000195278    5.804e-05    4.21988e-05      10     
end_path:out1:RootOutput                           6.883e-06    8.8166e-06     1.609e-05    7.7095e-06    2.59263e-06      10     
end_path:out1:RootOutput(write)                      4.047        5.91579       6.99322       5.93873      0.832868        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5114.04 MB
  Peak resident set size usage (VmHWM): 4265.57 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.40 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.19 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10701580
-rw-r--r-- 1 dune01 dune01 2493368944 Nov  2 18:42 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2361499226 Nov  2 18:38 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2218884497 Nov  2 21:26 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1930110843 Nov  2 14:56 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1869369680 Nov  2 14:32 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   84555590 Nov  2 21:13 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243213 Nov  2 21:26 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  2 21:26 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  2 21:26 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  2 21:26 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  2 18:42 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 14:56 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 21:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  2 14:20 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  2 21:26 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  2 18:56 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  2 18:42 debugprod.log
justIN time: 2025-11-05 04:50:02 UTC       justIN version: 01.05.01