justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242658.0@dunegpschedd01.fnal.gov

Jobsub ID242658.0@dunegpschedd01.fnal.gov
Workflow ID9572
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-03 21:49:18
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-04 03:53:17
From worker nodeHostnamecn048.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-03 21:50:40
Input filesmonte-carlo-009572-000001
JobscriptExit code0
Real time5h (21518s)
CPU time5h (21071s = 97%)
Max RSS bytes4606758912 (4393 MiB)
Outputting started2025-11-04 03:49:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09572/1/001/reco_stage1.root
Finished2025-11-04 03:53:17
Saved logsjustin-logs:242658.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 126800 traces, 0 summary
Retagger: tagging trace set: gauss with 97080 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 97080 traces tagged "gauss"
FrameSaver: q=3.38552e+06 n=1742823 tag=gauss
wclsFrameSaver: saving 126800 traces tagged "wiener"
FrameSaver: q=3.56874e+06 n=1547299 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 49778 XUs and 43343 XVs -> 694 XUVs
C:0 T:1 125921 XUs and 103989 XVs -> 8739 XUVs
C:0 T:2 70744 XUs and 80854 XVs -> 3108 XUVs
C:0 T:3 37325 XUs and 39933 XVs -> 278 XUVs
C:0 T:4 9937 XUs and 11277 XVs -> 80 XUVs
C:0 T:5 31996 XUs and 35933 XVs -> 4171 XUVs
C:0 T:6 122388 XUs and 133321 XVs -> 7652 XUVs
C:0 T:7 49380 XUs and 53460 XVs -> 537 XUVs
C:0 T:8 5728 XUs and 6639 XVs -> 64 XUVs
C:0 T:9 38381 XUs and 77882 XVs -> 7602 XUVs
C:0 T:10 24798 XUs and 16527 XVs -> 3327 XUVs
C:0 T:11 134 XUs and 78 XVs -> 18 XUVs
36270 XUVs total
11640 collection wire objects
33188 potential space points
Neighbour search...
2948512 tests to find 1459544 neighbours
Iterating with no regularization...
Begin: 7.53698e+07
0 6.63564e+07
1 6.56777e+07
2 6.55752e+07
3 6.55448e+07
Now with regularization...
Begin: 5.523e+07
0 5.52071e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410295603
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410295603
fLifetime: 30000 [us]
04-Nov-2025 03:49:16 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 03:49:16 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          683.27        936.079       1224.64       898.774       174.274        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0025655     0.051314      0.465451     0.00539218     0.138054        10     
reco:ophit:OpHitFinder                             0.394448      0.666527       1.49808      0.546084       0.30946        10     
reco:opflash:OpFlashFinder                         0.282216      0.423662      0.600509      0.412511      0.0947299       10     
reco:opslicer:OpSlicer                             0.966428       1.2119        1.67617       1.17912      0.195238        10     
reco:caldata:DataPrepModule                         9.2491        9.91383       10.4667       9.98226      0.389629        10     
reco:wclsdatasp:WireCellToolkit                     52.0275       71.2853       78.6771       73.2975       7.23792        10     
reco:gaushit:GausHitFinder                          1.64063       2.46048       3.14785       2.53673      0.451195        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000989795   0.00142945    0.00270567    0.00121023    0.000526715      10     
reco:reco3d:SpacePointSolver                        17.6991       28.088        45.9665       25.9117       8.88498        10     
reco:hitpdune:DisambigFromSpacePoints               3.08306       4.98393       7.41359       4.90817       1.46125        10     
reco:pandora:StandardPandora                        54.0072       100.057       197.341       83.8924       40.7739        10     
reco:pandoraWriter:StandardPandora                 0.834006       1.06285       1.27701       1.05515      0.123027        10     
reco:pandoraTrack:LArPandoraTrackCreation           16.7772       20.5107       24.5896       20.3887       2.5801         10     
reco:pandoraShower:LArPandoraShowerCreation         13.6991       24.6121        36.52        24.5225       6.30785        10     
reco:pandoracalo:Calorimetry                        8.16333       10.0222       12.4054       9.83754       1.36227        10     
reco:pandoracalonosce:Calorimetry                   6.51063       9.34378       11.8725       9.35989       1.52262        10     
reco:pandorapid:Chi2ParticleID                    0.00649847     0.0122278     0.0147503     0.0131142    0.00250499       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0760874     0.122527      0.159631      0.125014      0.0256795       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00644915     0.0110319     0.0145218     0.0117939    0.00226103       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.086        24.8438       40.9332       25.0277       7.98464        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       10.0424       24.1872       39.7366       24.3776       8.09451        10     
reco:emtrkmichelid:EmTrackMichelId                  285.624       389.315       486.517       381.766       62.6183        10     
reco:crttag:SingleCRTMatchingProducer              0.0328173     0.0505831     0.0793193     0.0494247     0.0134999       10     
reco:crtreco:TwoCRTMatchingProducer                0.0283337     0.042691      0.0705513     0.0406766     0.0111104       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.48552       2.29609       4.23091       2.1739       0.770226        10     
reco:pandora2Track:LArPandoraTrackCreation          48.4281       69.7416       103.751       64.9724       16.8734        10     
reco:pandora2calo:Calorimetry                       20.7698       32.0085       49.1103       31.1055       8.45668        10     
reco:pandora2calonosce:Calorimetry                  19.7007       30.5584       46.7605       30.2412       8.24424        10     
reco:pandora2pid:Chi2ParticleID                    0.0156297     0.0289422     0.0411582     0.0298414    0.00698087       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0962227      0.15422      0.228561       0.15791      0.0354041       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0161426     0.0279167     0.0412752     0.0296447    0.00703018       10     
reco:pandora2Shower:LArPandoraShowerCreation        16.0591       21.9081       30.9498       21.3204       5.37849        10     
reco:pandora2Showercalo:ShowerCalorimetry           16.5743       27.0971       40.7662       26.6277       7.7721         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      15.0241       23.7493       37.5224       22.7865       7.12199        10     
[art]:TriggerResults:TriggerResultInserter        5.2203e-05    0.000102513   0.000223611    8.703e-05    4.54584e-05      10     
end_path:out1:RootOutput                          1.0029e-05    1.44247e-05   2.2603e-05    1.3747e-05    3.38637e-06      10     
end_path:out1:RootOutput(write)                     3.22915       4.97237       6.75835       4.92994      0.977731        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5410.32 MB
  Peak resident set size usage (VmHWM): 4606.76 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.83 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.31 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11723896
-rw-r--r-- 1 pildune22 pildune 2696419975 Nov  4 01:08 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2554225661 Nov  4 01:03 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2449363064 Nov  4 03:49 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2156229734 Nov  3 22:30 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 2057788832 Nov  3 22:01 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   90624290 Nov  4 03:39 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242917 Nov  4 03:49 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  4 03:49 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  4 03:49 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  4 03:49 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 01:08 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  3 22:30 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 03:49 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  3 21:50 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  4 03:49 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 01:24 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 01:08 debugprod.log
justIN time: 2026-02-11 12:01:33 UTC       justIN version: 01.06.00