justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257947.0@dunegpschedd02.fnal.gov

Jobsub ID257947.0@dunegpschedd02.fnal.gov
Workflow ID11033
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-05 12:33:42
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-12-05 18:35:39
From worker nodeHostnamecn512.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-05 12:35:37
Input filesmonte-carlo-011033-000001
JobscriptExit code0
Real time5h (21392s)
CPU time5h (21074s = 98%)
Max RSS bytes4426543104 (4221 MiB)
Outputting started2025-12-05 18:32:09
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11033/1/001/reco_stage1.root
Finished2025-12-05 18:35:39
Saved logsjustin-logs:257947.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123471 traces, 0 summary
Retagger: tagging trace set: gauss with 94562 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94562 traces tagged "gauss"
FrameSaver: q=3.40682e+06 n=1687611 tag=gauss
wclsFrameSaver: saving 123471 traces tagged "wiener"
FrameSaver: q=3.59909e+06 n=1501071 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32884 XUs and 35795 XVs -> 564 XUVs
C:0 T:1 64445 XUs and 96299 XVs -> 9526 XUVs
C:0 T:2 92611 XUs and 101180 XVs -> 3171 XUVs
C:0 T:3 51341 XUs and 47001 XVs -> 189 XUVs
C:0 T:4 10479 XUs and 12627 XVs -> 82 XUVs
C:0 T:5 29688 XUs and 28372 XVs -> 2339 XUVs
C:0 T:6 134669 XUs and 131831 XVs -> 6125 XUVs
C:0 T:7 53959 XUs and 49729 XVs -> 580 XUVs
C:0 T:8 3749 XUs and 5243 XVs -> 101 XUVs
C:0 T:9 34980 XUs and 59758 XVs -> 10947 XUVs
C:0 T:10 23907 XUs and 17630 XVs -> 4431 XUVs
C:0 T:11 186 XUs and 116 XVs -> 19 XUVs
38074 XUVs total
10295 collection wire objects
35067 potential space points
Neighbour search...
4461565 tests to find 2120942 neighbours
Iterating with no regularization...
Begin: 6.89854e+07
0 5.99239e+07
1 5.91365e+07
2 5.90141e+07
3 5.89779e+07
Now with regularization...
Begin: 4.85594e+07
0 4.85315e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410931596
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410931596
fLifetime: 30000 [us]
05-Dec-2025 18:32:05 GMT  Closed output file "reco_stage1.root"
05-Dec-2025 18:32:05 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          650.606       844.939       1271.19       751.651       186.817        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00192054    0.00597296     0.0102849    0.00567077    0.00256605       10     
reco:ophit:OpHitFinder                             0.389839      0.556536      0.682029      0.561326      0.0714342       10     
reco:opflash:OpFlashFinder                         0.280338      0.343369      0.438146      0.321496      0.0534694       10     
reco:opslicer:OpSlicer                             0.959322       1.27518       1.61824       1.2516       0.189838        10     
reco:caldata:DataPrepModule                         7.45548       11.5343       13.3459       11.9199       1.45483        10     
reco:wclsdatasp:WireCellToolkit                     65.2088       82.9414       88.8186       84.3116       6.31568        10     
reco:gaushit:GausHitFinder                          1.89904       2.74577       3.70759       2.67043      0.493772        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000852693    0.0010861    0.00135566    0.00106798    0.000164037      10     
reco:reco3d:SpacePointSolver                        20.5906       27.1794       32.5747       27.5643       4.62877        10     
reco:hitpdune:DisambigFromSpacePoints               2.99758       4.33082       6.90786       3.91461       1.21975        10     
reco:pandora:StandardPandora                        50.1641       75.5973       160.018       60.1953       35.3928        10     
reco:pandoraWriter:StandardPandora                 0.764158      0.965517       1.30494      0.892433      0.187712        10     
reco:pandoraTrack:LArPandoraTrackCreation           10.6609       15.9265       25.492        14.7481       4.97225        10     
reco:pandoraShower:LArPandoraShowerCreation         8.80956       17.0291       31.4377       15.8555       6.95115        10     
reco:pandoracalo:Calorimetry                        4.89741       7.61668       12.5626       7.40188       2.40308        10     
reco:pandoracalonosce:Calorimetry                   4.57858       7.22316       11.7015       6.79739       2.27904        10     
reco:pandorapid:Chi2ParticleID                     0.010836      0.0130579     0.0170967     0.012558     0.00203154       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.102959      0.131166      0.174635      0.135848      0.021535        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00942874     0.0131295     0.0180498     0.0124566    0.00273935       10     
reco:pandoraShowercalo:ShowerCalorimetry            8.0955        17.6914       33.2565       16.9867       8.37702        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       8.24424       17.0245       32.7966       16.211        7.78934        10     
reco:emtrkmichelid:EmTrackMichelId                  323.362       394.306       544.372       371.083       67.1671        10     
reco:crttag:SingleCRTMatchingProducer              0.0371139     0.0466796     0.0579154     0.0481185    0.00708825       10     
reco:crtreco:TwoCRTMatchingProducer                0.0305251     0.0377853     0.0464224     0.0366771    0.00575581       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.63839       2.14772       2.71936       1.99035      0.362658        10     
reco:pandora2Track:LArPandoraTrackCreation          29.0568       50.5784       89.3112       44.3798       18.7234        10     
reco:pandora2calo:Calorimetry                       12.5283       23.2106       42.8782       18.7467       9.69285        10     
reco:pandora2calonosce:Calorimetry                  12.1288       22.4278       40.6347       19.1801       9.04634        10     
reco:pandora2pid:Chi2ParticleID                    0.0217306     0.0312339     0.043774      0.0297228    0.00774561       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.119535      0.165486      0.215732      0.164454      0.0342664       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0216845     0.0307773     0.0424904     0.0299981    0.00730877       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.1996       16.6656       24.8649       15.5412       5.01224        10     
reco:pandora2Showercalo:ShowerCalorimetry            11.26        20.4927       34.503        18.9775       7.64327        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      10.5668       19.071        30.542        18.6382       6.94099        10     
[art]:TriggerResults:TriggerResultInserter        7.4406e-05    0.00013357    0.000271916   0.000115789   5.21169e-05      10     
end_path:out1:RootOutput                          1.3246e-05    5.09464e-05   0.00034062    1.7947e-05    9.66964e-05      10     
end_path:out1:RootOutput(write)                     4.40964       5.32735       7.26062       4.97355      0.855696        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5225.02 MB
  Peak resident set size usage (VmHWM): 4426.54 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.45 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   7.21 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.41 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9916808
-rw-r--r-- 1 pildune22 pildune 2337773562 Dec  5 16:07 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2217778750 Dec  5 16:03 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2006317606 Dec  5 18:32 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1788020033 Dec  5 13:11 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1733860347 Dec  5 12:46 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   70450510 Dec  5 18:20 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243100 Dec  5 18:32 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Dec  5 18:32 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Dec  5 18:32 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Dec  5 18:32 time.db
-rw-r--r-- 1 pildune22 pildune        519 Dec  5 16:07 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec  5 13:11 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec  5 18:32 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Dec  5 12:35 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Dec  5 18:32 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Dec  5 16:18 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Dec  5 16:07 debugprod.log
justIN time: 2026-02-07 19:09:12 UTC       justIN version: 01.06.00