justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270847.0@dunegpschedd02.fnal.gov

Jobsub ID270847.0@dunegpschedd02.fnal.gov
Workflow ID11766
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 21:48:33
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-13 01:41:19
From worker nodeHostnamecn033.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 21:49:45
Input filesmonte-carlo-011766-000001
JobscriptExit code0
Real time3h (13648s)
CPU time3h (13468s = 98%)
Max RSS bytes4351655936 (4150 MiB)
Outputting started2026-01-13 01:37:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11766/1/001/reco_stage1.root
Finished2026-01-13 01:41:19
Saved logsjustin-logs:270847.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ng tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 161127 traces, 0 summary
Retagger: tagging trace set: gauss with 124383 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 124383 traces tagged "gauss"
FrameSaver: q=4.38319e+06 n=2254761 tag=gauss
wclsFrameSaver: saving 161127 traces tagged "wiener"
FrameSaver: q=4.62945e+06 n=2009616 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 28613 XUs and 24735 XVs -> 486 XUVs
C:0 T:1 62924 XUs and 76930 XVs -> 15319 XUVs
C:0 T:2 176051 XUs and 181677 XVs -> 8382 XUVs
C:0 T:3 76377 XUs and 65890 XVs -> 594 XUVs
C:0 T:4 17887 XUs and 15874 XVs -> 201 XUVs
C:0 T:5 68918 XUs and 47266 XVs -> 4312 XUVs
C:0 T:6 127653 XUs and 123051 XVs -> 5423 XUVs
C:0 T:7 49332 XUs and 46507 XVs -> 399 XUVs
C:0 T:8 9842 XUs and 8327 XVs -> 77 XUVs
C:0 T:9 60851 XUs and 84449 XVs -> 5961 XUVs
C:0 T:10 80567 XUs and 56736 XVs -> 6934 XUVs
C:0 T:11 228 XUs and 143 XVs -> 12 XUVs
48100 XUVs total
14766 collection wire objects
44478 potential space points
Neighbour search...
6931396 tests to find 2939330 neighbours
Iterating with no regularization...
Begin: 8.09751e+07
0 6.65628e+07
1 6.5611e+07
2 6.5486e+07
3 6.54532e+07
Now with regularization...
Begin: 5.42368e+07
0 5.41924e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411703793
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411703793
fLifetime: 30000 [us]
13-Jan-2026 01:37:11 GMT  Closed output file "reco_stage1.root"
13-Jan-2026 01:37:11 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          360.695       570.16        740.446       551.203       122.181        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00140698    0.00412387    0.00652655    0.00431442    0.00171598       10     
reco:ophit:OpHitFinder                              0.24562      0.297605      0.324002       0.30339      0.0226137       10     
reco:opflash:OpFlashFinder                         0.192182      0.258746      0.317088      0.257812      0.0317277       10     
reco:opslicer:OpSlicer                             0.567292      0.725569      0.851614      0.728879      0.0958458       10     
reco:caldata:DataPrepModule                         5.39601       5.67341       5.99746       5.69714      0.155843        10     
reco:wclsdatasp:WireCellToolkit                     46.7409       50.0007       52.7054       49.9559       1.88933        10     
reco:gaushit:GausHitFinder                          1.12429       1.64461       2.30873       1.55226       0.3167         10     
reco:nhitsfilter:NumberOfHitsFilter               0.000689402   0.000983894   0.00117426    0.00102144    0.000156594      10     
reco:reco3d:SpacePointSolver                        10.708        19.0351       29.2681       17.9781        5.537         10     
reco:hitpdune:DisambigFromSpacePoints               1.43659       2.76391       3.94066       2.53354       0.8307         10     
reco:pandora:StandardPandora                        28.4908       54.8913       93.4087       48.2766       21.1066        10     
reco:pandoraWriter:StandardPandora                 0.429837      0.581777       0.70112      0.585349      0.0849713       10     
reco:pandoraTrack:LArPandoraTrackCreation           9.17228       13.4671       16.1703       13.9357       2.17625        10     
reco:pandoraShower:LArPandoraShowerCreation         8.97493       18.1972       25.8449       17.1951       5.59553        10     
reco:pandoracalo:Calorimetry                        3.4665        6.22696       7.57875       6.69796       1.19762        10     
reco:pandoracalonosce:Calorimetry                   3.53857       5.93434       7.12341       6.31898       1.1263         10     
reco:pandorapid:Chi2ParticleID                    0.00543554    0.00656184    0.00767585    0.00673722    0.000630531      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0487734     0.0705887     0.085159      0.0725031     0.0106334       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00553422    0.00624096    0.00677124     0.006439     0.000440818      10     
reco:pandoraShowercalo:ShowerCalorimetry            7.2104        16.5114       23.8879        15.67        5.50076        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.87313       15.9281       23.1526       15.4408       5.36875        10     
reco:emtrkmichelid:EmTrackMichelId                  156.016       215.405       261.457       218.964       33.5782        10     
reco:crttag:SingleCRTMatchingProducer              0.0206575     0.0343018     0.0400761     0.037028     0.00589774       10     
reco:crtreco:TwoCRTMatchingProducer                0.0185891     0.0292557     0.0340514     0.0309815    0.00475128       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.987154       1.40534       1.74702       1.43884       0.21963        10     
reco:pandora2Track:LArPandoraTrackCreation          24.747        48.2127       69.2553       46.4557       13.7197        10     
reco:pandora2calo:Calorimetry                       10.3503       21.631        31.8779       20.8328       6.5678         10     
reco:pandora2calonosce:Calorimetry                  9.27841       20.5095       30.496        19.3774       6.50745        10     
reco:pandora2pid:Chi2ParticleID                    0.012837      0.0167799     0.0248171     0.0157198    0.00360293       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0650228     0.100826      0.183649      0.0942231     0.032194        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0127824     0.0174873     0.0325636     0.0152996    0.00578187       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.82746       15.1913       21.258         15.38        4.01621        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.13361       17.268        26.7221       17.2742       5.28638        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.21579       14.7894       23.7881       15.1382       4.86192        10     
[art]:TriggerResults:TriggerResultInserter        5.0333e-05    6.30444e-05   0.000121794   5.7283e-05    2.00988e-05      10     
end_path:out1:RootOutput                           7.606e-06     8.989e-06    1.4431e-05     8.331e-06    1.89332e-06      10     
end_path:out1:RootOutput(write)                     2.2223        3.12299       4.03895       3.08729      0.574112        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5160.19 MB
  Peak resident set size usage (VmHWM): 4351.66 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.53 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11148352
-rw-r--r-- 1 pildune32 pildune 2579359221 Jan 13 00:00 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2444400543 Jan 12 23:57 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2312807362 Jan 13 01:37 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2040072300 Jan 12 22:19 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1952357866 Jan 12 21:56 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   86299310 Jan 13 01:27 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243150 Jan 13 01:37 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 13 01:37 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 13 01:37 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 13 01:37 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 13 00:00 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 12 22:19 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 13 01:37 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 12 21:49 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 13 01:37 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 13 00:07 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 13 00:00 debugprod.log
justIN time: 2026-02-06 09:00:45 UTC       justIN version: 01.06.00