justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 279846.0@dunegpschedd01.fnal.gov

Jobsub ID279846.0@dunegpschedd01.fnal.gov
Workflow ID11999
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-17 15:54:35
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-17 18:35:19
From worker nodeHostnamecn044.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-17 15:55:24
Input filesmonte-carlo-011999-000001
JobscriptExit code0
Real time2h (9387s)
CPU time2h (9231s = 98%)
Max RSS bytes3988504576 (3803 MiB)
Outputting started2026-01-17 18:31:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11999/1/001/reco_stage1.root
Finished2026-01-17 18:35:19
Saved logsjustin-logs:279846.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

r
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 81908 traces, 0 summary
Retagger: tagging trace set: gauss with 60848 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 60848 traces tagged "gauss"
FrameSaver: q=1.9956e+06 n=1059993 tag=gauss
wclsFrameSaver: saving 81908 traces tagged "wiener"
FrameSaver: q=2.11618e+06 n=927999 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 17531 XUs and 15546 XVs -> 92 XUVs
C:0 T:1 31220 XUs and 26793 XVs -> 1406 XUVs
C:0 T:2 56492 XUs and 70114 XVs -> 3167 XUVs
C:0 T:3 27915 XUs and 30073 XVs -> 169 XUVs
C:0 T:4 7348 XUs and 6387 XVs -> 58 XUVs
C:0 T:5 29902 XUs and 21959 XVs -> 3029 XUVs
C:0 T:6 71997 XUs and 84386 XVs -> 3429 XUVs
C:0 T:7 29633 XUs and 32016 XVs -> 356 XUVs
C:0 T:9 25080 XUs and 38321 XVs -> 2188 XUVs
C:0 T:10 14918 XUs and 10683 XVs -> 1624 XUVs
C:0 T:11 129 XUs and 92 XVs -> 24 XUVs
15542 XUVs total
7374 collection wire objects
14340 potential space points
Neighbour search...
539848 tests to find 291710 neighbours
Iterating with no regularization...
Begin: 4.86184e+07
0 3.69827e+07
1 3.63569e+07
2 3.63124e+07
3 3.62995e+07
Now with regularization...
Begin: 3.14505e+07
0 3.14149e+07
1 3.14113e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411799420
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411799420
fLifetime: 30000 [us]
17-Jan-2026 18:31:49 GMT  Closed output file "reco_stage1.root"
17-Jan-2026 18:31:49 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          272.814       410.622       580.919       401.046       97.411         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00134209    0.00368639    0.00548669    0.00361129    0.00126422       10     
reco:ophit:OpHitFinder                             0.164686      0.231463       0.31027      0.226243      0.040867        10     
reco:opflash:OpFlashFinder                         0.0989844     0.165524      0.259796      0.169161      0.0401937       10     
reco:opslicer:OpSlicer                             0.335934      0.519826      0.743904      0.494412      0.118861        10     
reco:caldata:DataPrepModule                         4.95718       5.46039       7.82836       5.17153      0.827272        10     
reco:wclsdatasp:WireCellToolkit                     44.9285       47.321        49.4304       47.5344       1.64766        10     
reco:gaushit:GausHitFinder                          0.83545       1.11537       1.44027       1.10886      0.177155        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000543292   0.000793318   0.00121584    0.000807317   0.000194049      10     
reco:reco3d:SpacePointSolver                        5.09591       10.3217       15.1332       11.3166       3.21706        10     
reco:hitpdune:DisambigFromSpacePoints              0.765455       1.45029       2.25024       1.44856      0.411284        10     
reco:pandora:StandardPandora                        12.288        27.9131       41.1713       28.9897       11.203         10     
reco:pandoraWriter:StandardPandora                 0.301914      0.416749      0.526116      0.420336      0.068589        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.47524       10.3884       14.3415       11.0386       2.96495        10     
reco:pandoraShower:LArPandoraShowerCreation         4.78012       10.8432       16.8713       11.8109       3.66986        10     
reco:pandoracalo:Calorimetry                        2.40214       4.93838       9.06778       4.89255       1.95054        10     
reco:pandoracalonosce:Calorimetry                   2.37268       4.7195        7.48605       4.84027       1.73755        10     
reco:pandorapid:Chi2ParticleID                     0.003691     0.00561423    0.00760367    0.00555119    0.00113752       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0366371     0.0505054     0.0596257     0.0508562    0.00798814       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00387967    0.00521134    0.00643597    0.00528658    0.000713164      10     
reco:pandoraShowercalo:ShowerCalorimetry            3.86015       9.93882       16.2639       10.1421       3.73074        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.89035       9.80087       15.2733       10.9219       3.70825        10     
reco:emtrkmichelid:EmTrackMichelId                  118.799       162.618       235.306       163.597       32.469         10     
reco:crttag:SingleCRTMatchingProducer              0.0247289     0.0354732     0.053646      0.0316149     0.011262        10     
reco:crtreco:TwoCRTMatchingProducer                0.0202654     0.0305932     0.0450659     0.0268459    0.00928643       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.789867       1.20873       1.82082       1.08204      0.338632        10     
reco:pandora2Track:LArPandoraTrackCreation          17.9589       36.5126       58.2089       32.0618       14.1898        10     
reco:pandora2calo:Calorimetry                       7.78364       14.7843       26.5613       13.0392       6.11752        10     
reco:pandora2calonosce:Calorimetry                  7.1809        13.9111       25.5684       12.6528       5.91239        10     
reco:pandora2pid:Chi2ParticleID                   0.00860493     0.0138808     0.0253431     0.0119336    0.00509416       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0474656     0.0725789     0.120011      0.0657173     0.021684        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00826681     0.013653      0.024615      0.0117996    0.00480845       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.10167       11.3086       16.5989       11.5151       3.50597        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.33495       11.9059       22.1222       11.5166       5.30024        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.82612       10.0519       18.4521       9.55988       4.41763        10     
[art]:TriggerResults:TriggerResultInserter        4.0502e-05    5.82538e-05   0.000122809   5.1007e-05    2.35085e-05      10     
end_path:out1:RootOutput                           7.982e-06    1.11171e-05   2.6286e-05     9.171e-06    5.1327e-06       10     
end_path:out1:RootOutput(write)                     1.45359       2.35198       3.66852       2.08799      0.743289        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4796.3 MB
  Peak resident set size usage (VmHWM): 3988.5 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.31 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9070192
-rw-r--r-- 1 pildune32 pildune 2181284996 Jan 17 17:21 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2075373182 Jan 17 17:19 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1821255788 Jan 17 18:31 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1609970397 Jan 17 16:13 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1535690594 Jan 17 16:01 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   63688260 Jan 17 18:28 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242511 Jan 17 18:31 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 17 18:31 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 17 18:31 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 17 18:31 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 17 17:21 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 17 16:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 17 18:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 17 15:55 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 17 18:31 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 17 17:27 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 17 17:21 debugprod.log
justIN time: 2026-02-05 22:04:07 UTC       justIN version: 01.06.00