justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 20689.1@dunegpschedd01.fnal.gov

Jobsub ID20689.1@dunegpschedd01.fnal.gov
Workflow ID274
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-03 19:32:54
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-08-04 01:38:27
From worker nodeHostnamecn534.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-03 19:33:40
Input filesmonte-carlo-000274-000001
JobscriptExit code0
Real time6h (21689s)
CPU time5h (21333s = 98%)
Max RSS bytes3978047488 (3793 MiB)
Outputting started2025-08-04 01:35:10
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00274/1/001/reco_stage1.root
Finished2025-08-04 01:38:27
Saved logsjustin-logs:20689.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 108178 traces, 0 summary
Retagger: tagging trace set: gauss with 81993 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 81993 traces tagged "gauss"
FrameSaver: q=2.86171e+06 n=1445931 tag=gauss
wclsFrameSaver: saving 108178 traces tagged "wiener"
FrameSaver: q=3.03198e+06 n=1282018 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30200 XUs and 21951 XVs -> 256 XUVs
C:0 T:1 65253 XUs and 42639 XVs -> 4580 XUVs
C:0 T:2 107397 XUs and 103882 XVs -> 13007 XUVs
C:0 T:3 46239 XUs and 40725 XVs -> 536 XUVs
C:0 T:4 14537 XUs and 11398 XVs -> 126 XUVs
C:0 T:5 92643 XUs and 37235 XVs -> 7488 XUVs
C:0 T:6 29900 XUs and 39986 XVs -> 1014 XUVs
C:0 T:7 14155 XUs and 19470 XVs -> 90 XUVs
C:0 T:8 8921 XUs and 7442 XVs -> 98 XUVs
C:0 T:9 53737 XUs and 56000 XVs -> 3491 XUVs
C:0 T:10 23153 XUs and 26977 XVs -> 3013 XUVs
C:0 T:11 282 XUs and 171 XVs -> 32 XUVs
33731 XUVs total
9371 collection wire objects
30244 potential space points
Neighbour search...
4346162 tests to find 1947834 neighbours
Iterating with no regularization...
Begin: 6.34234e+07
0 5.32003e+07
1 5.26808e+07
2 5.26111e+07
3 5.2589e+07
Now with regularization...
Begin: 4.46159e+07
0 4.45939e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408442995
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408442995
fLifetime: 30000 [us]
04-Aug-2025 02:35:06 BST  Closed output file "reco_stage1.root"
04-Aug-2025 02:35:06 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          708.368       917.908        1218         871.038       171.842        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00119628    0.00469225    0.00719668    0.00463637    0.00182766       10     
reco:ophit:OpHitFinder                              0.29579      0.509452      0.623798      0.546842      0.0959978       10     
reco:opflash:OpFlashFinder                          0.18209      0.280147       0.38841      0.288462      0.0599386       10     
reco:opslicer:OpSlicer                             0.639558       1.27675       1.71219       1.34352      0.309914        10     
reco:caldata:DataPrepModule                         6.75576       12.3208       13.7513       12.897        1.90717        10     
reco:wclsdatasp:WireCellToolkit                     53.0307       89.2729       100.456       92.097        12.3973        10     
reco:gaushit:GausHitFinder                          1.92868       2.83716       3.9632        2.74427      0.561408        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000923655   0.00113597    0.00140851    0.00109954    0.000167442      10     
reco:reco3d:SpacePointSolver                        10.0808       31.3248       64.9146       28.9792       15.5057        10     
reco:hitpdune:DisambigFromSpacePoints               1.7134        4.62564       6.89184       4.98673       1.53911        10     
reco:pandora:StandardPandora                        52.2696       88.1072       143.136       82.7521       27.6854        10     
reco:pandoraWriter:StandardPandora                 0.561027      0.999784       1.3397        1.02122      0.199826        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.58234       15.3971       21.3387       16.1096       3.54907        10     
reco:pandoraShower:LArPandoraShowerCreation         9.41194       17.3478       25.7424       16.7345       5.51166        10     
reco:pandoracalo:Calorimetry                        4.13585       7.72023       10.5715       7.88555       2.11096        10     
reco:pandoracalonosce:Calorimetry                   3.98923       7.20233       10.4046       7.09474       1.9482         10     
reco:pandorapid:Chi2ParticleID                    0.00823656     0.0130257     0.0164768     0.0131842    0.00211991       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0638341     0.132644      0.197729      0.136815      0.037302        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00646738     0.0124375     0.0151759     0.0127409    0.00234923       10     
reco:pandoraShowercalo:ShowerCalorimetry            8.20635       18.0921       29.3325       17.3925       7.16366        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       8.09808       17.6356       28.1584       16.6858       7.02299        10     
reco:emtrkmichelid:EmTrackMichelId                  337.202       433.588       544.346       413.566       64.3206        10     
reco:crttag:SingleCRTMatchingProducer              0.0337924     0.0450218     0.0525741     0.0469342    0.00674381       10     
reco:crtreco:TwoCRTMatchingProducer                0.0293652     0.0377656     0.0448116     0.038357     0.00508134       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.88483       2.56976       3.34759       2.47454      0.429076        10     
reco:pandora2Track:LArPandoraTrackCreation          29.6319        52.57        76.2379       49.6888       13.5389        10     
reco:pandora2calo:Calorimetry                       12.9655       24.6408       35.609        23.4007       7.1868         10     
reco:pandora2calonosce:Calorimetry                  12.5347       23.9002       36.5644       22.7637       7.5128         10     
reco:pandora2pid:Chi2ParticleID                    0.0249445     0.0310146     0.039518      0.0300733    0.00463687       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.109138      0.166667      0.230491      0.169721      0.0303755       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0237977     0.0316606     0.0400178     0.032336     0.00532245       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.8032        17.4286       24.9322       17.6576       3.96282        10     
reco:pandora2Showercalo:ShowerCalorimetry           11.5955       22.0027       34.0077       20.4136       6.87317        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.57017       19.9253       30.5859       18.2593       6.12535        10     
[art]:TriggerResults:TriggerResultInserter         8.167e-05    0.000128072   0.000238098   0.000118484   4.29632e-05      10     
end_path:out1:RootOutput                          1.1978e-05    1.62222e-05   2.2464e-05    1.4971e-05    3.45664e-06      10     
end_path:out1:RootOutput(write)                     4.17244       5.60127       7.16124       5.39329      0.862254        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4780.59 MB
  Peak resident set size usage (VmHWM): 3978.05 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.48 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   7.88 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.43 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9337864
-rw-r--r-- 1 pildune32 pildune 2230171078 Aug  3 23:58 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2121641934 Aug  3 23:56 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1888389371 Aug  4 02:35 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1656791865 Aug  3 21:10 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1595994554 Aug  3 20:44 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   68368540 Aug  4 02:25 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242729 Aug  4 02:35 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Aug  4 02:35 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Aug  4 02:35 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Aug  4 02:35 time.db
-rw-r--r-- 1 pildune32 pildune        519 Aug  3 23:58 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Aug  3 21:10 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Aug  4 02:35 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Aug  3 20:33 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Aug  4 02:35 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Aug  4 00:09 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Aug  3 23:58 debugprod.log
justIN time: 2025-08-04 14:14:33 UTC       justIN version: 01.04.00