justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 268246.1@dunegpschedd01.fnal.gov

Jobsub ID268246.1@dunegpschedd01.fnal.gov
Workflow ID11215
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-14 09:55:42
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-12-14 13:31:16
From worker nodeHostnamecn068.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-14 09:56:50
Input filesmonte-carlo-011215-000001
JobscriptExit code0
Real time3h (12659s)
CPU time3h (12453s = 98%)
Max RSS bytes4267368448 (4069 MiB)
Outputting started2025-12-14 13:27:50
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11215/1/001/reco_stage1.root
Finished2025-12-14 13:31:16
Saved logsjustin-logs:268246.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150397 traces, 0 summary
Retagger: tagging trace set: gauss with 114303 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 114303 traces tagged "gauss"
FrameSaver: q=4.13947e+06 n=2041812 tag=gauss
wclsFrameSaver: saving 150397 traces tagged "wiener"
FrameSaver: q=4.38363e+06 n=1818466 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38365 XUs and 31689 XVs -> 796 XUVs
C:0 T:1 158694 XUs and 91813 XVs -> 24940 XUVs
C:0 T:2 73434 XUs and 84096 XVs -> 3927 XUVs
C:0 T:3 32579 XUs and 37038 XVs -> 186 XUVs
C:0 T:4 18922 XUs and 15207 XVs -> 176 XUVs
C:0 T:5 74136 XUs and 43434 XVs -> 5091 XUVs
C:0 T:6 163652 XUs and 177991 XVs -> 8770 XUVs
C:0 T:7 64867 XUs and 68803 XVs -> 1108 XUVs
C:0 T:8 9947 XUs and 10211 XVs -> 131 XUVs
C:0 T:9 67753 XUs and 83347 XVs -> 4863 XUVs
C:0 T:10 21829 XUs and 17862 XVs -> 4089 XUVs
C:0 T:11 282 XUs and 257 XVs -> 57 XUVs
54134 XUVs total
12625 collection wire objects
49138 potential space points
Neighbour search...
10146088 tests to find 4031920 neighbours
Iterating with no regularization...
Begin: 8.98523e+07
0 7.45136e+07
1 7.35826e+07
2 7.34325e+07
3 7.33809e+07
Now with regularization...
Begin: 6.11469e+07
0 6.10125e+07
1 6.09413e+07
2 6.08958e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411110263
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411110263
fLifetime: 30000 [us]
14-Dec-2025 13:27:47 GMT  Closed output file "reco_stage1.root"
14-Dec-2025 13:27:47 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          420.103       596.23        823.29        593.614       124.338        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00097526    0.00403187    0.00663508    0.00427477     0.0017045       10     
reco:ophit:OpHitFinder                             0.222622      0.309854      0.434298      0.268209      0.0728832       10     
reco:opflash:OpFlashFinder                         0.171162      0.229712      0.331328       0.21963      0.0394667       10     
reco:opslicer:OpSlicer                             0.373314      0.776781       1.45636      0.688344      0.301457        10     
reco:caldata:DataPrepModule                         5.05372       6.42254       8.94314       5.75881       1.42868        10     
reco:wclsdatasp:WireCellToolkit                     44.6309       52.9896       68.5652       50.4476       7.86893        10     
reco:gaushit:GausHitFinder                          1.09417       1.58317       2.61292       1.44942      0.428599        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000638003   0.000939328   0.00124409    0.000907207   0.000161596      10     
reco:reco3d:SpacePointSolver                        9.61849       19.8708       31.3117       20.5335       6.07934        10     
reco:hitpdune:DisambigFromSpacePoints               1.30357       2.70375       5.05723       2.48903       1.00816        10     
reco:pandora:StandardPandora                        32.8823       56.5464       112.592       46.8335       24.3984        10     
reco:pandoraWriter:StandardPandora                 0.374584       0.58213      0.842971      0.562755       0.12793        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.26665       16.2525       28.2628       15.5946       4.49103        10     
reco:pandoraShower:LArPandoraShowerCreation         8.80123       17.8738       25.4854       17.2013       5.0972         10     
reco:pandoracalo:Calorimetry                        4.23515       7.25876       13.4901       6.88997       2.25562        10     
reco:pandoracalonosce:Calorimetry                   3.92575       7.06477       11.6832       6.69966       1.89991        10     
reco:pandorapid:Chi2ParticleID                    0.00597723    0.00789785     0.012828     0.00699829    0.00215645       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0545631     0.0775809     0.111201      0.0723848     0.019036        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00571628    0.00786482     0.0165392    0.00620695    0.00316325       10     
reco:pandoraShowercalo:ShowerCalorimetry            8.83929       17.2448       24.3337       16.2461       5.36352        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.48666       16.677        22.9826       16.0137       5.1761         10     
reco:emtrkmichelid:EmTrackMichelId                  168.948       218.869       319.145       213.876       45.3511        10     
reco:crttag:SingleCRTMatchingProducer              0.0225774     0.0383293     0.0517177     0.0360011    0.00917714       10     
reco:crtreco:TwoCRTMatchingProducer                0.0202147     0.0335576     0.0465648     0.0311381    0.00802363       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.961224       1.42556       2.03088       1.30943      0.377835        10     
reco:pandora2Track:LArPandoraTrackCreation          28.3096       51.707        86.0354       49.0991       16.4824        10     
reco:pandora2calo:Calorimetry                       10.8527       22.7797       41.2357       22.0776       8.30365        10     
reco:pandora2calonosce:Calorimetry                  10.8456       21.7736       37.913        21.3504       7.62585        10     
reco:pandora2pid:Chi2ParticleID                    0.0120614     0.0184277     0.0413949     0.0161613     0.0082578       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.065628      0.0991662     0.212292      0.0862449     0.0429209       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0116899     0.0187289     0.0392861     0.0157444    0.00855063       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.32797       15.531        22.9825       16.3308       4.09029        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.62745       19.4464       34.7676       19.3302       6.68172        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.30754       16.5231       29.2298       16.4238       5.56052        10     
[art]:TriggerResults:TriggerResultInserter        4.8083e-05    7.02159e-05   0.000109002   6.29525e-05   1.99879e-05      10     
end_path:out1:RootOutput                           8.093e-06    1.10177e-05   1.9819e-05     9.133e-06    3.60193e-06      10     
end_path:out1:RootOutput(write)                     2.20263       3.27394       5.94238       2.81603       1.08644        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5070.39 MB
  Peak resident set size usage (VmHWM): 4267.37 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.25 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.99 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10418712
-rw-r--r-- 1 pildune22 pildune 2450662066 Dec 14 11:46 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2323413101 Dec 14 11:44 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2160553917 Dec 14 13:27 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1866110444 Dec 14 10:17 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1786594090 Dec 14 10:02 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   80813610 Dec 14 13:19 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242901 Dec 14 13:27 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Dec 14 13:27 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Dec 14 13:27 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Dec 14 13:27 time.db
-rw-r--r-- 1 pildune22 pildune        519 Dec 14 11:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec 14 10:17 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec 14 13:27 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Dec 14 09:56 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Dec 14 13:27 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Dec 14 11:52 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Dec 14 11:46 debugprod.log
justIN time: 2025-12-18 20:20:55 UTC       justIN version: 01.05.03