justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 14219.1@dunegpschedd02.fnal.gov

Jobsub ID14219.1@dunegpschedd02.fnal.gov
Workflow ID278
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-08-03 19:32:55
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-08-04 00:59:45
From worker nodeHostnamecn515.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-08-03 19:34:52
Input filesmonte-carlo-000278-000001
JobscriptExit code0
Real time5h (19282s)
CPU time5h (18945s = 98%)
Max RSS bytes3946377216 (3763 MiB)
Outputting started2025-08-04 00:56:15
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/00278/1/001/reco_stage1.root
Finished2025-08-04 00:59:45
Saved logsjustin-logs:14219.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

sedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 138127 traces, 0 summary
Retagger: tagging trace set: gauss with 104693 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104693 traces tagged "gauss"
FrameSaver: q=3.59978e+06 n=1851049 tag=gauss
wclsFrameSaver: saving 138127 traces tagged "wiener"
FrameSaver: q=3.82479e+06 n=1638000 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39753 XUs and 36722 XVs -> 744 XUVs
C:0 T:1 81258 XUs and 122120 XVs -> 11336 XUVs
C:0 T:2 137754 XUs and 148550 XVs -> 10202 XUVs
C:0 T:3 56580 XUs and 51090 XVs -> 389 XUVs
C:0 T:4 10072 XUs and 10950 XVs -> 176 XUVs
C:0 T:5 34751 XUs and 26053 XVs -> 2601 XUVs
C:0 T:6 80141 XUs and 70248 XVs -> 3467 XUVs
C:0 T:7 32821 XUs and 25309 XVs -> 227 XUVs
C:0 T:8 8033 XUs and 8731 XVs -> 271 XUVs
C:0 T:9 127459 XUs and 106510 XVs -> 17378 XUVs
C:0 T:10 50158 XUs and 26214 XVs -> 5670 XUVs
C:0 T:11 97 XUs and 71 XVs -> 10 XUVs
52471 XUVs total
12817 collection wire objects
47356 potential space points
Neighbour search...
6389894 tests to find 2988264 neighbours
Iterating with no regularization...
Begin: 6.49546e+07
0 5.25111e+07
1 5.16207e+07
2 5.14704e+07
3 5.14206e+07
Now with regularization...
Begin: 4.14066e+07
0 4.13294e+07
1 4.12757e+07
2 4.12374e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 408442953
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 408442953
fLifetime: 30000 [us]
04-Aug-2025 01:56:12 BST  Closed output file "reco_stage1.root"
04-Aug-2025 01:56:12 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          350.616       701.732       905.051       744.451       162.352        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00191852    0.00396072    0.00537176    0.00411185    0.00108245       10     
reco:ophit:OpHitFinder                             0.282708      0.438678      0.604793      0.481188      0.128249        10     
reco:opflash:OpFlashFinder                          0.16652      0.258464       0.36593       0.27824      0.0636375       10     
reco:opslicer:OpSlicer                             0.531449       1.03962       1.71465       1.0487       0.361333        10     
reco:caldata:DataPrepModule                         7.14014       10.1956       13.2673       11.6021       2.44655        10     
reco:wclsdatasp:WireCellToolkit                     55.9273       74.3516       96.7508       77.0419       13.7932        10     
reco:gaushit:GausHitFinder                          1.22778       2.25373       3.21787       2.2965       0.586175        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000678542   0.000940561   0.00151561    0.000931725   0.000220024      10     
reco:reco3d:SpacePointSolver                        12.2229       19.9164       32.4351       21.4497       5.68077        10     
reco:hitpdune:DisambigFromSpacePoints               1.42074       3.52239       5.05125       3.53839       1.08681        10     
reco:pandora:StandardPandora                        20.3447       57.9633       94.2642       61.2841       19.6587        10     
reco:pandoraWriter:StandardPandora                 0.445753      0.792463      0.948493      0.840457      0.152795        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.68381       13.6641       18.0493       14.1157       3.94138        10     
reco:pandoraShower:LArPandoraShowerCreation         5.56663       12.332        17.8104       13.4049       3.85149        10     
reco:pandoracalo:Calorimetry                        2.96194       6.45967       8.68624       6.78215       2.01158        10     
reco:pandoracalonosce:Calorimetry                   2.72643       6.25949       8.58605       6.55872       2.06775        10     
reco:pandorapid:Chi2ParticleID                    0.00600333     0.0118469     0.0148918     0.0130442    0.00299831       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.057258      0.109535      0.165288      0.111746      0.0274444       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00613914     0.0112308     0.0146838     0.0116548    0.00232872       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.25948       12.3083       18.8404       12.716        4.80041        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.25789       12.0247       18.7598       12.5094       4.77667        10     
reco:emtrkmichelid:EmTrackMichelId                  169.778       349.145       438.857       359.042       79.1005        10     
reco:crttag:SingleCRTMatchingProducer              0.0228141     0.0387405     0.0561193     0.038786      0.0108809       10     
reco:crtreco:TwoCRTMatchingProducer                0.0194795     0.0323246     0.0456268     0.0325031    0.00830587       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.24653       2.08052       3.0642        1.84447      0.592805        10     
reco:pandora2Track:LArPandoraTrackCreation          20.1332       38.1974       56.8419       38.2179       12.8155        10     
reco:pandora2calo:Calorimetry                       6.88236       17.0955       27.887        16.9816       6.83159        10     
reco:pandora2calonosce:Calorimetry                  6.11612       16.215        26.3734       16.0887       6.65534        10     
reco:pandora2pid:Chi2ParticleID                    0.0112507     0.0247344     0.0372516     0.0260011    0.00791043       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0624536     0.132422      0.198489      0.137303      0.0421281       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0111017     0.0241925     0.0336829     0.0252347    0.00734471       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.19122       12.6271       20.804        12.8366       4.56385        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.69372       14.6568       25.5172       14.6296       6.14096        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.51114       13.0761        23.8         12.8133       5.67015        10     
[art]:TriggerResults:TriggerResultInserter        5.8969e-05    0.000101282   0.000206411   9.18155e-05   4.15648e-05      10     
end_path:out1:RootOutput                           9.444e-06    1.45364e-05   2.0975e-05    1.3212e-05    4.0465e-06       10     
end_path:out1:RootOutput(write)                     2.26669       4.27072       5.78109       4.3655        1.17118        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4750.5 MB
  Peak resident set size usage (VmHWM): 3946.38 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.37 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.37 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9247772
-rw-r--r-- 1 pildune22 pildune 2205079655 Aug  3 23:55 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2097931520 Aug  3 23:51 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 1846418049 Aug  4 01:56 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1660338581 Aug  3 21:11 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1594870721 Aug  3 20:45 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   64464640 Aug  4 01:44 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242820 Aug  4 01:56 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Aug  4 01:56 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Aug  4 01:56 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Aug  4 01:56 time.db
-rw-r--r-- 1 pildune22 pildune        519 Aug  3 23:55 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Aug  3 21:11 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Aug  4 01:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Aug  3 20:34 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Aug  4 01:56 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Aug  4 00:06 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Aug  3 23:55 debugprod.log
justIN time: 2025-08-04 21:36:35 UTC       justIN version: 01.04.00