justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 271830.0@dunegpschedd02.fnal.gov

Jobsub ID271830.0@dunegpschedd02.fnal.gov
Workflow ID11881
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-14 06:32:15
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2026-01-14 13:32:12
From worker nodeHostnamecn534.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-14 06:33:16
Input filesmonte-carlo-011881-000001
JobscriptExit code0
Real time6h (24907s)
CPU time6h (24630s = 98%)
Max RSS bytes4944175104 (4715 MiB)
Outputting started2026-01-14 13:28:23
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11881/1/001/reco_stage1.root
Finished2026-01-14 13:32:12
Saved logsjustin-logs:271830.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123275 traces, 0 summary
Retagger: tagging trace set: gauss with 90522 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90522 traces tagged "gauss"
FrameSaver: q=3.16636e+06 n=1594284 tag=gauss
wclsFrameSaver: saving 123275 traces tagged "wiener"
FrameSaver: q=3.3616e+06 n=1396958 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 54021 XUs and 49907 XVs -> 1158 XUVs
C:0 T:1 125131 XUs and 107699 XVs -> 9326 XUVs
C:0 T:2 63084 XUs and 83838 XVs -> 6172 XUVs
C:0 T:3 29155 XUs and 31896 XVs -> 237 XUVs
C:0 T:4 9196 XUs and 10866 XVs -> 170 XUVs
C:0 T:5 37223 XUs and 36336 XVs -> 5612 XUVs
C:0 T:6 72128 XUs and 95642 XVs -> 5770 XUVs
C:0 T:7 29803 XUs and 31454 XVs -> 354 XUVs
C:0 T:8 7302 XUs and 9659 XVs -> 66 XUVs
C:0 T:9 59484 XUs and 108485 XVs -> 10453 XUVs
C:0 T:10 24610 XUs and 22835 XVs -> 4983 XUVs
C:0 T:11 107 XUs and 88 XVs -> 21 XUVs
44322 XUVs total
11053 collection wire objects
40217 potential space points
Neighbour search...
4846197 tests to find 2389284 neighbours
Iterating with no regularization...
Begin: 5.82235e+07
0 4.7722e+07
1 4.68717e+07
2 4.67305e+07
3 4.66853e+07
Now with regularization...
Begin: 3.77097e+07
0 3.76489e+07
1 3.76078e+07
2 3.75777e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411731008
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411731008
fLifetime: 30000 [us]
14-Jan-2026 13:28:20 GMT  Closed output file "reco_stage1.root"
14-Jan-2026 13:28:20 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          547.117       1306.8        5024.46       988.531       1253.88        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00124639    0.00464594    0.00745259    0.00528312    0.00191191       10     
reco:ophit:OpHitFinder                             0.362679      0.541052      0.701476       0.5567       0.0971801       10     
reco:opflash:OpFlashFinder                         0.268883      0.402651      0.486403      0.420541      0.0608131       10     
reco:opslicer:OpSlicer                              0.8696        1.46994       1.81098       1.54069      0.259996        10     
reco:caldata:DataPrepModule                         6.39465       9.1496        10.5906       9.97764       1.61437        10     
reco:wclsdatasp:WireCellToolkit                     53.3474       67.9742       77.4736       67.0742       8.4258         10     
reco:gaushit:GausHitFinder                          1.47452       2.99232       4.75547       3.23202      0.900196        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000732164   0.00117434    0.00188348    0.00117121    0.000283721      10     
reco:reco3d:SpacePointSolver                        13.536        45.1899       136.14        38.3973       32.5222        10     
reco:hitpdune:DisambigFromSpacePoints               2.19952       5.99021       14.4721       4.71412       3.47058        10     
reco:pandora:StandardPandora                        30.295        448.967       3546.18       111.732       1033.23        10     
reco:pandoraWriter:StandardPandora                 0.502264       1.00832       1.69112       1.05638      0.330612        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.53995       24.2843       72.8125       20.612        17.3269        10     
reco:pandoraShower:LArPandoraShowerCreation         7.4836        24.1771       52.5282       25.6262       11.947         10     
reco:pandoracalo:Calorimetry                        3.59732       11.0932       33.0764       9.23494       7.85668        10     
reco:pandoracalonosce:Calorimetry                   3.48686       10.6357       31.9406       8.98552       7.57705        10     
reco:pandorapid:Chi2ParticleID                    0.00769804     0.0140623     0.0256516     0.0139153    0.00480225       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0625417     0.116725      0.173068      0.125891      0.0298771       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00727075     0.0136927     0.0256086     0.0137029    0.00497738       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.34841       23.1452       52.5941       24.2587       11.8448        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.37793       22.3383       48.8438       23.2916       10.8077        10     
reco:emtrkmichelid:EmTrackMichelId                  261.741       418.088       560.684       423.005       81.9648        10     
reco:crttag:SingleCRTMatchingProducer              0.0296707     0.0377716     0.0495882     0.0359066    0.00673031       10     
reco:crtreco:TwoCRTMatchingProducer                0.0180928     0.0285959     0.0408131     0.028651     0.00693133       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.28157       1.88242       2.5391        1.82265      0.398588        10     
reco:pandora2Track:LArPandoraTrackCreation          26.0902       64.1558       105.612       70.6925       24.2424        10     
reco:pandora2calo:Calorimetry                       10.2336       28.7736       54.6362       30.2044       12.1566        10     
reco:pandora2calonosce:Calorimetry                  10.1961       26.6428       58.3784       27.3028       13.1575        10     
reco:pandora2pid:Chi2ParticleID                    0.016803      0.0337289     0.0574378     0.0344176     0.0127568       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0780137     0.157325      0.263723      0.158937      0.0561339       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0163085     0.0330875     0.0559623     0.0340736     0.0122918       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.51169       17.5744       38.7904       18.1396       8.66276        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.79323       23.1033       56.2606       23.6311       12.9287        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.6406        20.6849       53.4775       20.267        12.242         10     
[art]:TriggerResults:TriggerResultInserter        5.8496e-05    8.16098e-05   0.000105592   8.1559e-05    1.12378e-05      10     
end_path:out1:RootOutput                          1.0264e-05    1.17998e-05    1.332e-05    1.19605e-05   8.86512e-07      10     
end_path:out1:RootOutput(write)                     2.72678       5.89357       9.06162       5.91975       1.42456        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5749.31 MB
  Peak resident set size usage (VmHWM): 4944.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.76 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12618784
-rw-r--r-- 1 pildune32 pildune 2899692078 Jan 14 09:46 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2750187406 Jan 14 09:44 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2680123036 Jan 14 13:28 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2290016527 Jan 14 07:13 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 2201467356 Jan 14 06:44 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   99536490 Jan 14 13:21 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243584 Jan 14 13:28 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Jan 14 13:28 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Jan 14 13:28 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Jan 14 13:28 time.db
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 09:46 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 07:13 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Jan 14 13:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Jan 14 06:33 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Jan 14 13:28 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Jan 14 09:56 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Jan 14 09:47 debugprod.log
justIN time: 2026-02-06 03:24:59 UTC       justIN version: 01.06.00