justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 237619.1@dunegpschedd02.fnal.gov

Jobsub ID237619.1@dunegpschedd02.fnal.gov
Workflow ID9604
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-05 04:30:55
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-11-05 09:47:09
From worker nodeHostnamecn539.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-05 04:32:12
Input filesmonte-carlo-009604-000001
JobscriptExit code0
Real time5h (18681s)
CPU time5h (18434s = 98%)
Max RSS bytes4341358592 (4140 MiB)
Outputting started2025-11-05 09:43:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09604/1/001/reco_stage1.root
Finished2025-11-05 09:47:09
Saved logsjustin-logs:237619.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

th 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 134079 traces, 0 summary
Retagger: tagging trace set: gauss with 99448 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 99448 traces tagged "gauss"
FrameSaver: q=3.43932e+06 n=1747129 tag=gauss
wclsFrameSaver: saving 134079 traces tagged "wiener"
FrameSaver: q=3.64651e+06 n=1531981 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44232 XUs and 42793 XVs -> 518 XUVs
C:0 T:1 78860 XUs and 72419 XVs -> 4638 XUVs
C:0 T:2 103482 XUs and 117491 XVs -> 7542 XUVs
C:0 T:3 52692 XUs and 42481 XVs -> 215 XUVs
C:0 T:4 10644 XUs and 10994 XVs -> 68 XUVs
C:0 T:5 42283 XUs and 34904 XVs -> 3373 XUVs
C:0 T:6 118781 XUs and 75714 XVs -> 5494 XUVs
C:0 T:7 39424 XUs and 25270 XVs -> 388 XUVs
C:0 T:8 7537 XUs and 6429 XVs -> 82 XUVs
C:0 T:9 44177 XUs and 62602 XVs -> 4060 XUVs
C:0 T:10 60697 XUs and 33689 XVs -> 5950 XUVs
C:0 T:11 118 XUs and 87 XVs -> 16 XUVs
32344 XUVs total
12616 collection wire objects
29302 potential space points
Neighbour search...
1545368 tests to find 840450 neighbours
Iterating with no regularization...
Begin: 7.47669e+07
0 6.23752e+07
1 6.15304e+07
2 6.14015e+07
3 6.13617e+07
Now with regularization...
Begin: 5.1427e+07
0 5.13874e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410321257
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410321257
fLifetime: 30000 [us]
05-Nov-2025 09:43:31 GMT  Closed output file "reco_stage1.root"
05-Nov-2025 09:43:31 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          540.826       850.274       1365.87       741.576       241.232        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00142304    0.00510921     0.0105123    0.00515637    0.00252684       10     
reco:ophit:OpHitFinder                             0.367487      0.466987      0.651124       0.45846      0.0787557       10     
reco:opflash:OpFlashFinder                         0.262372      0.360401      0.539914       0.35235      0.0758468       10     
reco:opslicer:OpSlicer                             0.817791       1.16698       1.57535       1.08502      0.261697        10     
reco:caldata:DataPrepModule                         6.95256       9.31158       11.396        9.45302       1.37085        10     
reco:wclsdatasp:WireCellToolkit                     57.2535       69.8134       76.8595       71.1957       4.99174        10     
reco:gaushit:GausHitFinder                          1.68831       2.31758       3.57386       2.12684      0.554819        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000810507   0.00105622    0.00159938    0.00100009    0.000239499      10     
reco:reco3d:SpacePointSolver                        12.5191       27.1723       56.5183       25.3512       11.9457        10     
reco:hitpdune:DisambigFromSpacePoints               2.33379       4.45011       11.307        3.63396       2.53526        10     
reco:pandora:StandardPandora                        36.7355       93.7609       199.341       75.195        56.5324        10     
reco:pandoraWriter:StandardPandora                 0.586564      0.889639       1.33873      0.884551      0.212222        10     
reco:pandoraTrack:LArPandoraTrackCreation           11.1384       18.6005       35.7159       17.5293       6.74257        10     
reco:pandoraShower:LArPandoraShowerCreation         9.95544       20.3166       39.4578       17.8369       8.19361        10     
reco:pandoracalo:Calorimetry                        5.04399       8.85632       16.5471       8.39457       3.20557        10     
reco:pandoracalonosce:Calorimetry                   4.85274       8.43828       15.7634       7.51361       3.14206        10     
reco:pandorapid:Chi2ParticleID                    0.00737048     0.0117382     0.0183255     0.0118527    0.00341362       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0717342     0.111532      0.165928      0.101038      0.0297832       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0066475     0.0107779     0.0170995     0.0104415    0.00292308       10     
reco:pandoraShowercalo:ShowerCalorimetry            9.15416       20.2612       38.1334       16.2017       9.07628        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       8.72596       20.3465       43.435        15.8838       10.4183        10     
reco:emtrkmichelid:EmTrackMichelId                  271.909       357.793       509.234       334.619       75.6468        10     
reco:crttag:SingleCRTMatchingProducer              0.0335969     0.0459598     0.0600277     0.0461443    0.00963448       10     
reco:crtreco:TwoCRTMatchingProducer                0.0248714     0.0367635     0.0499471     0.0350085    0.00904575       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.49474       2.10862       3.03319       2.03825      0.466483        10     
reco:pandora2Track:LArPandoraTrackCreation          30.5814       60.1112       115.436        51.69        23.249         10     
reco:pandora2calo:Calorimetry                       13.0055       29.1316       53.4703       24.0878       11.9161        10     
reco:pandora2calonosce:Calorimetry                  12.0735       26.8215       41.0536       22.9017       9.55384        10     
reco:pandora2pid:Chi2ParticleID                    0.0158554     0.0278704     0.0423237     0.0272494    0.00720786       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.089545       0.14462      0.236295      0.134927      0.0396593       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0156746     0.0250622     0.0335636     0.024781     0.00511716       10     
reco:pandora2Shower:LArPandoraShowerCreation        9.41273       18.7991       28.9167       16.7513       5.94623        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.83267       23.3267       34.0737       22.2934       7.08545        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.45376        20.46        30.563        19.5025       6.52745        10     
[art]:TriggerResults:TriggerResultInserter        6.7949e-05    9.27582e-05   0.000166104   8.62995e-05   2.82195e-05      10     
end_path:out1:RootOutput                          1.1098e-05    1.28187e-05   1.8051e-05    1.2111e-05    2.00657e-06      10     
end_path:out1:RootOutput(write)                     3.13462       4.53662       6.07584       4.44375      0.908054        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5145.46 MB
  Peak resident set size usage (VmHWM): 4341.36 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.90 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10610484
-rw-r--r-- 1 pildune32 pildune 2470244665 Nov  5 07:18 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2339914403 Nov  5 07:15 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2182405007 Nov  5 09:43 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1933170715 Nov  5 05:04 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1857820134 Nov  5 04:40 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   80964860 Nov  5 09:34 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     243291 Nov  5 09:43 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Nov  5 09:43 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Nov  5 09:43 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Nov  5 09:43 time.db
-rw-r--r-- 1 pildune32 pildune        519 Nov  5 07:18 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov  5 05:04 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov  5 09:43 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Nov  5 04:32 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Nov  5 09:43 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Nov  5 07:27 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Nov  5 07:18 debugprod.log
justIN time: 2025-11-05 11:10:52 UTC       justIN version: 01.05.01