justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 231355.0@dunegpschedd02.fnal.gov

Jobsub ID231355.0@dunegpschedd02.fnal.gov
Workflow ID9201
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-20 16:20:01
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-10-20 20:49:23
From worker nodeHostnamecn101.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-20 16:20:36
Input filesmonte-carlo-009201-000001
JobscriptExit code0
Real time4h (15905s)
CPU time4h (15479s = 97%)
Max RSS bytes5724700672 (5459 MiB)
Outputting started2025-10-20 20:45:43
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09201/1/001/reco_stage1.root
Finished2025-10-20 20:49:23
Saved logsjustin-logs:231355.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ol pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123538 traces, 0 summary
Retagger: tagging trace set: gauss with 94321 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 94321 traces tagged "gauss"
FrameSaver: q=3.34291e+06 n=1676416 tag=gauss
wclsFrameSaver: saving 123538 traces tagged "wiener"
FrameSaver: q=3.5297e+06 n=1491137 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 22058 XUs and 22963 XVs -> 219 XUVs
C:0 T:1 38867 XUs and 35686 XVs -> 2470 XUVs
C:0 T:2 79363 XUs and 93170 XVs -> 10179 XUVs
C:0 T:3 25560 XUs and 35297 XVs -> 413 XUVs
C:0 T:4 12195 XUs and 10894 XVs -> 169 XUVs
C:0 T:5 62615 XUs and 112177 XVs -> 13657 XUVs
C:0 T:6 61828 XUs and 74348 XVs -> 3053 XUVs
C:0 T:7 26703 XUs and 30993 XVs -> 174 XUVs
C:0 T:8 13350 XUs and 10775 XVs -> 155 XUVs
C:0 T:9 73888 XUs and 78949 XVs -> 5066 XUVs
C:0 T:10 35285 XUs and 28897 XVs -> 4551 XUVs
C:0 T:11 439 XUs and 403 XVs -> 57 XUVs
40163 XUVs total
9758 collection wire objects
37162 potential space points
Neighbour search...
5650592 tests to find 2487046 neighbours
Iterating with no regularization...
Begin: 6.51936e+07
0 5.66623e+07
1 5.60416e+07
2 5.59515e+07
3 5.5927e+07
Now with regularization...
Begin: 4.68534e+07
0 4.68389e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410009301
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410009301
fLifetime: 30000 [us]
20-Oct-2025 21:45:40 BST  Closed output file "reco_stage1.root"
20-Oct-2025 21:45:40 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          436.966       662.456       908.858       672.441       141.901        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000998807   0.00440478    0.00632467    0.00478215    0.00175666       10     
reco:ophit:OpHitFinder                             0.236035      0.309323      0.433776      0.286897      0.0616119       10     
reco:opflash:OpFlashFinder                         0.193066      0.268564      0.392315      0.245675      0.064093        10     
reco:opslicer:OpSlicer                             0.575498      0.754647       1.02576      0.709787      0.153678        10     
reco:caldata:DataPrepModule                         4.9082        5.98163       8.59214       5.25208       1.30301        10     
reco:wclsdatasp:WireCellToolkit                     43.5438       50.0629       57.5218       50.142        4.30214        10     
reco:gaushit:GausHitFinder                          1.22153       1.55099       2.46136       1.50233      0.335119        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000730899   0.000992961   0.00132006    0.00101579    0.000162917      10     
reco:reco3d:SpacePointSolver                        10.1964       18.6607       30.4899       16.445        6.34883        10     
reco:hitpdune:DisambigFromSpacePoints               1.68428       2.72697       4.1959        2.51043      0.808042        10     
reco:pandora:StandardPandora                        28.1448       63.5182       82.5565       70.5581       18.5728        10     
reco:pandoraWriter:StandardPandora                 0.436202      0.709357       1.09631      0.675543       0.20711        10     
reco:pandoraTrack:LArPandoraTrackCreation           9.79816       17.5931       35.3297       16.9754       6.65931        10     
reco:pandoraShower:LArPandoraShowerCreation         10.863        20.1397       33.3913       20.6221       6.0547         10     
reco:pandoracalo:Calorimetry                        4.62665       8.74046       18.1057       7.99951       3.63109        10     
reco:pandoracalonosce:Calorimetry                   4.28554       8.31594       17.0546       7.66913       3.42054        10     
reco:pandorapid:Chi2ParticleID                    0.00471798    0.00829393     0.0137245    0.00664709    0.00308399       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0524057     0.0796262     0.118395      0.0715256     0.021978        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00470668    0.00776479     0.0131727    0.00634177    0.00276633       10     
reco:pandoraShowercalo:ShowerCalorimetry            10.7399       20.9655       33.0664       21.6765       6.17375        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       10.0871       20.1745       26.9768       21.342        5.38038        10     
reco:emtrkmichelid:EmTrackMichelId                  156.006       247.973       313.648       255.072       51.897         10     
reco:crttag:SingleCRTMatchingProducer              0.0325269     0.0401626     0.0474844     0.0398282    0.00512882       10     
reco:crtreco:TwoCRTMatchingProducer                0.0266966     0.0333797     0.040197      0.032792     0.00420039       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.935609       1.37398       2.4505        1.26336      0.437156        10     
reco:pandora2Track:LArPandoraTrackCreation          32.4795       55.0371       90.1379       55.4527       14.8845        10     
reco:pandora2calo:Calorimetry                       17.2209       27.6851       47.8501       24.7869       8.52508        10     
reco:pandora2calonosce:Calorimetry                  17.3406       26.3737       45.7343       23.978        7.96671        10     
reco:pandora2pid:Chi2ParticleID                    0.0140232     0.0195692     0.0315218     0.0168789    0.00617662       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0811937      0.11028      0.175784      0.096737      0.0332697       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.013285      0.0193297     0.029535      0.0159803    0.00605362       10     
reco:pandora2Shower:LArPandoraShowerCreation        12.6274       17.5786       27.7543       17.1049       4.43535        10     
reco:pandora2Showercalo:ShowerCalorimetry           12.8448       22.0399       31.9197       21.6701       5.73802        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      11.6976       20.0626       27.773        20.436        4.96106        10     
[art]:TriggerResults:TriggerResultInserter        4.3714e-05    6.6783e-05    0.000130081   5.4399e-05    2.62767e-05      10     
end_path:out1:RootOutput                           6.082e-06    8.3164e-06    1.3579e-05    7.4055e-06    2.15565e-06      10     
end_path:out1:RootOutput(write)                     2.2458        3.32157       4.69302       3.17304      0.789074        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5897 MB
  Peak resident set size usage (VmHWM): 5094.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.78 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 13003868
-rw-r--r-- 1 pildune32 pildune 2927386826 Oct 20 19:52 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2787701360 Oct 20 19:49 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 2658446518 Oct 20 21:45 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 2499225513 Oct 20 17:57 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 2357002411 Oct 20 17:28 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   85577160 Oct 20 21:39 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242974 Oct 20 21:45 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Oct 20 21:45 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Oct 20 21:45 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Oct 20 21:45 time.db
-rw-r--r-- 1 pildune32 pildune        519 Oct 20 19:52 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 20 17:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Oct 20 21:45 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Oct 20 17:20 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Oct 20 21:45 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Oct 20 20:01 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Oct 20 19:52 debugprod.log
justIN time: 2025-11-04 01:53:52 UTC       justIN version: 01.05.01