justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 252861.0@dunegpschedd01.fnal.gov

Jobsub ID252861.0@dunegpschedd01.fnal.gov
Workflow ID10264
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-16 16:33:06
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce03
Last heartbeat2025-11-16 19:40:07
From worker nodeHostnamecn090.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-16 16:33:50
Input filesmonte-carlo-010264-000001
JobscriptExit code0
Real time3h (10975s)
CPU time3h (10801s = 98%)
Max RSS bytes4003401728 (3817 MiB)
Outputting started2025-11-16 19:36:46
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/10264/1/001/reco_stage1.root
Finished2025-11-16 19:40:07
Saved logsjustin-logs:252861.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 122327 traces, 0 summary
Retagger: tagging trace set: gauss with 92528 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 92528 traces tagged "gauss"
FrameSaver: q=3.21276e+06 n=1601954 tag=gauss
wclsFrameSaver: saving 122327 traces tagged "wiener"
FrameSaver: q=3.40792e+06 n=1408880 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 51537 XUs and 42841 XVs -> 1211 XUVs
C:0 T:1 142294 XUs and 93800 XVs -> 20692 XUVs
C:0 T:2 99747 XUs and 97432 XVs -> 6411 XUVs
C:0 T:3 40932 XUs and 36636 XVs -> 423 XUVs
C:0 T:4 5176 XUs and 4722 XVs -> 16 XUVs
C:0 T:5 13659 XUs and 8504 XVs -> 852 XUVs
C:0 T:6 112055 XUs and 107920 XVs -> 5070 XUVs
C:0 T:7 45609 XUs and 41697 XVs -> 385 XUVs
C:0 T:8 5895 XUs and 6529 XVs -> 66 XUVs
C:0 T:9 28932 XUs and 44738 XVs -> 3200 XUVs
C:0 T:10 51047 XUs and 25579 XVs -> 4070 XUVs
C:0 T:11 219 XUs and 142 XVs -> 23 XUVs
42419 XUVs total
11226 collection wire objects
38018 potential space points
Neighbour search...
6288432 tests to find 2665266 neighbours
Iterating with no regularization...
Begin: 7.43026e+07
0 6.29684e+07
1 6.21492e+07
2 6.20103e+07
3 6.19663e+07
Now with regularization...
Begin: 5.21691e+07
0 5.21238e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410552703
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410552703
fLifetime: 30000 [us]
16-Nov-2025 19:36:42 GMT  Closed output file "reco_stage1.root"
16-Nov-2025 19:36:42 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          381.451       488.152       546.568       515.275       57.136         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00117042     0.0035632    0.00588346    0.00352241    0.00144476       10     
reco:ophit:OpHitFinder                             0.213319      0.242314      0.269703      0.248148      0.0186685       10     
reco:opflash:OpFlashFinder                         0.128795      0.170022      0.199075      0.180605      0.0225145       10     
reco:opslicer:OpSlicer                             0.512449      0.587566      0.704508      0.570586      0.0571233       10     
reco:caldata:DataPrepModule                         5.06574       5.40824       5.66757        5.404       0.189989        10     
reco:wclsdatasp:WireCellToolkit                     45.1235       49.8509       53.3464       50.3826       2.48962        10     
reco:gaushit:GausHitFinder                          1.21455       1.39512       1.58397       1.40096      0.111066        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00070608    0.00089871    0.00108724    0.000884537   0.000122549      10     
reco:reco3d:SpacePointSolver                        8.72035       13.5063       20.5005       12.2787       3.55153        10     
reco:hitpdune:DisambigFromSpacePoints               1.30236       2.08781       2.80486       2.14608      0.407696        10     
reco:pandora:StandardPandora                        25.0252       40.4903       51.8472       43.0236       9.02009        10     
reco:pandoraWriter:StandardPandora                 0.417497      0.530791       0.65076       0.5456       0.0729385       10     
reco:pandoraTrack:LArPandoraTrackCreation           9.5876        11.7048       14.9594       11.3169       1.88515        10     
reco:pandoraShower:LArPandoraShowerCreation         10.287        14.9099       19.4811       14.9956       2.66813        10     
reco:pandoracalo:Calorimetry                        4.33269       5.53925       6.89854       5.58073      0.873274        10     
reco:pandoracalonosce:Calorimetry                   4.09907       5.33428       6.77135       5.37973        0.882         10     
reco:pandorapid:Chi2ParticleID                     0.005468     0.00657703     0.0104693    0.00626107    0.00135776       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.054218      0.0710402     0.106289      0.0665751     0.0185297       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00525288    0.00602061    0.00741698    0.00597704    0.000631597      10     
reco:pandoraShowercalo:ShowerCalorimetry            9.46164       14.1585       18.3093       14.4999       2.57619        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       9.16079       13.6757       17.6859       13.7597        2.492         10     
reco:emtrkmichelid:EmTrackMichelId                  148.096       181.748       206.442       187.836       18.7077        10     
reco:crttag:SingleCRTMatchingProducer              0.0295047     0.0362426     0.0423985     0.036766     0.00405827       10     
reco:crtreco:TwoCRTMatchingProducer                0.0250115     0.0307523     0.0361519     0.0311065    0.00367605       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.06166       1.25157       1.43596       1.25579      0.132656        10     
reco:pandora2Track:LArPandoraTrackCreation          30.5331       41.9323       50.0459       44.2236       5.64468        10     
reco:pandora2calo:Calorimetry                       13.1483       18.5483       23.4584       20.1588       3.23314        10     
reco:pandora2calonosce:Calorimetry                  12.5539       17.8417       21.3994       18.9226       3.03797        10     
reco:pandora2pid:Chi2ParticleID                    0.0112827     0.0150975     0.0199163     0.0149965    0.00234315       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0659245     0.0819369     0.109543      0.0821271     0.0122466       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0112092     0.0147642     0.0194638     0.0145502    0.00219362       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.4867       13.7315       17.1326       14.4194       2.40519        10     
reco:pandora2Showercalo:ShowerCalorimetry           10.6304        16.05        19.3903       17.7573       3.03872        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.48621       14.2937       17.9729       14.8648       3.02477        10     
[art]:TriggerResults:TriggerResultInserter        4.5309e-05    6.59511e-05   0.000120291   5.7778e-05    2.12745e-05      10     
end_path:out1:RootOutput                           7.333e-06    8.4428e-06    1.2157e-05     8.129e-06    1.27879e-06      10     
end_path:out1:RootOutput(write)                     2.19834       2.68841       3.60276       2.6428       0.420833        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4803.56 MB
  Peak resident set size usage (VmHWM): 4003.4 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.33 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9467392
-rw-r--r-- 1 pildune32 pildune 2255755097 Nov 16 18:13 detsim_stage2.root
-rw-r--r-- 1 pildune32 pildune 2142012189 Nov 16 18:11 detsim_stage1.root
-rw-r--r-- 1 pildune32 pildune 1946271393 Nov 16 19:36 reco_stage1.root
-rw-r--r-- 1 pildune32 pildune 1666521124 Nov 16 16:52 g4_stage2.root
-rw-r--r-- 1 pildune32 pildune 1607659799 Nov 16 16:39 g4_stage1.root
-rw-r--r-- 1 pildune32 pildune   75775060 Nov 16 19:29 Pandora_Events.pndr
-rw-r--r-- 1 pildune32 pildune     242684 Nov 16 19:36 jobscript.log
-rw-r--r-- 1 pildune32 pildune     217088 Nov 16 19:36 mem.db
-rw-r--r-- 1 pildune32 pildune      62376 Nov 16 19:36 Pandora_Geometry.xml
-rw-r--r-- 1 pildune32 pildune      36864 Nov 16 19:36 time.db
-rw-r--r-- 1 pildune32 pildune        519 Nov 16 18:13 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 16 16:52 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune32 pildune        519 Nov 16 19:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune32 pildune         52 Nov 16 16:33 all-input-dids.txt
-rw-r--r-- 1 pildune32 pildune          7 Nov 16 19:36 justin-processed-pfns.txt
-rw-r--r-- 1 pildune32 pildune          0 Nov 16 18:20 ProtoDUNE.log
-rw-r--r-- 1 pildune32 pildune          0 Nov 16 18:13 debugprod.log
justIN time: 2026-02-09 10:52:22 UTC       justIN version: 01.06.00