justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242833.1@dunegpschedd01.fnal.gov

Jobsub ID242833.1@dunegpschedd01.fnal.gov
Workflow ID9580
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-04 07:51:50
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-04 14:58:20
From worker nodeHostnamewn2206291.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 07:53:24
Input filesmonte-carlo-009580-000001
JobscriptExit code0
Real time7h (25379s)
CPU time6h (24628s = 97%)
Max RSS bytes4283637760 (4085 MiB)
Outputting started2025-11-04 14:56:24
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09580/1/001/reco_stage1.root
Finished2025-11-04 14:58:20
Saved logsjustin-logs:242833.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

rvice:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 140628 traces, 0 summary
Retagger: tagging trace set: gauss with 104671 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 104671 traces tagged "gauss"
FrameSaver: q=3.72889e+06 n=1786543 tag=gauss
wclsFrameSaver: saving 140628 traces tagged "wiener"
FrameSaver: q=3.97916e+06 n=1565465 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38703 XUs and 33979 XVs -> 1150 XUVs
C:0 T:1 177966 XUs and 149152 XVs -> 27779 XUVs
C:0 T:2 58168 XUs and 90438 XVs -> 5385 XUVs
C:0 T:3 30847 XUs and 37037 XVs -> 494 XUVs
C:0 T:4 17250 XUs and 15736 XVs -> 270 XUVs
C:0 T:5 124131 XUs and 81817 XVs -> 21247 XUVs
C:0 T:6 101379 XUs and 119959 XVs -> 6820 XUVs
C:0 T:7 39641 XUs and 44802 XVs -> 259 XUVs
C:0 T:8 8481 XUs and 10098 XVs -> 104 XUVs
C:0 T:9 62483 XUs and 79816 XVs -> 6381 XUVs
C:0 T:10 23449 XUs and 18928 XVs -> 2155 XUVs
C:0 T:11 277 XUs and 244 XVs -> 28 XUVs
72072 XUVs total
12259 collection wire objects
67058 potential space points
Neighbour search...
15684236 tests to find 6415268 neighbours
Iterating with no regularization...
Begin: 8.75953e+07
0 6.89369e+07
1 6.76388e+07
2 6.74604e+07
3 6.74102e+07
Now with regularization...
Begin: 5.49513e+07
0 5.48452e+07
1 5.47854e+07
2 5.47417e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410304026
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410304026
fLifetime: 30000 [us]
04-Nov-2025 14:56:20 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 14:56:20 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          663.688       890.986       1175.6        873.008       139.416        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00196587    0.00869788     0.0232845    0.00601693    0.00694914       10     
reco:ophit:OpHitFinder                             0.404106      0.500769      0.595226       0.50496      0.0466429       10     
reco:opflash:OpFlashFinder                         0.264732       0.34508       0.45926      0.343374      0.0569921       10     
reco:opslicer:OpSlicer                              1.0127        1.2012        1.39079       1.17826      0.134703        10     
reco:caldata:DataPrepModule                         9.81152       10.0716       10.3002       10.0481       0.18603        10     
reco:wclsdatasp:WireCellToolkit                     68.0193       73.1578       82.9476       72.7137       3.69548        10     
reco:gaushit:GausHitFinder                          2.26479       2.67463       3.36477       2.63401      0.345142        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000828342   0.00108558    0.00143587     0.0010722    0.000159611      10     
reco:reco3d:SpacePointSolver                        21.9299       29.8977       36.1986       30.022        4.76439        10     
reco:hitpdune:DisambigFromSpacePoints               3.22849       4.91695       7.68475       4.84127       1.09459        10     
reco:pandora:StandardPandora                        61.6203       90.1525       142.566       78.717        28.4816        10     
reco:pandoraWriter:StandardPandora                 0.741169      0.926456       1.14687      0.911175      0.0966952       10     
reco:pandoraTrack:LArPandoraTrackCreation           13.9224       17.8814       22.439        18.6031       2.94359        10     
reco:pandoraShower:LArPandoraShowerCreation         12.7045       20.3659       30.8334       18.9298       5.28325        10     
reco:pandoracalo:Calorimetry                        6.60996       8.7164        10.8211       9.06941       1.34935        10     
reco:pandoracalonosce:Calorimetry                   6.29465       8.53077       10.3425       8.89761       1.44306        10     
reco:pandorapid:Chi2ParticleID                     0.0109323     0.0131867     0.0159285     0.013028     0.00152095       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP          0.10238      0.145375      0.251489      0.130645      0.0464273       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00960585     0.0126873     0.0185095     0.0124445    0.00245835       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.7166       21.8525       32.7973       20.8155       5.75109        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       12.5294       21.4827       32.8476       20.2053       5.79301        10     
reco:emtrkmichelid:EmTrackMichelId                  306.284       387.444       486.212       383.31        47.4888        10     
reco:crttag:SingleCRTMatchingProducer              0.0361662     0.0536364     0.069157      0.0538173     0.0104662       10     
reco:crtreco:TwoCRTMatchingProducer                0.0263039     0.0391073     0.0625226     0.0373737    0.00863445       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.27608       1.88248       2.35597       1.96267      0.289841        10     
reco:pandora2Track:LArPandoraTrackCreation          37.5548       60.5388       88.5353       60.221        13.9806        10     
reco:pandora2calo:Calorimetry                       17.8676       29.165        41.8767       29.6771       6.60287        10     
reco:pandora2calonosce:Calorimetry                  16.7883       27.6522       39.3829       26.8334       6.32584        10     
reco:pandora2pid:Chi2ParticleID                    0.0239245     0.0327973     0.0552926     0.0303693    0.00843325       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.126217      0.181184      0.239786      0.180335      0.0315732       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0151134     0.0305559     0.0508993     0.0285029    0.00932142       10     
reco:pandora2Shower:LArPandoraShowerCreation        11.4204       18.7804       25.7227       19.2828       3.59925        10     
reco:pandora2Showercalo:ShowerCalorimetry           14.8961       24.5859       34.8037       25.7358       5.29392        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      13.3015       22.1711       31.6833       22.746        4.94772        10     
[art]:TriggerResults:TriggerResultInserter        9.8368e-05    0.000137946   0.000204248   0.000130251   3.08429e-05      10     
end_path:out1:RootOutput                          1.2355e-05    1.46906e-05   1.9164e-05    1.38735e-05   2.31622e-06      10     
end_path:out1:RootOutput(write)                     4.03317       5.33508       6.70449       5.39633       0.70969        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5098.14 MB
  Peak resident set size usage (VmHWM): 4283.64 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.27 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10782228
-rw-r--r-- 1 dune01 dune01 2503188698 Nov  4 12:24 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2373262118 Nov  4 12:20 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2216061472 Nov  4 14:56 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1977538011 Nov  4 08:29 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1888818808 Nov  4 08:03 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   81499350 Nov  4 14:45 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242967 Nov  4 14:56 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  4 14:56 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  4 14:56 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  4 14:56 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  4 12:24 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  4 08:29 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  4 14:56 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  4 07:53 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  4 14:56 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  4 12:34 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  4 12:24 debugprod.log
justIN time: 2026-02-11 10:43:47 UTC       justIN version: 01.06.00