justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 235485.0@dunegpschedd02.fnal.gov

Jobsub ID235485.0@dunegpschedd02.fnal.gov
Workflow ID9487
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-30 18:27:59
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-10-30 22:33:42
From worker nodeHostnamewn2209250.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 18:29:34
Input filesmonte-carlo-009487-000001
JobscriptExit code0
Real time4h (14543s)
CPU time3h (13647s = 93%)
Max RSS bytes4179550208 (3985 MiB)
Outputting started2025-10-30 22:31:58
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09487/1/001/reco_stage1.root
Finished2025-10-30 22:33:42
Saved logsjustin-logs:235485.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 149257 traces, 0 summary
Retagger: tagging trace set: gauss with 114583 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 114583 traces tagged "gauss"
FrameSaver: q=3.98798e+06 n=2013402 tag=gauss
wclsFrameSaver: saving 149257 traces tagged "wiener"
FrameSaver: q=4.21268e+06 n=1781783 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44742 XUs and 35223 XVs -> 516 XUVs
C:0 T:1 122583 XUs and 80933 XVs -> 8308 XUVs
C:0 T:2 126528 XUs and 143693 XVs -> 8316 XUVs
C:0 T:3 60303 XUs and 58715 XVs -> 353 XUVs
C:0 T:4 10632 XUs and 8992 XVs -> 117 XUVs
C:0 T:5 36827 XUs and 22694 XVs -> 2048 XUVs
C:0 T:6 162436 XUs and 167190 XVs -> 6529 XUVs
C:0 T:7 65933 XUs and 59871 XVs -> 703 XUVs
C:0 T:8 7659 XUs and 7065 XVs -> 64 XUVs
C:0 T:9 34136 XUs and 42328 XVs -> 2970 XUVs
C:0 T:10 30537 XUs and 23192 XVs -> 4141 XUVs
C:0 T:11 248 XUs and 217 XVs -> 35 XUVs
34100 XUVs total
13644 collection wire objects
29553 potential space points
Neighbour search...
1330687 tests to find 692906 neighbours
Iterating with no regularization...
Begin: 9.92038e+07
0 8.53007e+07
1 8.43128e+07
2 8.41784e+07
3 8.41432e+07
Now with regularization...
Begin: 7.09724e+07
0 7.09581e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410212022
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410212022
fLifetime: 30000 [us]
30-Oct-2025 22:31:54 GMT  Closed output file "reco_stage1.root"
30-Oct-2025 22:31:55 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          360.631       476.666       716.197       453.042       110.087        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00143622    0.00792234     0.0159968    0.00758349    0.00447732       10     
reco:ophit:OpHitFinder                             0.302102      0.378659      0.500218      0.372166      0.0490128       10     
reco:opflash:OpFlashFinder                         0.128584      0.208003      0.333933      0.196287      0.0509098       10     
reco:opslicer:OpSlicer                              0.72201      0.844845       1.1712       0.798957      0.122484        10     
reco:caldata:DataPrepModule                         7.2032        7.62143       8.11178       7.52531      0.260357        10     
reco:wclsdatasp:WireCellToolkit                     43.6325       46.9015       54.6368       45.5586       3.83704        10     
reco:gaushit:GausHitFinder                          1.23877       1.64084       2.17155       1.58694      0.291523        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00060389    0.00087716    0.00129115    0.000816181   0.000218508      10     
reco:reco3d:SpacePointSolver                        11.4958       24.6063       37.5016       24.659        7.72059        10     
reco:hitpdune:DisambigFromSpacePoints               1.36349       2.1134        2.99464       2.1049       0.506049        10     
reco:pandora:StandardPandora                        22.2177       45.4401       116.794       39.9042       25.5929        10     
reco:pandoraWriter:StandardPandora                 0.510249      0.671765       0.98766       0.63632      0.138809        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.80005       7.41674       11.0854       6.62039       1.78251        10     
reco:pandoraShower:LArPandoraShowerCreation         5.19726       8.71788       15.721        7.68943       3.44397        10     
reco:pandoracalo:Calorimetry                        2.95311       4.23025       6.50601       4.05117       1.1698         10     
reco:pandoracalonosce:Calorimetry                   2.61554       3.85961       5.8898        3.5172        1.13294        10     
reco:pandorapid:Chi2ParticleID                    0.00750253    0.00885534     0.0117329    0.00859103    0.00131916       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0648056     0.0814415     0.105581      0.0785468     0.0145767       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00672605    0.00802414    0.00964191    0.00785699    0.000986275      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.48085       8.69718       16.517        8.26713       4.02365        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.29261       8.42667       16.7905       7.62201       3.95208        10     
reco:emtrkmichelid:EmTrackMichelId                  178.331       226.818       289.501       217.748       36.7834        10     
reco:crttag:SingleCRTMatchingProducer              0.0188015     0.0334369     0.055114      0.0308959     0.0121671       10     
reco:crtreco:TwoCRTMatchingProducer                0.0114031     0.0189752     0.028173      0.0179154    0.00604647       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.905686       1.0639        1.26832       1.0414       0.118243        10     
reco:pandora2Track:LArPandoraTrackCreation          14.4292       22.6405       37.7956       20.1842       7.63873        10     
reco:pandora2calo:Calorimetry                       6.79573       11.8241       20.1076       11.1381       4.35612        10     
reco:pandora2calonosce:Calorimetry                  6.07818       11.031        20.0977       10.2162       4.46341        10     
reco:pandora2pid:Chi2ParticleID                    0.015488      0.0193928     0.0266973     0.0183195    0.00314521       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0812593     0.102902      0.153979      0.101704      0.019346        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0146302     0.0187616     0.0276763     0.0175717    0.00353084       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.13011       8.48169       13.5267       7.56855       2.70502        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.68537       9.9325        16.4638       9.24017       3.76403        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.8109        8.84378       14.6791       8.01486       3.30611        10     
[art]:TriggerResults:TriggerResultInserter        8.9428e-05    0.000121008   0.000247528   9.92825e-05   4.69818e-05      10     
end_path:out1:RootOutput                          1.4778e-05    1.82075e-05   3.0708e-05    1.6957e-05    4.36751e-06      10     
end_path:out1:RootOutput(write)                     3.04598       3.72043       4.99643       3.64351      0.607153        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5036.35 MB
  Peak resident set size usage (VmHWM): 4179.55 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.36 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.67 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9832192
-rw-r--r-- 1 dune01 dune01 2325989045 Oct 30 21:09 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2208562870 Oct 30 21:06 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 1997844993 Oct 30 22:31 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 1769749250 Oct 30 18:57 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1693610960 Oct 30 18:37 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   71775240 Oct 30 22:24 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243107 Oct 30 22:31 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 30 22:31 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 30 22:31 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 30 22:31 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 30 21:09 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 30 18:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 30 22:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 30 18:29 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 30 22:31 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 30 21:17 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 30 21:09 debugprod.log
justIN time: 2025-11-03 18:43:33 UTC       justIN version: 01.05.01