justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265103.1@dunegpschedd02.fnal.gov

Jobsub ID265103.1@dunegpschedd02.fnal.gov
Workflow ID11418
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 14:14:44
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-12-30 17:45:25
From worker nodeHostnamecn540.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6252 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 14:15:29
Input filesmonte-carlo-011418-000001
JobscriptExit code0
Real time3h (12383s)
CPU time3h (12204s = 98%)
Max RSS bytes4058935296 (3870 MiB)
Outputting started2025-12-30 17:41:53
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11418/1/001/reco_stage1.root
Finished2025-12-30 17:45:25
Saved logsjustin-logs:265103.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

emoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 131097 traces, 0 summary
Retagger: tagging trace set: gauss with 101061 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101061 traces tagged "gauss"
FrameSaver: q=3.59828e+06 n=1826552 tag=gauss
wclsFrameSaver: saving 131097 traces tagged "wiener"
FrameSaver: q=3.80976e+06 n=1633712 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 37575 XUs and 44230 XVs -> 697 XUVs
C:0 T:1 91270 XUs and 157103 XVs -> 15568 XUVs
C:0 T:2 107338 XUs and 130151 XVs -> 5153 XUVs
C:0 T:3 56476 XUs and 58448 XVs -> 494 XUVs
C:0 T:4 8738 XUs and 11368 XVs -> 97 XUVs
C:0 T:5 50801 XUs and 52032 XVs -> 8781 XUVs
C:0 T:6 55816 XUs and 87974 XVs -> 12015 XUVs
C:0 T:7 18995 XUs and 24627 XVs -> 556 XUVs
C:0 T:8 9487 XUs and 11591 XVs -> 156 XUVs
C:0 T:9 135528 XUs and 96338 XVs -> 16673 XUVs
C:0 T:10 46633 XUs and 40661 XVs -> 5725 XUVs
C:0 T:11 301 XUs and 295 XVs -> 43 XUVs
65958 XUVs total
11507 collection wire objects
59726 potential space points
Neighbour search...
12597588 tests to find 5492716 neighbours
Iterating with no regularization...
Begin: 8.05276e+07
0 7.22851e+07
1 7.15498e+07
2 7.14189e+07
3 7.13733e+07
Now with regularization...
Begin: 6.10445e+07
0 6.10078e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411435879
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411435879
fLifetime: 30000 [us]
30-Dec-2025 17:41:50 GMT  Closed output file "reco_stage1.root"
30-Dec-2025 17:41:50 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          323.356       498.828       654.183       488.333       108.217        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00169571    0.00371271    0.00515116    0.00417655    0.00120411       10     
reco:ophit:OpHitFinder                             0.247951      0.337151      0.445653      0.334641      0.0549056       10     
reco:opflash:OpFlashFinder                         0.150788      0.235774      0.285955      0.237891      0.0465109       10     
reco:opslicer:OpSlicer                             0.630859      0.807612       1.03035       0.78857      0.121786        10     
reco:caldata:DataPrepModule                         6.42412       7.29384       10.212        6.71837       1.33084        10     
reco:wclsdatasp:WireCellToolkit                     53.215        56.1002       57.5117       56.623        1.36199        10     
reco:gaushit:GausHitFinder                          1.08532        1.582        2.00145       1.60304       0.27272        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000587791   0.000757245   0.000958557   0.000750303   0.000124342      10     
reco:reco3d:SpacePointSolver                        10.4336       15.6718       22.6705       14.695        4.46307        10     
reco:hitpdune:DisambigFromSpacePoints               1.3447        2.23571       3.1296        1.99031      0.667274        10     
reco:pandora:StandardPandora                        16.6456       42.262        72.8718       39.7477        17.78         10     
reco:pandoraWriter:StandardPandora                 0.383916      0.537267      0.676989      0.530413      0.0942255       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.75884       9.75321       13.3709       11.0587       2.86624        10     
reco:pandoraShower:LArPandoraShowerCreation         5.19608       11.2747       17.0376       11.3234       4.26238        10     
reco:pandoracalo:Calorimetry                        2.23105       4.77034       7.61584       5.25153       1.69492        10     
reco:pandoracalonosce:Calorimetry                   2.03566       4.58562       8.01498       5.01798       1.75944        10     
reco:pandorapid:Chi2ParticleID                    0.00444651    0.00695233     0.0122818    0.00677423    0.00213116       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0516239     0.075908      0.133098      0.068146      0.0223831       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00446119    0.00661244     0.0116655    0.00661328    0.00189673       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.17697       10.996        21.6863       10.396        5.35601        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.08753       10.7022       21.5988       10.2124       5.33391        10     
reco:emtrkmichelid:EmTrackMichelId                  155.066       213.875       265.702       227.576       36.8255        10     
reco:crttag:SingleCRTMatchingProducer              0.0219826     0.0314777     0.0435397     0.0311943    0.00549638       10     
reco:crtreco:TwoCRTMatchingProducer                0.0196658     0.0272648     0.0388679     0.0267326    0.00547432       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.08162       1.60208       2.53145       1.61304      0.398457        10     
reco:pandora2Track:LArPandoraTrackCreation          15.6599       35.8341       61.5147       36.3759       12.4941        10     
reco:pandora2calo:Calorimetry                       6.16294       15.915        29.1763       16.5636       6.19838        10     
reco:pandora2calonosce:Calorimetry                  5.96429       14.0734       20.3066       13.8723       4.54699        10     
reco:pandora2pid:Chi2ParticleID                    0.0104295     0.0164417     0.0212877     0.0170023    0.00303774       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0632278     0.0959648     0.117149       0.1012       0.0165896       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0104107     0.0163548     0.0214897     0.0168732    0.00308418       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.0356        11.1281       17.521        10.146        3.24831        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.52464       12.4816       18.461        11.7242       3.97217        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.22481       11.1045       17.3833       10.3956       3.55013        10     
[art]:TriggerResults:TriggerResultInserter        4.3967e-05    6.43654e-05   0.000123264   5.6923e-05    2.23585e-05      10     
end_path:out1:RootOutput                           6.783e-06    9.8134e-06    1.4196e-05     9.295e-06    2.22102e-06      10     
end_path:out1:RootOutput(write)                     2.2675        3.20903       4.00719       3.26134      0.587479        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4867.44 MB
  Peak resident set size usage (VmHWM): 4058.94 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.27 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.55 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9712368
-rw-r--r-- 1 pildune22 pildune 2298327966 Dec 30 16:16 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2183376864 Dec 30 16:14 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 1966757133 Dec 30 17:41 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 1751760686 Dec 30 14:39 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1674284670 Dec 30 14:22 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   70348650 Dec 30 17:34 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     242934 Dec 30 17:41 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Dec 30 17:41 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Dec 30 17:41 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Dec 30 17:41 time.db
-rw-r--r-- 1 pildune22 pildune        519 Dec 30 16:16 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec 30 14:39 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Dec 30 17:41 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Dec 30 14:15 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Dec 30 17:41 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Dec 30 16:24 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Dec 30 16:16 debugprod.log
justIN time: 2026-02-07 00:48:56 UTC       justIN version: 01.06.00