justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270848.1@dunegpschedd02.fnal.gov

Jobsub ID270848.1@dunegpschedd02.fnal.gov
Workflow ID11767
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-12 21:48:33
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2026-01-13 01:13:02
From worker nodeHostnameb9p02p9999.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit343800 (95 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-12 21:50:01
Input filesmonte-carlo-011767-000001
JobscriptExit code0
Real time3h (12095s)
CPU time3h (11313s = 93%)
Max RSS bytes4476248064 (4268 MiB)
Outputting started2026-01-13 01:11:37
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11767/1/001/reco_stage1.root
Finished2026-01-13 01:13:02
Saved logsjustin-logs:270848.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 112206 traces, 0 summary
Retagger: tagging trace set: gauss with 88633 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88633 traces tagged "gauss"
FrameSaver: q=3.09271e+06 n=1613926 tag=gauss
wclsFrameSaver: saving 112206 traces tagged "wiener"
FrameSaver: q=3.26794e+06 n=1444835 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25243 XUs and 25273 XVs -> 162 XUVs
C:0 T:1 47591 XUs and 52411 XVs -> 6304 XUVs
C:0 T:2 70605 XUs and 78601 XVs -> 2944 XUVs
C:0 T:3 40038 XUs and 38845 XVs -> 297 XUVs
C:0 T:4 6488 XUs and 4052 XVs -> 36 XUVs
C:0 T:5 21410 XUs and 9763 XVs -> 1044 XUVs
C:0 T:6 123922 XUs and 104825 XVs -> 4986 XUVs
C:0 T:7 44128 XUs and 35460 XVs -> 425 XUVs
C:0 T:8 7295 XUs and 7331 XVs -> 129 XUVs
C:0 T:9 45916 XUs and 64516 XVs -> 6656 XUVs
C:0 T:10 113912 XUs and 53284 XVs -> 9756 XUVs
C:0 T:11 516 XUs and 428 XVs -> 59 XUVs
32798 XUVs total
10841 collection wire objects
30634 potential space points
Neighbour search...
3143710 tests to find 1491364 neighbours
Iterating with no regularization...
Begin: 7.35896e+07
0 5.84001e+07
1 5.76296e+07
2 5.75125e+07
3 5.74777e+07
Now with regularization...
Begin: 4.63938e+07
0 4.63382e+07
1 4.63135e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411703780
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411703780
fLifetime: 30000 [us]
13-Jan-2026 02:11:35 CET  Closed output file "reco_stage1.root"
13-Jan-2026 02:11:35 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          211.753       295.108       434.221       271.442       74.0595        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000900426   0.00327294    0.00468832    0.00357024    0.00127439       10     
reco:ophit:OpHitFinder                             0.241005      0.274313      0.326447      0.259677      0.0303153       10     
reco:opflash:OpFlashFinder                         0.156495      0.199149      0.273691      0.191498      0.0371581       10     
reco:opslicer:OpSlicer                             0.485872      0.613972       0.7276       0.623813      0.0797093       10     
reco:caldata:DataPrepModule                         3.8345        4.41712       6.73262       4.03564      0.834415        10     
reco:wclsdatasp:WireCellToolkit                     26.5586       27.8775       29.7475       27.726       0.949496        10     
reco:gaushit:GausHitFinder                         0.812992       1.05881       1.39203       1.00068      0.202424        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000331793   0.00042121    0.000681985   0.000373968   0.000107848      10     
reco:reco3d:SpacePointSolver                        7.36755       14.9115       36.3187       12.9526       8.25654        10     
reco:hitpdune:DisambigFromSpacePoints              0.824307       1.50159       2.6109        1.34744      0.587759        10     
reco:pandora:StandardPandora                        16.942        31.8363       60.9712       27.4046       14.0284        10     
reco:pandoraWriter:StandardPandora                  0.26261      0.347373       0.47566      0.328438      0.0741569       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.76063       2.89157       4.49785       2.66663      0.823284        10     
reco:pandoraShower:LArPandoraShowerCreation         2.05117       3.49572       6.72381       2.9246        1.55146        10     
reco:pandoracalo:Calorimetry                        1.10176       1.80928       2.50462       1.73626      0.445448        10     
reco:pandoracalonosce:Calorimetry                  0.894606       1.68804       2.65127       1.60427      0.509375        10     
reco:pandorapid:Chi2ParticleID                    0.00354488    0.00475218    0.00779732    0.00423895    0.00127918       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0333521     0.0438489     0.0851101     0.0373759     0.0151323       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00343184    0.00446348    0.00769429    0.00418884    0.00117218       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.76924       3.99394       7.18722       3.48226       1.81239        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.74406       3.8656        7.37891       3.31093       1.69129        10     
reco:emtrkmichelid:EmTrackMichelId                  117.424       157.306       210.149       152.157       33.105         10     
reco:crttag:SingleCRTMatchingProducer             0.00635589     0.0129861     0.0179385     0.0126073    0.00342358       10     
reco:crtreco:TwoCRTMatchingProducer               0.00465985    0.00824818     0.0113341    0.00793368    0.00189084       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.637144      0.863166       1.0209        0.89723      0.145571        10     
reco:pandora2Track:LArPandoraTrackCreation          5.27064       10.4162       18.5306       8.97878       3.8948         10     
reco:pandora2calo:Calorimetry                       2.51426       5.34147       9.16858       4.87804       2.00436        10     
reco:pandora2calonosce:Calorimetry                   2.373        5.01473       8.83546       4.48449       2.06468        10     
reco:pandora2pid:Chi2ParticleID                   0.00733101     0.0110336     0.0180351     0.010001     0.00329559       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0383345     0.0527533      0.07216      0.0490475     0.0121043       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00721685    0.00997346     0.0150314     0.0094546    0.00229186       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.92422       3.5887        6.48055       3.27674       1.51269        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.24599       4.67782       7.74492       4.55949       1.72311        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.02098       4.32511       7.22347       4.13893       1.56839        10     
[art]:TriggerResults:TriggerResultInserter         3.728e-05    5.51694e-05   0.000108691    4.672e-05    2.04679e-05      10     
end_path:out1:RootOutput                           6.86e-06     9.9921e-06    1.3161e-05     9.935e-06    1.5414e-06       10     
end_path:out1:RootOutput(write)                     1.70373       2.55745       3.43249       2.60438      0.526557        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5306.02 MB
  Peak resident set size usage (VmHWM): 4476.25 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.79 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11020908
-rw-r--r--. 1 duneprd np-comp 2556916935 Jan 13 01:19 detsim_stage2.root
-rw-r--r--. 1 duneprd np-comp 2422307254 Jan 13 01:17 detsim_stage1.root
-rw-r--r--. 1 duneprd np-comp 2272894512 Jan 13 02:11 reco_stage1.root
-rw-r--r--. 1 duneprd np-comp 2019954111 Jan 12 23:13 g4_stage2.root
-rw-r--r--. 1 duneprd np-comp 1931224519 Jan 12 22:58 g4_stage1.root
-rw-r--r--. 1 duneprd np-comp   81500670 Jan 13 02:08 Pandora_Events.pndr
-rw-r--r--. 1 duneprd np-comp     243080 Jan 13 02:11 jobscript.log
-rw-r--r--. 1 duneprd np-comp     217088 Jan 13 02:11 mem.db
-rw-r--r--. 1 duneprd np-comp      62376 Jan 13 02:11 Pandora_Geometry.xml
-rw-r--r--. 1 duneprd np-comp      36864 Jan 13 02:11 time.db
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 01:19 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 12 23:13 g4_protoDUNE_hist.root
-rw-r--r--. 1 duneprd np-comp        519 Jan 13 02:11 protoDUNE_reco1_hist.root
-rw-r--r--. 1 duneprd np-comp         52 Jan 12 22:50 all-input-dids.txt
-rw-r--r--. 1 duneprd np-comp          7 Jan 13 02:11 justin-processed-pfns.txt
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:27 ProtoDUNE.log
-rw-r--r--. 1 duneprd np-comp          0 Jan 13 01:19 debugprod.log
justIN time: 2026-02-06 08:59:46 UTC       justIN version: 01.06.00