justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264454.1@dunegpschedd02.fnal.gov

Jobsub ID264454.1@dunegpschedd02.fnal.gov
Workflow ID11349
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-26 18:47:47
SiteUK_Brunel
EntryCMSHTPC_T2_UK_London_Brunel_dc2_26
Last heartbeat2025-12-27 00:53:05
From worker nodeHostnamewn-a4-05
cpuinfoAMD EPYC 7452 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-26 18:48:22
Input filesmonte-carlo-011349-000001
JobscriptExit code0
Real time6h (21784s)
CPU time5h (20862s = 95%)
Max RSS bytes4610179072 (4396 MiB)
Outputting started2025-12-27 00:51:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11349/1/001/reco_stage1.root
Finished2025-12-27 00:53:05
Saved logsjustin-logs:264454.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

adChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 151745 traces, 0 summary
Retagger: tagging trace set: gauss with 116490 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 116490 traces tagged "gauss"
FrameSaver: q=4.14236e+06 n=2070959 tag=gauss
wclsFrameSaver: saving 151745 traces tagged "wiener"
FrameSaver: q=4.35867e+06 n=1830311 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39885 XUs and 36830 XVs -> 473 XUVs
C:0 T:1 82109 XUs and 76151 XVs -> 8211 XUVs
C:0 T:2 155156 XUs and 237335 XVs -> 8850 XUVs
C:0 T:3 68394 XUs and 86533 XVs -> 869 XUVs
C:0 T:4 15264 XUs and 17034 XVs -> 212 XUVs
C:0 T:5 62789 XUs and 54402 XVs -> 5120 XUVs
C:0 T:6 110067 XUs and 77909 XVs -> 4357 XUVs
C:0 T:7 44708 XUs and 30240 XVs -> 280 XUVs
C:0 T:8 6739 XUs and 5858 XVs -> 58 XUVs
C:0 T:9 37952 XUs and 43019 XVs -> 2785 XUVs
C:0 T:10 34941 XUs and 25748 XVs -> 4228 XUVs
C:0 T:11 123 XUs and 160 XVs -> 14 XUVs
35457 XUVs total
13637 collection wire objects
32539 potential space points
Neighbour search...
2060751 tests to find 1064080 neighbours
Iterating with no regularization...
Begin: 9.22582e+07
0 7.14967e+07
1 7.04556e+07
2 7.0333e+07
3 7.03001e+07
Now with regularization...
Begin: 5.84322e+07
0 5.84046e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411359229
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411359229
fLifetime: 30000 [us]
27-Dec-2025 00:51:24 GMT  Closed output file "reco_stage1.root"
27-Dec-2025 00:51:24 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          529.725        753.5        1059.67       742.005       167.358        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00172258    0.00552463     0.0091663     0.0053425    0.00270604       10     
reco:ophit:OpHitFinder                             0.402028      0.500993      0.581497      0.506531      0.0636052       10     
reco:opflash:OpFlashFinder                         0.250245      0.328408      0.429236      0.320742      0.0569397       10     
reco:opslicer:OpSlicer                             0.959515       1.33591       1.51896       1.4001       0.177446        10     
reco:caldata:DataPrepModule                         6.68064       8.30015       8.91424       8.42873      0.616291        10     
reco:wclsdatasp:WireCellToolkit                     44.5602       51.9505       55.1587       52.8503       3.17253        10     
reco:gaushit:GausHitFinder                          1.50717       2.17103       2.81073       2.32313      0.365704        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000604711   0.000937456   0.00134012    0.000898877   0.000227852      10     
reco:reco3d:SpacePointSolver                        20.2395       33.0932       48.2308       33.0429       10.143         10     
reco:hitpdune:DisambigFromSpacePoints               2.36957       3.46457       5.89327       3.05188       1.1844         10     
reco:pandora:StandardPandora                        38.4273       85.227        173.848       75.7651       39.6518        10     
reco:pandoraWriter:StandardPandora                 0.457528      0.862964       1.17366      0.871958      0.221327        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.24168       10.2953       16.2273       10.713        3.10836        10     
reco:pandoraShower:LArPandoraShowerCreation         6.05709       11.8501       20.0102       11.1132       4.24742        10     
reco:pandoracalo:Calorimetry                        3.30415       5.56989       7.68367       5.65272       1.50156        10     
reco:pandoracalonosce:Calorimetry                   2.17056       5.08924       7.58013       5.45088       1.54937        10     
reco:pandorapid:Chi2ParticleID                    0.00678242     0.0126345     0.0181141     0.0131125    0.00306674       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0515461     0.118845      0.163735       0.1213       0.0293652       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00663893     0.0120207     0.0155802     0.0126902    0.00265868       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.11842       11.7176       21.459        12.1483       4.73174        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.90968       11.313        20.1597       11.549        4.40491        10     
reco:emtrkmichelid:EmTrackMichelId                  298.704       404.138       513.613       393.772       70.8206        10     
reco:crttag:SingleCRTMatchingProducer              0.0170268     0.0276455     0.0360788     0.0276663    0.00527874       10     
reco:crtreco:TwoCRTMatchingProducer                0.0118127     0.0196781     0.0241341     0.0194456    0.00324644       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.27016       1.62559       1.97974       1.59347      0.239539        10     
reco:pandora2Track:LArPandoraTrackCreation          17.0604       33.2328       51.3348       33.4433       11.0919        10     
reco:pandora2calo:Calorimetry                       5.98365       15.6676       23.9062       16.3935       5.51723        10     
reco:pandora2calonosce:Calorimetry                  6.08799       14.9425       23.6263       15.5505       5.13762        10     
reco:pandora2pid:Chi2ParticleID                    0.016388      0.030595      0.0434149     0.0337879     0.0083582       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0689206     0.140936      0.204933      0.147015      0.0449184       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0157872      0.02789      0.039008      0.0280602    0.00763779       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.4617        11.8739       19.4322       12.2056       4.11759        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.16635       11.8813       18.7064       11.1648       3.5847         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.66415       11.1089       18.7703       10.8111       3.53995        10     
[art]:TriggerResults:TriggerResultInserter        6.6991e-05    9.41366e-05   0.000153213   8.66215e-05   2.29172e-05      10     
end_path:out1:RootOutput                           9.68e-06     1.26964e-05   1.5331e-05    1.30105e-05   1.83961e-06      10     
end_path:out1:RootOutput(write)                     3.90075       5.3819        7.29315       5.19405      0.986898        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5443.96 MB
  Peak resident set size usage (VmHWM): 4610.18 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.25 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11831308
-rw-r--r--. 1 dune000 dune 2705449102 Dec 26 22:41 detsim_stage2.root
-rw-r--r--. 1 dune000 dune 2565052194 Dec 26 22:37 detsim_stage1.root
-rw-r--r--. 1 dune000 dune 2450304466 Dec 27 00:51 reco_stage1.root
-rw-r--r--. 1 dune000 dune 2197379745 Dec 26 19:22 g4_stage2.root
-rw-r--r--. 1 dune000 dune 2107133948 Dec 26 18:59 g4_stage1.root
-rw-r--r--. 1 dune000 dune   89309240 Dec 27 00:42 Pandora_Events.pndr
-rw-r--r--. 1 dune000 dune     243897 Dec 27 00:51 jobscript.log
-rw-r--r--. 1 dune000 dune     217088 Dec 27 00:51 mem.db
-rw-r--r--. 1 dune000 dune      62376 Dec 27 00:51 Pandora_Geometry.xml
-rw-r--r--. 1 dune000 dune      36864 Dec 27 00:51 time.db
-rw-r--r--. 1 dune000 dune        519 Dec 26 22:41 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune        519 Dec 26 19:22 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune000 dune        519 Dec 27 00:51 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune000 dune         52 Dec 26 18:48 all-input-dids.txt
-rw-r--r--. 1 dune000 dune          7 Dec 27 00:51 justin-processed-pfns.txt
-rw-r--r--. 1 dune000 dune          0 Dec 26 22:52 ProtoDUNE.log
-rw-r--r--. 1 dune000 dune          0 Dec 26 22:41 debugprod.log
justIN time: 2026-02-07 03:35:08 UTC       justIN version: 01.06.00