justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 274094.1@dunegpschedd01.fnal.gov

Jobsub ID274094.1@dunegpschedd01.fnal.gov
Workflow ID11712
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2026-01-11 05:48:26
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce15-multicore
Last heartbeat2026-01-11 12:40:42
From worker nodeHostnamehnode70.pic.es
cpuinfoAMD EPYC 7443P 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2026-01-11 05:49:18
Input filesmonte-carlo-011712-000001
JobscriptExit code0
Real time6h (24612s)
CPU time3h (12398s = 50%)
Max RSS bytes4378804224 (4175 MiB)
Outputting started2026-01-11 12:39:31
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11712/1/001/reco_stage1.root
Finished2026-01-11 12:40:42
Saved logsjustin-logs:274094.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     (HTCondor job logs unavailable)

Jobscript log (last 10,000 characters)

ocessing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 161649 traces, 0 summary
Retagger: tagging trace set: gauss with 119916 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119916 traces tagged "gauss"
FrameSaver: q=4.02551e+06 n=2078724 tag=gauss
wclsFrameSaver: saving 161649 traces tagged "wiener"
FrameSaver: q=4.28517e+06 n=1813798 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 27124 XUs and 24436 XVs -> 235 XUVs
C:0 T:1 46275 XUs and 42674 XVs -> 3478 XUVs
C:0 T:2 221606 XUs and 201459 XVs -> 22731 XUVs
C:0 T:3 78402 XUs and 66958 XVs -> 937 XUVs
C:0 T:4 10415 XUs and 8461 XVs -> 66 XUVs
C:0 T:5 39297 XUs and 24637 XVs -> 2345 XUVs
C:0 T:6 232557 XUs and 190059 XVs -> 19840 XUVs
C:0 T:7 71807 XUs and 60705 XVs -> 924 XUVs
C:0 T:8 7504 XUs and 6904 XVs -> 100 XUVs
C:0 T:9 41269 XUs and 50099 XVs -> 3265 XUVs
C:0 T:10 148136 XUs and 67211 XVs -> 22313 XUVs
C:0 T:11 214 XUs and 173 XVs -> 17 XUVs
76251 XUVs total
15575 collection wire objects
70106 potential space points
Neighbour search...
14478838 tests to find 6208762 neighbours
Iterating with no regularization...
Begin: 6.7433e+07
0 4.98604e+07
1 4.86464e+07
2 4.84548e+07
3 4.83924e+07
4 4.83633e+07
Now with regularization...
Begin: 3.80295e+07
0 3.79651e+07
1 3.79104e+07
2 3.78648e+07
3 3.78264e+07
4 3.7796e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411670216
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411670216
fLifetime: 30000 [us]
11-Jan-2026 13:39:28 CET  Closed output file "reco_stage1.root"
11-Jan-2026 13:39:29 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          332.216       777.641       1923.78       472.144       583.341        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000807961    0.0036636    0.00588075     0.0038142    0.00163204       10     
reco:ophit:OpHitFinder                             0.239979      0.304742      0.396593      0.283393      0.0576586       10     
reco:opflash:OpFlashFinder                         0.151124      0.214566      0.280762      0.212002      0.0391442       10     
reco:opslicer:OpSlicer                             0.469126      0.707893       1.01222      0.704366      0.180886        10     
reco:caldata:DataPrepModule                         4.09787       5.03181       6.48735       4.80854       0.91064        10     
reco:wclsdatasp:WireCellToolkit                     29.5417       34.0019       42.1195       32.5303       4.10146        10     
reco:gaushit:GausHitFinder                          1.05128       1.38592       1.91429       1.28042      0.260464        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000344959   0.000573748   0.000808921   0.000573855   0.000158558      10     
reco:reco3d:SpacePointSolver                        9.52216       19.3426       30.3304       18.5475       6.66245        10     
reco:hitpdune:DisambigFromSpacePoints               1.22499       2.00506       3.64896       1.72208      0.759071        10     
reco:pandora:StandardPandora                        24.9528       49.5048       96.8934       41.4133       22.167         10     
reco:pandoraWriter:StandardPandora                 0.273263      0.473161       0.77614      0.441668      0.167359        10     
reco:pandoraTrack:LArPandoraTrackCreation           3.57116       5.69479       9.49832       5.37123       1.9338         10     
reco:pandoraShower:LArPandoraShowerCreation         3.10254       6.7076        15.5581       6.60871       3.51179        10     
reco:pandoracalo:Calorimetry                        1.64325       3.07501       5.47764       3.02479       1.10308        10     
reco:pandoracalonosce:Calorimetry                   1.83694       2.94974       4.95502       2.83348      0.969765        10     
reco:pandorapid:Chi2ParticleID                    0.00440741    0.00679878    0.00910733    0.00689523    0.00138307       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0355575     0.0633229     0.102538      0.0602038     0.019053        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00347166    0.00639426    0.00909864    0.00676775    0.00154258       10     
reco:pandoraShowercalo:ShowerCalorimetry            3.03746       6.78944       13.9207       6.65368       3.05724        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.23712       6.66934       13.8587       6.40543       3.16295        10     
reco:emtrkmichelid:EmTrackMichelId                  171.311       370.655       1690.48       212.535       442.102        10     
reco:crttag:SingleCRTMatchingProducer             0.00921992     0.0168633     0.0367016     0.0143138    0.00801462       10     
reco:crtreco:TwoCRTMatchingProducer               0.00794524     0.0124242     0.0213684     0.0116349    0.00395662       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.686923       1.09208       1.49739       1.02197      0.243164        10     
reco:pandora2Track:LArPandoraTrackCreation          9.92631       92.5628       399.322       15.3829       152.565        10     
reco:pandora2calo:Calorimetry                       5.12259       46.997        263.947       7.67145       82.4395        10     
reco:pandora2calonosce:Calorimetry                  4.47599       59.4665       514.335       7.39166       151.676        10     
reco:pandora2pid:Chi2ParticleID                   0.00889471     0.017601      0.0455538     0.0150325     0.0102817       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0401143     0.0998132     0.310722      0.0879365     0.0745572       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00865615     0.0205171     0.0619079     0.0173568     0.0148253       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.50395       31.2285       256.105       6.24112       75.0003        10     
reco:pandora2Showercalo:ShowerCalorimetry           3.18892       20.2335       131.935       8.0356        37.3418        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.88634       6.97819       11.2344       6.54433       2.45933        10     
[art]:TriggerResults:TriggerResultInserter        2.1201e-05    4.19422e-05   0.000116743   3.4871e-05    2.61489e-05      10     
end_path:out1:RootOutput                            4.4e-06      5.993e-06     1.095e-05      5.4e-06     1.82598e-06      10     
end_path:out1:RootOutput(write)                     1.97528       3.18662       3.86819       3.28296      0.532346        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5187.46 MB
  Peak resident set size usage (VmHWM): 4378.8 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.27 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.13 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10890672
-rw-r--r-- 1 dune001 dune 2531103497 Jan 11 11:27 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2395139624 Jan 11 11:25 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2260772931 Jan 11 13:39 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1982785038 Jan 11 07:24 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1896316158 Jan 11 07:03 g4_stage1.root
-rw-r--r-- 1 dune001 dune   85292700 Jan 11 13:09 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243209 Jan 11 13:39 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Jan 11 13:39 mem.db
-rw-r--r-- 1 dune001 dune      62376 Jan 11 13:39 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Jan 11 13:39 time.db
-rw-r--r-- 1 dune001 dune        519 Jan 11 11:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 11 07:24 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Jan 11 13:39 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Jan 11 06:49 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Jan 11 13:39 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Jan 11 11:34 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Jan 11 11:27 debugprod.log
justIN time: 2026-02-06 11:42:47 UTC       justIN version: 01.06.00