justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269042.0@dunegpschedd01.fnal.gov

Jobsub ID269042.0@dunegpschedd01.fnal.gov
Workflow ID11242
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-18 10:38:44
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-18 13:32:20
From worker nodeHostnamewn-s04-05-38-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-18 10:39:44
Input filesmonte-carlo-011242-000001
JobscriptExit code0
Real time2h (10030s)
CPU time2h (9662s = 96%)
Max RSS bytes4452331520 (4246 MiB)
Outputting started2025-12-18 13:26:55
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11242/1/001/reco_stage1.root
Finished2025-12-18 13:32:20
Saved logsjustin-logs:269042.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 177596 traces, 0 summary
Retagger: tagging trace set: gauss with 136336 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 136336 traces tagged "gauss"
FrameSaver: q=4.9583e+06 n=2450070 tag=gauss
wclsFrameSaver: saving 177596 traces tagged "wiener"
FrameSaver: q=5.23973e+06 n=2185834 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 64567 XUs and 71743 XVs -> 1784 XUVs
C:0 T:1 160250 XUs and 174159 XVs -> 17277 XUVs
C:0 T:2 129456 XUs and 155960 XVs -> 5995 XUVs
C:0 T:3 55456 XUs and 63683 XVs -> 722 XUVs
C:0 T:4 14808 XUs and 16983 XVs -> 240 XUVs
C:0 T:5 58102 XUs and 53881 XVs -> 11924 XUVs
C:0 T:6 135021 XUs and 152560 XVs -> 7646 XUVs
C:0 T:7 39715 XUs and 44991 XVs -> 662 XUVs
C:0 T:8 8696 XUs and 7528 XVs -> 68 XUVs
C:0 T:9 47644 XUs and 58171 XVs -> 3477 XUVs
C:0 T:10 48186 XUs and 33860 XVs -> 4676 XUVs
C:0 T:11 153 XUs and 96 XVs -> 13 XUVs
54484 XUVs total
15260 collection wire objects
47544 potential space points
Neighbour search...
4679602 tests to find 2252148 neighbours
Iterating with no regularization...
Begin: 1.01569e+08
0 7.84052e+07
1 7.68117e+07
2 7.6579e+07
3 7.6509e+07
Now with regularization...
Begin: 6.23951e+07
0 6.23341e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411191460
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411191460
fLifetime: 30000 [us]
18-Dec-2025 14:26:54 CET  Closed output file "reco_stage1.root"
18-Dec-2025 14:26:54 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          165.195       301.709       483.851       285.048       80.8788        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000778431   0.00331935    0.00636702    0.00329576    0.00157889       10     
reco:ophit:OpHitFinder                             0.204007      0.255003      0.322558      0.240065      0.0438925       10     
reco:opflash:OpFlashFinder                         0.114883      0.165762      0.216913      0.166084      0.030054        10     
reco:opslicer:OpSlicer                             0.325283      0.576463       0.80704      0.520945      0.157186        10     
reco:caldata:DataPrepModule                         3.47234       4.48111       5.57192       4.36708      0.923036        10     
reco:wclsdatasp:WireCellToolkit                     22.5946       28.6007       34.9276       28.1776       4.6058         10     
reco:gaushit:GausHitFinder                         0.618481       1.15518       1.77438       1.13761      0.364375        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000332674   0.000581527   0.000962316   0.000538962   0.000173433      10     
reco:reco3d:SpacePointSolver                        5.95843       16.7266       39.5068       13.0118       9.8181         10     
reco:hitpdune:DisambigFromSpacePoints               0.70135       1.67032       3.15942       1.42685      0.765145        10     
reco:pandora:StandardPandora                        13.636        33.9183       78.9609       26.4574       18.949         10     
reco:pandoraWriter:StandardPandora                 0.213888      0.374639      0.606704      0.328768      0.120449        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.55329       3.62273       6.7037        3.44614       1.43547        10     
reco:pandoraShower:LArPandoraShowerCreation         1.46665       4.21849       9.58706       3.46429       2.32554        10     
reco:pandoracalo:Calorimetry                       0.879177       2.15385       3.83043       2.05913      0.820423        10     
reco:pandoracalonosce:Calorimetry                  0.778003       1.96877       3.50173       1.87885      0.771608        10     
reco:pandorapid:Chi2ParticleID                    0.00325286    0.00485533    0.00757324    0.00463992    0.00140828       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0329919     0.0467308     0.074174      0.0370956     0.0153001       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00297506    0.00449559    0.00734651    0.00389841    0.00141874       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.5463        4.66604       10.0938       3.94561       2.43931        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.57478       4.53087       9.79945       3.70787       2.38038        10     
reco:emtrkmichelid:EmTrackMichelId                  93.0713       151.353       216.087       141.488       38.3376        10     
reco:crttag:SingleCRTMatchingProducer              0.0088815     0.0119876     0.0161938     0.0113749    0.00257879       10     
reco:crtreco:TwoCRTMatchingProducer               0.00428833    0.00752219     0.0126405     0.0072823    0.00238713       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.485998      0.762487       1.07029      0.731851      0.162682        10     
reco:pandora2Track:LArPandoraTrackCreation          4.15111       12.3607       20.802        12.586        4.4377         10     
reco:pandora2calo:Calorimetry                       2.16321       6.05558       8.29881       5.90816       1.84109        10     
reco:pandora2calonosce:Calorimetry                  2.04286       5.68091       7.78761       5.55414       1.71671        10     
reco:pandora2pid:Chi2ParticleID                   0.00624369     0.0107995     0.0160319     0.0106383    0.00271613       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0326864     0.0553369     0.080301       0.05109      0.0141088       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00606648     0.0106927     0.0157335    0.00996264    0.00295115       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.52537       4.01225       5.41211       4.19078       1.16074        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.86415       5.23778       6.71845        5.56         1.4194         10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.7071        4.49607       5.80301       4.68509       1.18455        10     
[art]:TriggerResults:TriggerResultInserter        3.2301e-05    5.54862e-05   0.000132538   4.4659e-05    2.8831e-05       10     
end_path:out1:RootOutput                           7.885e-06    1.08612e-05   1.6862e-05    9.7525e-06    2.7904e-06       10     
end_path:out1:RootOutput(write)                     1.84425       2.41609       2.86669       2.41977      0.323897        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5253.73 MB
  Peak resident set size usage (VmHWM): 4452.33 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.84 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.12 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.04 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11293612
-rw-r--r-- 1 neutrino002 neutrino 2614946349 Dec 18 13:35 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2487345665 Dec 18 13:33 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2337594787 Dec 18 14:26 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2055020664 Dec 18 11:58 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1985689888 Dec 18 11:48 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   83433040 Dec 18 14:22 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243221 Dec 18 14:26 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 18 14:26 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 18 14:26 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 18 14:26 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 18 13:35 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 18 11:58 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 18 14:26 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 18 11:39 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 18 14:26 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 18 13:37 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 18 13:35 debugprod.log
justIN time: 2026-02-07 09:24:51 UTC       justIN version: 01.06.00