justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 244101.1@dunegpschedd01.fnal.gov

Jobsub ID244101.1@dunegpschedd01.fnal.gov
Workflow ID9799
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-06 07:50:57
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2025-11-06 12:04:54
From worker nodeHostnameccwcondor0031
cpuinfoAMD EPYC 9334 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-06 07:51:42
Input filesmonte-carlo-009799-000001
JobscriptExit code0
Real time4h (15082s)
CPU time4h (14608s = 96%)
Max RSS bytes4574441472 (4362 MiB)
Outputting started2025-11-06 12:03:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09799/1/001/reco_stage1.root
Finished2025-11-06 12:04:54
Saved logsjustin-logs:244101.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 139598 traces, 0 summary
Retagger: tagging trace set: gauss with 106912 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106912 traces tagged "gauss"
FrameSaver: q=3.71234e+06 n=1912111 tag=gauss
wclsFrameSaver: saving 139598 traces tagged "wiener"
FrameSaver: q=3.92475e+06 n=1707133 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34999 XUs and 27430 XVs -> 378 XUVs
C:0 T:1 86660 XUs and 60802 XVs -> 7496 XUVs
C:0 T:2 109051 XUs and 160674 XVs -> 8927 XUVs
C:0 T:3 53412 XUs and 73983 XVs -> 471 XUVs
C:0 T:4 11531 XUs and 12105 XVs -> 119 XUVs
C:0 T:5 100504 XUs and 45316 XVs -> 12285 XUVs
C:0 T:6 83314 XUs and 104788 XVs -> 4343 XUVs
C:0 T:7 35965 XUs and 41645 XVs -> 324 XUVs
C:0 T:8 9945 XUs and 10230 XVs -> 147 XUVs
C:0 T:9 53147 XUs and 72089 XVs -> 3118 XUVs
C:0 T:10 33530 XUs and 24290 XVs -> 3924 XUVs
C:0 T:11 178 XUs and 153 XVs -> 27 XUVs
41559 XUVs total
11759 collection wire objects
38423 potential space points
Neighbour search...
5264089 tests to find 2379412 neighbours
Iterating with no regularization...
Begin: 6.69104e+07
0 5.78937e+07
1 5.7209e+07
2 5.71184e+07
3 5.70908e+07
Now with regularization...
Begin: 4.6958e+07
0 4.69451e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410344149
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410344149
fLifetime: 30000 [us]
06-Nov-2025 13:03:02 CET  Closed output file "reco_stage1.root"
06-Nov-2025 13:03:02 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          454.189       528.289       618.589       526.149       54.111         10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00158561    0.00620715     0.0233742    0.00420924    0.00596789       10     
reco:ophit:OpHitFinder                             0.295016      0.357393      0.423938       0.35547      0.0382309       10     
reco:opflash:OpFlashFinder                          0.17284      0.251145      0.325955      0.250602      0.047951        10     
reco:opslicer:OpSlicer                              0.63868       0.82405       1.00746      0.804791      0.115092        10     
reco:caldata:DataPrepModule                         5.94525       6.15174       6.61807       6.11063      0.180104        10     
reco:wclsdatasp:WireCellToolkit                     36.4181       39.3474       43.8264       39.1188       1.90214        10     
reco:gaushit:GausHitFinder                          1.29463       1.73436       2.17251       1.72655      0.230968        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00051112    0.000956152   0.00287178    0.000744557   0.000658535      10     
reco:reco3d:SpacePointSolver                        17.3716       23.3066       31.3103       22.1815       4.34871        10     
reco:hitpdune:DisambigFromSpacePoints               1.73258       2.54967       3.19882       2.50502      0.458244        10     
reco:pandora:StandardPandora                        30.9522       50.2508       66.5991       51.0908       9.76476        10     
reco:pandoraWriter:StandardPandora                 0.574577      0.700232      0.781864      0.707308      0.0691049       10     
reco:pandoraTrack:LArPandoraTrackCreation           8.32119       9.28917       10.7001       9.27919      0.747781        10     
reco:pandoraShower:LArPandoraShowerCreation         7.04021       11.2353       15.4161       11.1218       2.3816         10     
reco:pandoracalo:Calorimetry                        3.66406       4.87491       6.80657       4.65872      0.949657        10     
reco:pandoracalonosce:Calorimetry                   3.48366       4.62113       6.37003       4.30681      0.957498        10     
reco:pandorapid:Chi2ParticleID                     0.0062589    0.00793014     0.0091868    0.00793831    0.000811079      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0685384     0.0876346     0.115007      0.0834773     0.0139681       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00633038    0.00747418    0.00862819    0.00743909    0.000786093      10     
reco:pandoraShowercalo:ShowerCalorimetry            7.51393       10.8086       15.1395       10.7854       2.19581        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.64461       11.082        14.7997       10.9274       1.99569        10     
reco:emtrkmichelid:EmTrackMichelId                  228.073       256.116       296.826       256.106       20.9496        10     
reco:crttag:SingleCRTMatchingProducer              0.0206253     0.0335166     0.0586753     0.0275671     0.0130706       10     
reco:crtreco:TwoCRTMatchingProducer                0.0111948     0.0188364     0.0283429     0.0181052    0.00642784       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.02048       1.22031       1.49555       1.17871      0.165513        10     
reco:pandora2Track:LArPandoraTrackCreation          22.0544       29.1606       40.0105       27.7375       6.10654        10     
reco:pandora2calo:Calorimetry                       12.3866       15.1729       19.851        14.6912       2.26029        10     
reco:pandora2calonosce:Calorimetry                  9.53043       14.0719       18.7663       14.3147       2.62378        10     
reco:pandora2pid:Chi2ParticleID                    0.0159384     0.0187181     0.0214499     0.0188852    0.00140628       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0768057      0.1064       0.132997      0.104041      0.0161814       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0134472     0.0175311     0.0197201     0.0178967    0.00166858       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.80477       9.31136       11.4894       9.69425       1.79523        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.05578       11.713        15.4486       11.7674       1.89052        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.24436       9.93419       12.2102       10.5582       1.62461        10     
[art]:TriggerResults:TriggerResultInserter        6.0803e-05    8.7111e-05    0.000220016   7.17295e-05   4.49431e-05      10     
end_path:out1:RootOutput                          1.0346e-05    1.59092e-05   3.4853e-05    1.2309e-05    8.18559e-06      10     
end_path:out1:RootOutput(write)                     3.09638       3.66133       4.44557       3.64382      0.416107        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5420.56 MB
  Peak resident set size usage (VmHWM): 4574.44 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.32 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.83 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11678992
-rw-r--r-- 1 dune001 lbno 2689166823 Nov  6 11:32 detsim_stage2.root
-rw-r--r-- 1 dune001 lbno 2550835701 Nov  6 11:29 detsim_stage1.root
-rw-r--r-- 1 dune001 lbno 2434087839 Nov  6 13:03 reco_stage1.root
-rw-r--r-- 1 dune001 lbno 2141329431 Nov  6 09:16 g4_stage2.root
-rw-r--r-- 1 dune001 lbno 2054319270 Nov  6 09:00 g4_stage1.root
-rw-r--r-- 1 dune001 lbno   88931830 Nov  6 12:57 Pandora_Events.pndr
-rw-r--r-- 1 dune001 lbno     242947 Nov  6 13:03 jobscript.log
-rw-r--r-- 1 dune001 lbno     217088 Nov  6 13:03 mem.db
-rw-r--r-- 1 dune001 lbno      62376 Nov  6 13:03 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 lbno      36864 Nov  6 13:03 time.db
-rw-r--r-- 1 dune001 lbno        519 Nov  6 11:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Nov  6 09:16 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 lbno        519 Nov  6 13:03 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 lbno         52 Nov  6 08:51 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno          7 Nov  6 13:03 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno          0 Nov  6 11:41 ProtoDUNE.log
-rw-r--r-- 1 dune001 lbno          0 Nov  6 11:32 debugprod.log
justIN time: 2026-02-10 16:44:47 UTC       justIN version: 01.06.00