justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265238.1@dunegpschedd02.fnal.gov

Jobsub ID265238.1@dunegpschedd02.fnal.gov
Workflow ID11435
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-31 08:47:39
SiteCZ_FZU
EntryNova_CZ_FZU_osgce2
Last heartbeat2025-12-31 13:24:05
From worker nodeHostnameminis05.farm.particle.cz
cpuinfoAMD EPYC 7351P 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-31 08:49:03
Input filesmonte-carlo-011435-000001
JobscriptExit code0
Real time4h (16455s)
CPU time4h (15814s = 96%)
Max RSS bytes4178075648 (3984 MiB)
Outputting started2025-12-31 13:23:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11435/1/001/reco_stage1.root
Finished2025-12-31 13:24:05
Saved logsjustin-logs:265238.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

awDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 125112 traces, 0 summary
Retagger: tagging trace set: gauss with 102380 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 102380 traces tagged "gauss"
FrameSaver: q=3.47946e+06 n=1983023 tag=gauss
wclsFrameSaver: saving 125112 traces tagged "wiener"
FrameSaver: q=3.64309e+06 n=1798060 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30183 XUs and 27520 XVs -> 190 XUVs
C:0 T:1 53602 XUs and 49407 XVs -> 2381 XUVs
C:0 T:2 168643 XUs and 145607 XVs -> 10117 XUVs
C:0 T:3 70194 XUs and 52859 XVs -> 476 XUVs
C:0 T:4 9457 XUs and 8467 XVs -> 70 XUVs
C:0 T:5 28652 XUs and 18483 XVs -> 1734 XUVs
C:0 T:6 135705 XUs and 118305 XVs -> 4472 XUVs
C:0 T:7 54706 XUs and 45343 XVs -> 395 XUVs
C:0 T:8 10554 XUs and 10120 XVs -> 114 XUVs
C:0 T:9 73210 XUs and 78852 XVs -> 4836 XUVs
C:0 T:10 77492 XUs and 37248 XVs -> 4970 XUVs
C:0 T:11 159 XUs and 170 XVs -> 25 XUVs
29780 XUVs total
12958 collection wire objects
27722 potential space points
Neighbour search...
2057120 tests to find 972046 neighbours
Iterating with no regularization...
Begin: 5.99413e+07
0 5.30249e+07
1 5.25162e+07
2 5.2449e+07
3 5.24323e+07
Now with regularization...
Begin: 4.39249e+07
0 4.39147e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411451471
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411451471
fLifetime: 30000 [us]
31-Dec-2025 14:23:17 CET  Closed output file "reco_stage1.root"
31-Dec-2025 14:23:17 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          430.692       573.926       805.198       552.102       116.721        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00173009    0.00345925    0.00448399    0.00383468    0.000890674      10     
reco:ophit:OpHitFinder                             0.330189      0.384311      0.482979      0.372687      0.0490362       10     
reco:opflash:OpFlashFinder                         0.146365      0.201758      0.283537       0.19228      0.043022        10     
reco:opslicer:OpSlicer                             0.701133       1.12073       1.57158       1.07819       0.25028        10     
reco:caldata:DataPrepModule                         8.06757       8.32061       8.52272       8.36849      0.148504        10     
reco:wclsdatasp:WireCellToolkit                     52.7305       57.5608       61.7603       57.8173       2.86138        10     
reco:gaushit:GausHitFinder                          1.55919       2.00454       2.69345       1.92985      0.349286        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000625611   0.000738547   0.000932605   0.000705777   9.83488e-05      10     
reco:reco3d:SpacePointSolver                        12.6349       19.6744       29.941        19.042        6.20608        10     
reco:hitpdune:DisambigFromSpacePoints                1.676        2.45597       3.79367       2.3821       0.646175        10     
reco:pandora:StandardPandora                        32.0423       55.4484       94.0423       51.2905       19.7266        10     
reco:pandoraWriter:StandardPandora                 0.527389      0.639302      0.832363      0.625658      0.0899731       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.87187       6.50878       9.88732       6.49178       1.39423        10     
reco:pandoraShower:LArPandoraShowerCreation         5.09778       7.60318       12.4565       6.84882       2.46842        10     
reco:pandoracalo:Calorimetry                        2.30255       3.72289       5.93541       3.74557       1.02395        10     
reco:pandoracalonosce:Calorimetry                   1.95449       3.36528       5.37279       3.17974      0.950613        10     
reco:pandorapid:Chi2ParticleID                    0.00633387    0.00965868     0.0117006     0.0102643    0.00151428       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0710231     0.089535      0.118295      0.0867283     0.0135999       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0060825    0.00955982     0.0117057    0.00961481    0.00136436       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.37766       7.73882       13.4846       7.47348       3.09688        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.06988       7.41273       12.8921       7.03203       3.03626        10     
reco:emtrkmichelid:EmTrackMichelId                  245.873       318.252       433.949       301.292       56.2166        10     
reco:crttag:SingleCRTMatchingProducer              0.0165819     0.0213699     0.0275578     0.0207523    0.00383843       10     
reco:crtreco:TwoCRTMatchingProducer                0.0124431     0.0174147     0.0399518     0.0152744    0.00767341       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.05838       1.29856       1.71245       1.32529      0.196479        10     
reco:pandora2Track:LArPandoraTrackCreation          13.2737       20.7957       33.7415       19.7255       6.34778        10     
reco:pandora2calo:Calorimetry                       6.06491       10.3323       17.7003       9.71036       3.81066        10     
reco:pandora2calonosce:Calorimetry                  5.5018        9.89757       16.7432       9.58329       3.52515        10     
reco:pandora2pid:Chi2ParticleID                    0.0200584     0.025369      0.0324352     0.0251869    0.00310564       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.100827      0.116914      0.155014       0.1099       0.0175576       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0193209     0.024599      0.0328588     0.0246488    0.00426099       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.74067       7.72067       12.781        7.23574       2.31054        10     
reco:pandora2Showercalo:ShowerCalorimetry           4.73674       8.73245       15.2455       8.50494       3.12729        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      4.2691        7.84011       13.7524       7.39995       2.88323        10     
[art]:TriggerResults:TriggerResultInserter        6.6569e-05    9.14981e-05    0.0001653    8.48645e-05   2.70997e-05      10     
end_path:out1:RootOutput                          1.1262e-05    1.27278e-05   1.7875e-05    1.18285e-05   1.96474e-06      10     
end_path:out1:RootOutput(write)                     3.70881       4.44283       5.80417       4.3129       0.673416        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4986.04 MB
  Peak resident set size usage (VmHWM): 4178.08 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.19 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 9778708
-rw-r--r--. 1 fermilab users 2320062633 Dec 31 12:43 detsim_stage2.root
-rw-r--r--. 1 fermilab users 2206875392 Dec 31 12:40 detsim_stage1.root
-rw-r--r--. 1 fermilab users 1991551523 Dec 31 14:23 reco_stage1.root
-rw-r--r--. 1 fermilab users 1746419561 Dec 31 10:16 g4_stage2.root
-rw-r--r--. 1 fermilab users 1676109739 Dec 31 09:58 g4_stage1.root
-rw-r--r--. 1 fermilab users   71753020 Dec 31 14:13 Pandora_Events.pndr
-rw-r--r--. 1 fermilab users     242627 Dec 31 14:23 jobscript.log
-rw-r--r--. 1 fermilab users     217088 Dec 31 14:23 mem.db
-rw-r--r--. 1 fermilab users      62376 Dec 31 14:23 Pandora_Geometry.xml
-rw-r--r--. 1 fermilab users      36864 Dec 31 14:23 time.db
-rw-r--r--. 1 fermilab users        519 Dec 31 12:43 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 fermilab users        519 Dec 31 10:16 g4_protoDUNE_hist.root
-rw-r--r--. 1 fermilab users        519 Dec 31 14:23 protoDUNE_reco1_hist.root
-rw-r--r--. 1 fermilab users         52 Dec 31 09:49 all-input-dids.txt
-rw-r--r--. 1 fermilab users          7 Dec 31 14:23 justin-processed-pfns.txt
-rw-r--r--. 1 fermilab users          0 Dec 31 12:51 ProtoDUNE.log
-rw-r--r--. 1 fermilab users          0 Dec 31 12:43 debugprod.log
justIN time: 2026-02-07 00:49:52 UTC       justIN version: 01.06.00