justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269678.0@dunegpschedd01.fnal.gov

Jobsub ID269678.0@dunegpschedd01.fnal.gov
Workflow ID11307
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-22 13:44:35
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-22 16:13:18
From worker nodeHostnamewn-s04-05-34-04.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-22 13:45:06
Input filesmonte-carlo-011307-000001
JobscriptExit code0
Real time2h (8573s)
CPU time2h (8170s = 95%)
Max RSS bytes4311744512 (4112 MiB)
Outputting started2025-12-22 16:08:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11307/1/001/reco_stage1.root
Finished2025-12-22 16:13:18
Saved logsjustin-logs:269678.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ervice:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 123069 traces, 0 summary
Retagger: tagging trace set: gauss with 95385 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 95385 traces tagged "gauss"
FrameSaver: q=3.27397e+06 n=1667002 tag=gauss
wclsFrameSaver: saving 123069 traces tagged "wiener"
FrameSaver: q=3.44932e+06 n=1474807 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 30247 XUs and 27898 XVs -> 389 XUVs
C:0 T:1 54638 XUs and 46560 XVs -> 6249 XUVs
C:0 T:2 75470 XUs and 100178 XVs -> 3681 XUVs
C:0 T:3 42709 XUs and 43797 XVs -> 232 XUVs
C:0 T:4 15389 XUs and 11271 XVs -> 159 XUVs
C:0 T:5 54668 XUs and 38489 XVs -> 5842 XUVs
C:0 T:6 79072 XUs and 78708 XVs -> 2205 XUVs
C:0 T:7 38058 XUs and 34799 XVs -> 202 XUVs
C:0 T:8 7958 XUs and 5869 XVs -> 61 XUVs
C:0 T:9 54910 XUs and 49468 XVs -> 6724 XUVs
C:0 T:10 31585 XUs and 20444 XVs -> 3144 XUVs
C:0 T:11 180 XUs and 124 XVs -> 17 XUVs
28905 XUVs total
10102 collection wire objects
27057 potential space points
Neighbour search...
1781119 tests to find 969750 neighbours
Iterating with no regularization...
Begin: 6.52645e+07
0 5.71811e+07
1 5.65614e+07
2 5.64771e+07
3 5.64489e+07
Now with regularization...
Begin: 4.67489e+07
0 4.67249e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411274539
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411274539
fLifetime: 30000 [us]
22-Dec-2025 17:07:58 CET  Closed output file "reco_stage1.root"
22-Dec-2025 17:07:58 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          171.22        228.98        291.912       226.647       36.0012        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000704195   0.00287917    0.00459684    0.00318742    0.00122515       10     
reco:ophit:OpHitFinder                             0.198198      0.218608      0.235477      0.220423      0.0122086       10     
reco:opflash:OpFlashFinder                         0.113888      0.151509      0.177454      0.159377      0.0194012       10     
reco:opslicer:OpSlicer                             0.383617      0.451184      0.527321      0.455972      0.0470802       10     
reco:caldata:DataPrepModule                         3.51276       3.62637       3.71996       3.62633      0.0588662       10     
reco:wclsdatasp:WireCellToolkit                     23.762        25.965        31.2317       25.0684       2.16735        10     
reco:gaushit:GausHitFinder                         0.578616      0.761169      0.874825      0.781562      0.0961635       10     
reco:nhitsfilter:NumberOfHitsFilter               0.000300275   0.000385845   0.000641226   0.000350825   0.000105387      10     
reco:reco3d:SpacePointSolver                        6.76886       10.9726       21.5742       10.0331       4.58763        10     
reco:hitpdune:DisambigFromSpacePoints              0.658968       1.10252       1.64131       1.00529      0.308806        10     
reco:pandora:StandardPandora                        9.69213       21.2932       36.4039       20.2982       7.52455        10     
reco:pandoraWriter:StandardPandora                 0.207491      0.307356      0.498277      0.293438      0.0769805       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.36268       2.25638       3.13565       2.27948      0.508032        10     
reco:pandoraShower:LArPandoraShowerCreation        0.994592       2.50875       3.4828         2.553       0.663963        10     
reco:pandoracalo:Calorimetry                        0.77785       1.41344       1.80234       1.42507      0.319496        10     
reco:pandoracalonosce:Calorimetry                  0.708136       1.29476       1.63642       1.32517      0.292247        10     
reco:pandorapid:Chi2ParticleID                    0.00299415    0.00364314    0.00509353    0.00343461    0.000603713      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0289363     0.0328932     0.0363165     0.0328095    0.00250993       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00282649    0.00325705    0.00365111    0.00335142    0.000298065      10     
reco:pandoraShowercalo:ShowerCalorimetry            1.06408       2.83493       3.98515       2.86348      0.757706        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.03773       2.77755       3.87756       2.78224      0.744035        10     
reco:emtrkmichelid:EmTrackMichelId                  105.241       123.384       152.417       121.492       15.4399        10     
reco:crttag:SingleCRTMatchingProducer             0.00825837     0.0100008     0.0152411    0.00908798    0.00214943       10     
reco:crtreco:TwoCRTMatchingProducer               0.00412807    0.00591054    0.00906516    0.00535946    0.00139692       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.477355      0.661399      0.867806      0.674083      0.107292        10     
reco:pandora2Track:LArPandoraTrackCreation            3.4         7.68395       11.2905       7.5418        2.13751        10     
reco:pandora2calo:Calorimetry                       1.79797       4.19145       5.29728       4.42571       1.06497        10     
reco:pandora2calonosce:Calorimetry                  1.63304       3.69512       5.1113        3.92523      0.933912        10     
reco:pandora2pid:Chi2ParticleID                   0.00540603    0.00833298     0.0109147    0.00866234     0.0014505       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0321235     0.0413989     0.0500325     0.0404487    0.00546388       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00555191    0.00784504     0.0102983    0.00796017    0.00123751       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.31568       2.68314       3.74302       2.73817       0.66685        10     
reco:pandora2Showercalo:ShowerCalorimetry           1.53461       3.44495       4.64963       3.66151      0.833639        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      1.40895       3.14791       4.14785       3.38563      0.753351        10     
[art]:TriggerResults:TriggerResultInserter        3.5887e-05    5.88638e-05   0.000104376   5.23085e-05   2.06593e-05      10     
end_path:out1:RootOutput                           8.385e-06    1.04495e-05   1.3745e-05    9.1925e-06    2.13744e-06      10     
end_path:out1:RootOutput(write)                     1.44103       1.9551        2.28978       1.99021       0.25582        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5116.69 MB
  Peak resident set size usage (VmHWM): 4311.74 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.18 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10563672
-rw-r--r-- 1 neutrino002 neutrino 2473906427 Dec 22 16:28 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2348527566 Dec 22 16:27 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2166507529 Dec 22 17:07 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 1912215079 Dec 22 15:05 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1838913068 Dec 22 14:52 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   76499190 Dec 22 17:05 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243182 Dec 22 17:08 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 22 17:07 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 22 17:07 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 22 17:07 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 16:28 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 15:05 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 17:07 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 22 14:45 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 22 17:07 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 22 16:32 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 22 16:28 debugprod.log
justIN time: 2026-02-07 06:23:20 UTC       justIN version: 01.06.00