justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269214.1@dunegpschedd01.fnal.gov

Jobsub ID269214.1@dunegpschedd01.fnal.gov
Workflow ID11275
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-19 08:14:42
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-19 10:42:08
From worker nodeHostnamewn-s04-05-38-02.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-19 08:15:56
Input filesmonte-carlo-011275-000001
JobscriptExit code0
Real time2h (8451s)
CPU time2h (8179s = 96%)
Max RSS bytes4473430016 (4266 MiB)
Outputting started2025-12-19 10:36:48
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11275/1/001/reco_stage1.root
Finished2025-12-19 10:42:08
Saved logsjustin-logs:269214.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

repService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 114010 traces, 0 summary
Retagger: tagging trace set: gauss with 88311 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88311 traces tagged "gauss"
FrameSaver: q=2.99601e+06 n=1555025 tag=gauss
wclsFrameSaver: saving 114010 traces tagged "wiener"
FrameSaver: q=3.17068e+06 n=1378396 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 44371 XUs and 38274 XVs -> 574 XUVs
C:0 T:1 65035 XUs and 57683 XVs -> 4134 XUVs
C:0 T:2 180739 XUs and 82249 XVs -> 7716 XUVs
C:0 T:3 47523 XUs and 30861 XVs -> 205 XUVs
C:0 T:4 8412 XUs and 9689 XVs -> 160 XUVs
C:0 T:5 40132 XUs and 86662 XVs -> 9909 XUVs
C:0 T:6 80337 XUs and 71850 XVs -> 2839 XUVs
C:0 T:7 27690 XUs and 29143 XVs -> 217 XUVs
C:0 T:8 8580 XUs and 9294 XVs -> 135 XUVs
C:0 T:9 57262 XUs and 71594 XVs -> 5254 XUVs
C:0 T:10 11411 XUs and 8393 XVs -> 2530 XUVs
C:0 T:11 149 XUs and 88 XVs -> 16 XUVs
33689 XUVs total
9671 collection wire objects
30122 potential space points
Neighbour search...
3275386 tests to find 1621520 neighbours
Iterating with no regularization...
Begin: 7.03921e+07
0 5.58099e+07
1 5.51876e+07
2 5.50955e+07
3 5.50675e+07
Now with regularization...
Begin: 4.68755e+07
0 4.6829e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411209102
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411209102
fLifetime: 30000 [us]
19-Dec-2025 11:36:47 CET  Closed output file "reco_stage1.root"
19-Dec-2025 11:36:47 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          170.25        249.806       345.47        250.245       44.8186        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000684755    0.0027236     0.0050804    0.00270754    0.00136368       10     
reco:ophit:OpHitFinder                             0.202545      0.230704       0.2628        0.22963      0.0156158       10     
reco:opflash:OpFlashFinder                         0.122427      0.166087      0.198764      0.173185      0.0232433       10     
reco:opslicer:OpSlicer                             0.420855      0.508232      0.602614      0.506496      0.0453552       10     
reco:caldata:DataPrepModule                         3.38547       3.4821        3.7237        3.46129      0.0900153       10     
reco:wclsdatasp:WireCellToolkit                     21.9943       22.6086       23.6769       22.4048      0.582082        10     
reco:gaushit:GausHitFinder                         0.644951      0.882318       1.09693      0.879198      0.113887        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000258105   0.000376896   0.000600747   0.000361153   8.9862e-05       10     
reco:reco3d:SpacePointSolver                        7.32727       14.3209       21.1401       14.3827       4.24758        10     
reco:hitpdune:DisambigFromSpacePoints              0.798576       1.31057       1.93405       1.26125      0.297211        10     
reco:pandora:StandardPandora                        12.0711       32.7321       64.9197       27.7994       17.2814        10     
reco:pandoraWriter:StandardPandora                 0.207113       0.28711      0.368642      0.292973      0.0428077       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.52793       2.63044       3.88558       2.5713       0.706408        10     
reco:pandoraShower:LArPandoraShowerCreation         1.67317       2.96348       4.90117       2.90046      0.854371        10     
reco:pandoracalo:Calorimetry                       0.953704       1.57486       2.22953       1.62936       0.37212        10     
reco:pandoracalonosce:Calorimetry                  0.870487       1.46426       2.08875       1.47329      0.365729        10     
reco:pandorapid:Chi2ParticleID                    0.00283979    0.00389156    0.00563244    0.00368871    0.000761971      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0289325     0.0353167     0.0429669     0.0350622    0.00355908       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00263933    0.00353299    0.00436971    0.00350229    0.00056277       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.91094       3.38042       5.44098       3.35091      0.922305        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.87331       3.31146       5.30116       3.28928       0.90006        10     
reco:emtrkmichelid:EmTrackMichelId                  95.4601       126.374       162.349       128.032       17.1013        10     
reco:crttag:SingleCRTMatchingProducer             0.00490379    0.00979959     0.0165131    0.00914479    0.00297638       10     
reco:crtreco:TwoCRTMatchingProducer               0.00407362     0.0053813    0.00674438    0.00544467    0.000727889      10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.422066      0.596494      0.838464      0.575113      0.129715        10     
reco:pandora2Track:LArPandoraTrackCreation          4.82975       8.58336       13.5838       8.60315       2.28582        10     
reco:pandora2calo:Calorimetry                       2.70685       4.72139       7.20858       4.75348       1.16058        10     
reco:pandora2calonosce:Calorimetry                  2.58761       4.49126       7.07328       4.49425       1.15531        10     
reco:pandora2pid:Chi2ParticleID                   0.00637867    0.00928405     0.0118833    0.00938546    0.00168274       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0327386     0.0448577     0.0571491     0.0446412     0.0062371       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00623743    0.00894686     0.0116863    0.00900267    0.00153951       10     
reco:pandora2Shower:LArPandoraShowerCreation        1.87786       3.04543       4.60088       3.06666      0.698394        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.43325       4.08142       5.99725       4.12689      0.908443        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.26364       3.76071       5.48102       3.81306      0.824195        10     
[art]:TriggerResults:TriggerResultInserter        2.6219e-05    3.80575e-05   0.000104005   3.0918e-05    2.23182e-05      10     
end_path:out1:RootOutput                           6.031e-06     6.907e-06     1.102e-05    6.3275e-06    1.42319e-06      10     
end_path:out1:RootOutput(write)                     1.55323       2.11674       2.73621       2.11427      0.300028        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5274.78 MB
  Peak resident set size usage (VmHWM): 4473.43 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.20 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11173448
-rw-r--r-- 1 neutrino002 neutrino 2587719072 Dec 19 10:54 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2453342446 Dec 19 10:52 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2322813670 Dec 19 11:36 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2028310616 Dec 19 09:31 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1962456835 Dec 19 09:22 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   86331870 Dec 19 11:34 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     242876 Dec 19 11:36 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 19 11:36 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 19 11:36 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 19 11:36 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 19 10:54 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 19 09:31 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 19 11:36 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 19 09:15 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 19 11:36 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 19 10:58 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 19 10:54 debugprod.log
justIN time: 2026-02-07 07:55:48 UTC       justIN version: 01.06.00