justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269680.0@dunegpschedd01.fnal.gov

Jobsub ID269680.0@dunegpschedd01.fnal.gov
Workflow ID11309
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-22 13:46:35
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2025-12-22 16:24:22
From worker nodeHostnamewn-s04-05-34-04.cr.cnaf.infn.it
cpuinfoAMD EPYC 7313 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-22 13:47:50
Input filesmonte-carlo-011309-000001
JobscriptExit code0
Real time2h (9071s)
CPU time2h (8793s = 96%)
Max RSS bytes4556677120 (4345 MiB)
Outputting started2025-12-22 16:19:02
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11309/1/001/reco_stage1.root
Finished2025-12-22 16:24:22
Saved logsjustin-logs:269680.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 148865 traces, 0 summary
Retagger: tagging trace set: gauss with 110868 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 110868 traces tagged "gauss"
FrameSaver: q=3.75839e+06 n=1929802 tag=gauss
wclsFrameSaver: saving 148865 traces tagged "wiener"
FrameSaver: q=4.01673e+06 n=1691627 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 33320 XUs and 32587 XVs -> 452 XUVs
C:0 T:1 125469 XUs and 84011 XVs -> 12074 XUVs
C:0 T:2 96765 XUs and 98920 XVs -> 9655 XUVs
C:0 T:3 44541 XUs and 38961 XVs -> 368 XUVs
C:0 T:4 9171 XUs and 13482 XVs -> 169 XUVs
C:0 T:5 71295 XUs and 44880 XVs -> 9363 XUVs
C:0 T:6 114704 XUs and 122252 XVs -> 8566 XUVs
C:0 T:7 35605 XUs and 41503 XVs -> 335 XUVs
C:0 T:8 12642 XUs and 8798 XVs -> 103 XUVs
C:0 T:9 110597 XUs and 83031 XVs -> 7310 XUVs
C:0 T:10 66899 XUs and 44827 XVs -> 5890 XUVs
C:0 T:11 124 XUs and 108 XVs -> 12 XUVs
54297 XUVs total
14495 collection wire objects
50231 potential space points
Neighbour search...
7320093 tests to find 3144732 neighbours
Iterating with no regularization...
Begin: 8.80194e+07
0 6.91445e+07
1 6.7928e+07
2 6.77423e+07
3 6.76847e+07
Now with regularization...
Begin: 5.56257e+07
0 5.55259e+07
1 5.54604e+07
2 5.54152e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411274516
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411274516
fLifetime: 30000 [us]
22-Dec-2025 17:19:00 CET  Closed output file "reco_stage1.root"
22-Dec-2025 17:19:00 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          192.174       257.071       336.491       251.181       43.6533        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00080791    0.00303633    0.00498389    0.00323391    0.00130709       10     
reco:ophit:OpHitFinder                             0.208709      0.245573      0.296354      0.234424      0.0267215       10     
reco:opflash:OpFlashFinder                         0.145203      0.188029      0.245922      0.182778      0.0286244       10     
reco:opslicer:OpSlicer                             0.453528      0.513196      0.613114      0.494966      0.0532397       10     
reco:caldata:DataPrepModule                         3.51835       3.64278       3.86391       3.6297       0.0989929       10     
reco:wclsdatasp:WireCellToolkit                     23.5319       26.1159       30.0267       25.5242       2.18563        10     
reco:gaushit:GausHitFinder                         0.712258      0.940154       1.37704      0.925845       0.18523        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000334269   0.000456304   0.000601832   0.000454881   8.20365e-05      10     
reco:reco3d:SpacePointSolver                        8.26985       13.0214       22.0508       11.2988       4.29384        10     
reco:hitpdune:DisambigFromSpacePoints              0.862048       1.3329        2.12024       1.19907      0.364607        10     
reco:pandora:StandardPandora                        14.3065       26.2661       41.6711       23.6161       9.43007        10     
reco:pandoraWriter:StandardPandora                 0.243137      0.325479      0.409063      0.317049      0.050718        10     
reco:pandoraTrack:LArPandoraTrackCreation           1.77839       2.97471       4.50441       2.98522      0.765979        10     
reco:pandoraShower:LArPandoraShowerCreation         1.74296       3.29115       5.0558        2.93245       1.0274         10     
reco:pandoracalo:Calorimetry                        1.13701       1.72804       2.46034       1.77656       0.36386        10     
reco:pandoracalonosce:Calorimetry                   1.02782       1.60148       2.21364       1.64723      0.332425        10     
reco:pandorapid:Chi2ParticleID                    0.00290275    0.00389731    0.00509715    0.00383674    0.000636604      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0307621     0.0376707     0.0476868     0.0362993    0.00477682       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00280203    0.00369193    0.00459262    0.00378932    0.000487606      10     
reco:pandoraShowercalo:ShowerCalorimetry            2.02221       3.62547       5.55706       3.45473       1.01408        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.9813        3.54468       5.14397       3.35862      0.979487        10     
reco:emtrkmichelid:EmTrackMichelId                  106.943       132.368       168.419       126.563       18.4346        10     
reco:crttag:SingleCRTMatchingProducer             0.00748472     0.0101569     0.0128596     0.0100789    0.00162025       10     
reco:crtreco:TwoCRTMatchingProducer               0.00479219    0.00627734     0.0099669    0.00603792    0.00139852       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.529984      0.709318      0.934288      0.666946      0.136131        10     
reco:pandora2Track:LArPandoraTrackCreation          5.36599       9.83747       13.9503       9.55649       2.71316        10     
reco:pandora2calo:Calorimetry                       3.02097       5.37104       7.66147       5.2476        1.41114        10     
reco:pandora2calonosce:Calorimetry                  2.90684       5.05139       6.86803       4.93447       1.23135        10     
reco:pandora2pid:Chi2ParticleID                   0.00641496    0.00994578     0.0141772    0.00997945    0.00207741       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0381162     0.0506396     0.0789748     0.0476544     0.0113413       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00632854    0.00966011     0.013762      0.0093667    0.00192075       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.12017       3.42418       4.61478       3.42307      0.726308        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.74675       4.45357       5.9837        4.45473      0.962503        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.6799        4.06943       5.58047       4.00014       0.86792        10     
[art]:TriggerResults:TriggerResultInserter        3.2692e-05    4.8372e-05    0.000101681   4.28255e-05   1.88701e-05      10     
end_path:out1:RootOutput                           7.073e-06    9.0539e-06    1.2844e-05    8.5155e-06    1.88076e-06      10     
end_path:out1:RootOutput(write)                     1.74918       2.21723       2.83128       2.09158      0.346135        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5360.69 MB
  Peak resident set size usage (VmHWM): 4556.68 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.23 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11354516
-rw-r--r-- 1 neutrino002 neutrino 2620283306 Dec 22 16:34 detsim_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 2477451382 Dec 22 16:33 detsim_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2357876809 Dec 22 17:19 reco_stage1.root
-rw-r--r-- 1 neutrino002 neutrino 2091271441 Dec 22 15:07 g4_stage2.root
-rw-r--r-- 1 neutrino002 neutrino 1992134814 Dec 22 14:54 g4_stage1.root
-rw-r--r-- 1 neutrino002 neutrino   87373570 Dec 22 17:16 Pandora_Events.pndr
-rw-r--r-- 1 neutrino002 neutrino     243178 Dec 22 17:19 jobscript.log
-rw-r--r-- 1 neutrino002 neutrino     217088 Dec 22 17:19 mem.db
-rw-r--r-- 1 neutrino002 neutrino      62376 Dec 22 17:19 Pandora_Geometry.xml
-rw-r--r-- 1 neutrino002 neutrino      36864 Dec 22 17:19 time.db
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 16:34 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 15:07 g4_protoDUNE_hist.root
-rw-r--r-- 1 neutrino002 neutrino        519 Dec 22 17:19 protoDUNE_reco1_hist.root
-rw-r--r-- 1 neutrino002 neutrino         52 Dec 22 14:47 all-input-dids.txt
-rw-r--r-- 1 neutrino002 neutrino          7 Dec 22 17:19 justin-processed-pfns.txt
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 22 16:39 ProtoDUNE.log
-rw-r--r-- 1 neutrino002 neutrino          0 Dec 22 16:34 debugprod.log
justIN time: 2026-02-07 06:22:03 UTC       justIN version: 01.06.00