justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 236249.0@dunegpschedd02.fnal.gov

Jobsub ID236249.0@dunegpschedd02.fnal.gov
Workflow ID9520
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 07:19:15
SiteES_PIC
EntryDUNE_T1_ES_PIC_ce16-multicore
Last heartbeat2025-11-02 12:21:18
From worker nodeHostnametds423.pic.es
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 07:20:05
Input filesmonte-carlo-009520-000001
JobscriptExit code0
Real time5h (18026s)
CPU time4h (17156s = 95%)
Max RSS bytes4444286976 (4238 MiB)
Outputting started2025-11-02 12:20:32
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09520/1/001/reco_stage1.root
Finished2025-11-02 12:21:18
Saved logsjustin-logs:236249.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

unning tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 142645 traces, 0 summary
Retagger: tagging trace set: gauss with 106403 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 106403 traces tagged "gauss"
FrameSaver: q=3.60801e+06 n=1861624 tag=gauss
wclsFrameSaver: saving 142645 traces tagged "wiener"
FrameSaver: q=3.83993e+06 n=1636572 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38009 XUs and 33860 XVs -> 451 XUVs
C:0 T:1 72149 XUs and 69255 XVs -> 7220 XUVs
C:0 T:2 67911 XUs and 82919 XVs -> 3207 XUVs
C:0 T:3 36626 XUs and 39551 XVs -> 334 XUVs
C:0 T:4 14505 XUs and 11676 XVs -> 179 XUVs
C:0 T:5 87228 XUs and 50715 XVs -> 13817 XUVs
C:0 T:6 98936 XUs and 84196 XVs -> 4240 XUVs
C:0 T:7 43588 XUs and 34789 XVs -> 346 XUVs
C:0 T:8 15201 XUs and 12234 XVs -> 279 XUVs
C:0 T:9 272585 XUs and 201959 XVs -> 44033 XUVs
C:0 T:10 40956 XUs and 21005 XVs -> 4913 XUVs
C:0 T:11 43 XUs and 22 XVs -> 8 XUVs
79027 XUVs total
12850 collection wire objects
74286 potential space points
Neighbour search...
18876614 tests to find 7712700 neighbours
Iterating with no regularization...
Begin: 6.84787e+07
0 5.30714e+07
1 5.19288e+07
2 5.17643e+07
3 5.17151e+07
Now with regularization...
Begin: 4.20625e+07
0 4.20209e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 410263099
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410263099
fLifetime: 30000 [us]
02-Nov-2025 13:20:29 CET  Closed output file "reco_stage1.root"
02-Nov-2025 13:20:29 CET  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          498.95        629.281       758.186       633.138       90.7738        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00140266    0.00458022    0.00741399    0.00466411    0.00174229       10     
reco:ophit:OpHitFinder                             0.281864      0.417626      0.503332      0.434635      0.0604577       10     
reco:opflash:OpFlashFinder                         0.181486      0.273164      0.389421      0.264368      0.0640952       10     
reco:opslicer:OpSlicer                              0.60575       1.11846       1.52023        1.164        0.25444        10     
reco:caldata:DataPrepModule                         5.89692       7.15202       8.58385       7.53489      0.872341        10     
reco:wclsdatasp:WireCellToolkit                     43.2164       48.5697       53.9867       49.3584       3.16779        10     
reco:gaushit:GausHitFinder                          1.65409       2.01395       2.52778       2.07165      0.266781        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000693833   0.000952443   0.00158413    0.00088119    0.000247606      10     
reco:reco3d:SpacePointSolver                        14.808        26.3662       41.5335       25.1364       8.74815        10     
reco:hitpdune:DisambigFromSpacePoints               1.8955        2.70219       3.32339       2.75873      0.502895        10     
reco:pandora:StandardPandora                        44.1221       63.3396       96.0433       62.3224       14.6542        10     
reco:pandoraWriter:StandardPandora                 0.540289      0.738773      0.941994      0.692077      0.123566        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.49742       9.48222       13.2337       8.61698       2.16821        10     
reco:pandoraShower:LArPandoraShowerCreation         6.90993       9.62629       12.7635       9.74565       2.07873        10     
reco:pandoracalo:Calorimetry                        3.61024       5.10308       7.32172       4.70237       1.20593        10     
reco:pandoracalonosce:Calorimetry                   3.02622       4.83003       6.77897       4.62675       1.07855        10     
reco:pandorapid:Chi2ParticleID                    0.00883519     0.0105618     0.012607      0.0105991    0.000971717      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0720402     0.0924628     0.111169      0.0932099     0.0117844       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00784753    0.00973342     0.0108921     0.0100978    0.000946443      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.18737       9.00235       14.7387       8.05155       2.73614        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       3.91697       9.36634       15.8688       8.82808       3.83586        10     
reco:emtrkmichelid:EmTrackMichelId                  275.753       341.626       397.266       338.196       41.7702        10     
reco:crttag:SingleCRTMatchingProducer              0.0177204     0.0268626     0.0380759     0.0268601    0.00528184       10     
reco:crtreco:TwoCRTMatchingProducer                0.0119071     0.0183402     0.0237997     0.0184112    0.00356025       10     
reco:anodepiercerst0:T0RecoAnodePiercers            0.80739       1.28318       1.67848       1.31871      0.263715        10     
reco:pandora2Track:LArPandoraTrackCreation          17.3826       27.4131       37.9392       28.1965       6.92768        10     
reco:pandora2calo:Calorimetry                       7.90318       13.7838       20.4577       13.2706       3.68975        10     
reco:pandora2calonosce:Calorimetry                  6.37271       12.2425       18.2307       11.7176       3.59653        10     
reco:pandora2pid:Chi2ParticleID                    0.0141497     0.0211599     0.0279095     0.0201165    0.00465186       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0682623     0.0984526     0.132109      0.0940048     0.0236552       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0136226     0.0207102     0.0275715     0.0192918    0.00465795       10     
reco:pandora2Shower:LArPandoraShowerCreation        5.90005       8.95128       12.5277       9.53324       2.04634        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.94324       9.95661       14.3572       9.38714       2.37588        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.92159       9.08038       12.4177       8.78065       2.1045         10     
[art]:TriggerResults:TriggerResultInserter        6.0391e-05    8.66377e-05   0.000205123   7.62465e-05   4.03048e-05      10     
end_path:out1:RootOutput                           1.031e-05    1.24571e-05    1.967e-05     1.146e-05    2.6201e-06       10     
end_path:out1:RootOutput(write)                     3.14791       4.33821       5.32829       4.48982      0.653847        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5274.63 MB
  Peak resident set size usage (VmHWM): 4444.29 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.30 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.45 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10921696
-rw-r--r-- 1 dune001 dune 2539462992 Nov  2 11:32 detsim_stage2.root
-rw-r--r-- 1 dune001 dune 2408472760 Nov  2 11:28 detsim_stage1.root
-rw-r--r-- 1 dune001 dune 2249187610 Nov  2 13:20 reco_stage1.root
-rw-r--r-- 1 dune001 dune 1993534181 Nov  2 08:50 g4_stage2.root
-rw-r--r-- 1 dune001 dune 1912051706 Nov  2 08:31 g4_stage1.root
-rw-r--r-- 1 dune001 dune   80472610 Nov  2 13:12 Pandora_Events.pndr
-rw-r--r-- 1 dune001 dune     243186 Nov  2 13:20 jobscript.log
-rw-r--r-- 1 dune001 dune     217088 Nov  2 13:20 mem.db
-rw-r--r-- 1 dune001 dune      62376 Nov  2 13:20 Pandora_Geometry.xml
-rw-r--r-- 1 dune001 dune      36864 Nov  2 13:20 time.db
-rw-r--r-- 1 dune001 dune        519 Nov  2 11:32 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov  2 08:50 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune001 dune        519 Nov  2 13:20 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune001 dune         52 Nov  2 08:20 all-input-dids.txt
-rw-r--r-- 1 dune001 dune          7 Nov  2 13:20 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 dune          0 Nov  2 11:41 ProtoDUNE.log
-rw-r--r-- 1 dune001 dune          0 Nov  2 11:32 debugprod.log
justIN time: 2025-11-05 04:50:02 UTC       justIN version: 01.05.01