justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 237719.1@dunegpschedd01.fnal.gov

Jobsub ID237719.1@dunegpschedd01.fnal.gov
Workflow ID9274
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-22 21:28:56
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat2025-10-23 01:56:35
From worker nodeHostnamedunegli-7310360-0-fnpc8003.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-22 21:29:32
Input filesmonte-carlo-009274-000001
JobscriptExit code0
Real time4h (15875s)
CPU time4h (15566s = 98%)
Max RSS bytes4570144768 (4358 MiB)
Outputting started2025-10-23 01:54:08
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09274/1/001/reco_stage1.root
Finished2025-10-23 01:56:35
Saved logsjustin-logs:237719.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

pare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 153929 traces, 0 summary
Retagger: tagging trace set: gauss with 113645 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 113645 traces tagged "gauss"
FrameSaver: q=4.0927e+06 n=1974258 tag=gauss
wclsFrameSaver: saving 153929 traces tagged "wiener"
FrameSaver: q=4.35303e+06 n=1737131 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32837 XUs and 36214 XVs -> 579 XUVs
C:0 T:1 98907 XUs and 104492 XVs -> 10774 XUVs
C:0 T:2 120865 XUs and 113385 XVs -> 7729 XUVs
C:0 T:3 49112 XUs and 43877 XVs -> 253 XUVs
C:0 T:4 12796 XUs and 16717 XVs -> 349 XUVs
C:0 T:5 40754 XUs and 42802 XVs -> 5406 XUVs
C:0 T:6 104547 XUs and 129166 XVs -> 5000 XUVs
C:0 T:7 41859 XUs and 47195 XVs -> 391 XUVs
C:0 T:8 9744 XUs and 10717 XVs -> 175 XUVs
C:0 T:9 48359 XUs and 64206 XVs -> 5168 XUVs
C:0 T:10 46298 XUs and 41466 XVs -> 7509 XUVs
C:0 T:11 232 XUs and 197 XVs -> 29 XUVs
43362 XUVs total
13678 collection wire objects
39543 potential space points
Neighbour search...
3298787 tests to find 1678892 neighbours
Iterating with no regularization...
Begin: 9.74628e+07
0 7.99251e+07
1 7.87603e+07
2 7.85624e+07
3 7.85001e+07
Now with regularization...
Begin: 6.45448e+07
0 6.44943e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410053919
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410053919
fLifetime: 30000 [us]
23-Oct-2025 01:54:01 UTC  Closed output file "reco_stage1.root"
23-Oct-2025 01:54:01 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          367.431       501.745       727.734       480.506       103.966        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000815806   0.00366793    0.00664892    0.00385679    0.00159068       10     
reco:ophit:OpHitFinder                             0.300974      0.326708      0.366364      0.327153      0.0208355       10     
reco:opflash:OpFlashFinder                         0.220795      0.260025      0.303012      0.262023      0.0257105       10     
reco:opslicer:OpSlicer                             0.681869      0.828554      0.898979      0.821443      0.0628225       10     
reco:caldata:DataPrepModule                         5.7743         5.988        6.43574       5.91109      0.206542        10     
reco:wclsdatasp:WireCellToolkit                     44.8386       53.1321       62.0466        51.38        6.62198        10     
reco:gaushit:GausHitFinder                          1.51061       1.75369       2.04994       1.64632      0.203101        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000447999   0.000608294   0.000724004   0.000627452   8.60466e-05      10     
reco:reco3d:SpacePointSolver                        12.1206       19.1303       31.9989       16.3666       6.53343        10     
reco:hitpdune:DisambigFromSpacePoints               2.54881       3.38524       4.29182       3.23333      0.632872        10     
reco:pandora:StandardPandora                        33.6988       81.2778       221.298       53.2081       58.6996        10     
reco:pandoraWriter:StandardPandora                 0.447553       0.53692      0.662236      0.522563      0.0684369       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.08547       6.04204       8.12374       5.96502       1.26549        10     
reco:pandoraShower:LArPandoraShowerCreation         4.11531       6.55462       8.85306       6.59764       1.55516        10     
reco:pandoracalo:Calorimetry                        2.36561       3.22854       4.04897        3.113       0.508278        10     
reco:pandoracalonosce:Calorimetry                   2.11235       2.93074       3.73874       2.84251      0.490215        10     
reco:pandorapid:Chi2ParticleID                    0.00638655    0.00736456    0.00848725    0.00734174    0.000684948      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0761078     0.0878217     0.104046      0.0861358     0.0080564       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00605302    0.00708807     0.0080438    0.00735925    0.000698978      10     
reco:pandoraShowercalo:ShowerCalorimetry            4.50305       7.15541       9.6814        7.2211        1.62686        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.42785       7.06076       9.63404       7.16035       1.63443        10     
reco:emtrkmichelid:EmTrackMichelId                  195.606        230.9        273.659       228.039       25.5843        10     
reco:crttag:SingleCRTMatchingProducer              0.0165448     0.0224136     0.0258608     0.0236308    0.00273289       10     
reco:crtreco:TwoCRTMatchingProducer                0.0103707     0.0123702     0.0157287     0.0123013     0.0014647       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.74506       2.23852       2.95268       2.20626      0.355646        10     
reco:pandora2Track:LArPandoraTrackCreation          13.1534       20.9777       28.031        20.8672       4.85723        10     
reco:pandora2calo:Calorimetry                       6.52806       10.0958       13.0444       10.258        2.08004        10     
reco:pandora2calonosce:Calorimetry                  6.20585       9.55138       12.4042       9.51163       2.02889        10     
reco:pandora2pid:Chi2ParticleID                    0.0143804     0.0189099     0.0220653     0.0193018    0.00246393       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0880382     0.107138      0.124238      0.108892      0.0123374       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.013805      0.0182641     0.0220284     0.0178936    0.00254125       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.75372       7.26681       9.61312       7.24907       1.59003        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.80869       8.91156       11.5902       9.01167       1.84325        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.61114       8.30063       10.8809       8.24718       1.68711        10     
[art]:TriggerResults:TriggerResultInserter         5.265e-05    6.34093e-05   0.000116738   5.64495e-05   1.83919e-05      10     
end_path:out1:RootOutput                           9.612e-06    1.07758e-05   1.5033e-05    1.03735e-05   1.4609e-06       10     
end_path:out1:RootOutput(write)                     2.90017       3.53559       4.74976       3.42158      0.536995        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5378.84 MB
  Peak resident set size usage (VmHWM): 4570.14 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   3.88 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11937912
-rw-r--r-- 1 dunegli fnalgrid 2731937851 Oct 23 00:26 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2589634559 Oct 23 00:23 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2481207910 Oct 23 01:54 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2212150350 Oct 22 21:57 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2118910931 Oct 22 21:36 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   89965720 Oct 23 01:48 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     242792 Oct 23 01:54 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Oct 23 01:54 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Oct 23 01:54 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Oct 23 01:54 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 23 00:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 22 21:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Oct 23 01:54 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Oct 22 21:29 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Oct 23 01:54 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 23 00:33 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Oct 23 00:27 debugprod.log
justIN time: 2025-11-05 04:51:32 UTC       justIN version: 01.05.01