justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 267282.0@dunegpschedd01.fnal.gov

Jobsub ID267282.0@dunegpschedd01.fnal.gov
Workflow ID11180
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-12 08:07:11
SiteUS_FNAL-FermiGrid
EntryFNAL_GPGrid_ce04_mcore_op_duneonly
Last heartbeat2025-12-12 17:30:33
From worker nodeHostnamedunegli-8047007-0-fnpc19104.fnal.gov
cpuinfoAMD EPYC 7502 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-12 11:34:27
Input filesmonte-carlo-011180-000001
JobscriptExit code0
Real time5h (21278s)
CPU time5h (18478s = 86%)
Max RSS bytes4405075968 (4201 MiB)
Outputting started2025-12-12 17:29:05
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11180/1/001/reco_stage1.root
Finished2025-12-12 17:30:33
Saved logsjustin-logs:267282.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 162315 traces, 0 summary
Retagger: tagging trace set: gauss with 126573 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 126573 traces tagged "gauss"
FrameSaver: q=4.32228e+06 n=2265501 tag=gauss
wclsFrameSaver: saving 162315 traces tagged "wiener"
FrameSaver: q=4.60096e+06 n=2025062 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 34130 XUs and 43048 XVs -> 523 XUVs
C:0 T:1 64540 XUs and 88199 XVs -> 6272 XUVs
C:0 T:2 75542 XUs and 123164 XVs -> 6718 XUVs
C:0 T:3 37641 XUs and 42434 XVs -> 311 XUVs
C:0 T:4 22630 XUs and 23649 XVs -> 295 XUVs
C:0 T:5 119677 XUs and 126251 XVs -> 12792 XUVs
C:0 T:6 101799 XUs and 110766 XVs -> 7732 XUVs
C:0 T:7 44147 XUs and 40613 XVs -> 335 XUVs
C:0 T:8 9496 XUs and 12782 XVs -> 167 XUVs
C:0 T:9 85165 XUs and 178316 XVs -> 13504 XUVs
C:0 T:10 60548 XUs and 28963 XVs -> 5170 XUVs
C:0 T:11 217 XUs and 188 XVs -> 34 XUVs
53853 XUVs total
15376 collection wire objects
49376 potential space points
Neighbour search...
6474262 tests to find 3090594 neighbours
Iterating with no regularization...
Begin: 7.35362e+07
0 5.72446e+07
1 5.61965e+07
2 5.60285e+07
3 5.59702e+07
4 5.59416e+07
Now with regularization...
Begin: 4.48768e+07
0 4.4799e+07
1 4.47467e+07
2 4.47059e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
run: 1 ; subrun: 0 ; event: 10
evttime: 411068698
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411068698
fLifetime: 30000 [us]
12-Dec-2025 17:29:00 UTC  Closed output file "reco_stage1.root"
12-Dec-2025 17:29:00 UTC  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          697.315       1154.47       1913.4        1056.6        400.414        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00204309     0.0133618     0.0270547     0.0105415    0.00881742       10     
reco:ophit:OpHitFinder                             0.347651      0.476878      0.652862      0.480766      0.0944736       10     
reco:opflash:OpFlashFinder                         0.157808      0.403685      0.515426      0.437863      0.116878        10     
reco:opslicer:OpSlicer                             0.675538      0.985204       1.51066      0.909482      0.243199        10     
reco:caldata:DataPrepModule                         5.69981       7.70064       10.3214       7.33759       1.39888        10     
reco:wclsdatasp:WireCellToolkit                     72.1943       90.3306       127.599       78.1907       20.9317        10     
reco:gaushit:GausHitFinder                          1.39015       2.22349       3.58348       1.88466      0.693306        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00186386    0.00473902     0.0139755    0.00276284    0.00431578       10     
reco:reco3d:SpacePointSolver                        20.9625       37.866        106.848       30.0248       24.8514        10     
reco:hitpdune:DisambigFromSpacePoints               2.79031       4.65295       10.9029        3.942        2.28146        10     
reco:pandora:StandardPandora                        49.2189       101.875       178.701       90.4383       42.3495        10     
reco:pandoraWriter:StandardPandora                  0.85881       1.24188       1.92904       1.08533      0.385915        10     
reco:pandoraTrack:LArPandoraTrackCreation           16.6832       31.0253       56.1339       27.3615       12.4209        10     
reco:pandoraShower:LArPandoraShowerCreation         17.4774       39.7163       70.1062       34.7622       16.8455        10     
reco:pandoracalo:Calorimetry                        8.55137       15.9494       34.4146       13.6054       7.51259        10     
reco:pandoracalonosce:Calorimetry                   7.19409       16.4714       35.5062       13.3276       9.65484        10     
reco:pandorapid:Chi2ParticleID                     0.0067377     0.011846      0.0223937     0.0103899    0.00515686       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0539629     0.101215      0.178653      0.0835163     0.0441073       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0054317     0.0106955     0.0238161     0.007071     0.00652403       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.1975       40.0781       87.6798       37.1421       20.8076        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       14.7308       40.6996       76.383        35.5516       19.3208        10     
reco:emtrkmichelid:EmTrackMichelId                  241.376       349.896       522.101       319.484       88.2461        10     
reco:crttag:SingleCRTMatchingProducer              0.0378463     0.0985207     0.187297      0.0932904     0.0470525       10     
reco:crtreco:TwoCRTMatchingProducer                0.0353757     0.0731415     0.110458      0.061563      0.0284926       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.18648       2.12319       3.79879       1.75529      0.862094        10     
reco:pandora2Track:LArPandoraTrackCreation          44.0465       108.471       198.714       96.4689       46.9595        10     
reco:pandora2calo:Calorimetry                        16.33        62.3052       110.702       58.1788       27.8231        10     
reco:pandora2calonosce:Calorimetry                  13.1988       60.9343       105.626       55.2802       30.5141        10     
reco:pandora2pid:Chi2ParticleID                    0.0167333     0.040504      0.0761416     0.0356762     0.0205306       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0775374     0.179163      0.319636      0.165799      0.0740167       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0165542     0.0287707     0.0503564     0.0223415     0.0133489       10     
reco:pandora2Shower:LArPandoraShowerCreation        10.6771       34.7494       62.1936       33.0337       16.0792        10     
reco:pandora2Showercalo:ShowerCalorimetry           10.9135       52.3948       97.5378       48.8282       26.1702        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      9.42102       45.0461       86.2108       40.5276       23.9471        10     
[art]:TriggerResults:TriggerResultInserter        9.0631e-05    0.000200637   0.000587906   0.000157467   0.000133383      10     
end_path:out1:RootOutput                           1.734e-05    0.00141219     0.0130251     2.599e-05    0.00387932       10     
end_path:out1:RootOutput(write)                     2.9452        5.58087       9.62756       4.48328       2.27731        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5306.74 MB
  Peak resident set size usage (VmHWM): 4405.08 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.26 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.44 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.14 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.16 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.16 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.17 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11707780
-rw-r--r-- 1 dunegli fnalgrid 2683231761 Dec 12 14:12 detsim_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2548761057 Dec 12 14:09 detsim_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2420864307 Dec 12 17:29 reco_stage1.root
-rw-r--r-- 1 dunegli fnalgrid 2170148909 Dec 12 11:53 g4_stage2.root
-rw-r--r-- 1 dunegli fnalgrid 2078226566 Dec 12 11:39 g4_stage1.root
-rw-r--r-- 1 dunegli fnalgrid   86844030 Dec 12 17:13 Pandora_Events.pndr
-rw-r--r-- 1 dunegli fnalgrid     243358 Dec 12 17:29 jobscript.log
-rw-r--r-- 1 dunegli fnalgrid     217088 Dec 12 17:29 mem.db
-rw-r--r-- 1 dunegli fnalgrid      62376 Dec 12 17:29 Pandora_Geometry.xml
-rw-r--r-- 1 dunegli fnalgrid      36864 Dec 12 17:29 time.db
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 12 14:12 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 12 11:53 g4_protoDUNE_hist.root
-rw-r--r-- 1 dunegli fnalgrid        519 Dec 12 17:29 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dunegli fnalgrid         52 Dec 12 11:34 all-input-dids.txt
-rw-r--r-- 1 dunegli fnalgrid          7 Dec 12 17:29 justin-processed-pfns.txt
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 12 14:24 ProtoDUNE.log
-rw-r--r-- 1 dunegli fnalgrid          0 Dec 12 14:12 debugprod.log
justIN time: 2025-12-20 00:16:38 UTC       justIN version: 01.05.03