justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 238303.1@dunegpschedd01.fnal.gov

Jobsub ID238303.1@dunegpschedd01.fnal.gov
Workflow ID9325
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-24 18:49:24
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-10-25 01:36:43
From worker nodeHostnamewn5917301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-24 18:50:04
Input filesmonte-carlo-009325-000001
JobscriptExit code0
Real time6h (24287s)
CPU time6h (23337s = 96%)
Max RSS bytes4411580416 (4207 MiB)
Outputting started2025-10-25 01:34:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09325/1/001/reco_stage1.root
Finished2025-10-25 01:36:43
Saved logsjustin-logs:238303.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

ol adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 133527 traces, 0 summary
Retagger: tagging trace set: gauss with 100564 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 100564 traces tagged "gauss"
FrameSaver: q=3.67575e+06 n=1711320 tag=gauss
wclsFrameSaver: saving 133527 traces tagged "wiener"
FrameSaver: q=3.92854e+06 n=1513563 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 58750 XUs and 59042 XVs -> 2546 XUVs
C:0 T:1 160362 XUs and 166316 XVs -> 27893 XUVs
C:0 T:2 60988 XUs and 58898 XVs -> 2496 XUVs
C:0 T:3 29870 XUs and 23801 XVs -> 278 XUVs
C:0 T:4 11623 XUs and 12523 XVs -> 561 XUVs
C:0 T:5 33931 XUs and 33085 XVs -> 4605 XUVs
C:0 T:6 87741 XUs and 93475 XVs -> 4175 XUVs
C:0 T:7 36867 XUs and 39072 XVs -> 300 XUVs
C:0 T:8 3273 XUs and 2728 XVs -> 41 XUVs
C:0 T:9 16028 XUs and 16956 XVs -> 1012 XUVs
C:0 T:10 56054 XUs and 41405 XVs -> 4555 XUVs
C:0 T:11 55 XUs and 43 XVs -> 7 XUVs
48469 XUVs total
10551 collection wire objects
42733 potential space points
Neighbour search...
5635235 tests to find 2580418 neighbours
Iterating with no regularization...
Begin: 1.05294e+08
0 8.94631e+07
1 8.8208e+07
2 8.80077e+07
3 8.79567e+07
Now with regularization...
Begin: 7.45109e+07
0 7.44395e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410091893
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410091893
fLifetime: 30000 [us]
25-Oct-2025 02:34:49 BST  Closed output file "reco_stage1.root"
25-Oct-2025 02:34:49 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          533.081       878.226       1229.62       853.539       200.933        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                             0.0018738    0.00779534     0.0266987    0.00537319    0.00727576       10     
reco:ophit:OpHitFinder                             0.431053      0.511753      0.585988      0.509915      0.0380574       10     
reco:opflash:OpFlashFinder                         0.239183      0.311651      0.426433      0.308449      0.0483761       10     
reco:opslicer:OpSlicer                              1.1594        1.43808       1.75265       1.42323       0.14789        10     
reco:caldata:DataPrepModule                         8.86758       9.31432       9.63189       9.36374      0.219111        10     
reco:wclsdatasp:WireCellToolkit                     54.1664       63.6718       73.0387       62.8076       6.00091        10     
reco:gaushit:GausHitFinder                          1.79784       2.58747       3.24175       2.55601      0.476072        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000689221    0.0010824    0.00145414    0.00108063    0.000199251      10     
reco:reco3d:SpacePointSolver                        16.2463       30.8101       40.9711       31.361        7.25334        10     
reco:hitpdune:DisambigFromSpacePoints               2.06554       3.92309       5.53275       3.81341       1.08753        10     
reco:pandora:StandardPandora                        30.6513       99.328        222.344       82.0812       53.9252        10     
reco:pandoraWriter:StandardPandora                 0.685651      0.975384       1.21732      0.944075      0.177128        10     
reco:pandoraTrack:LArPandoraTrackCreation           5.8207        12.8704       18.3786       12.9218       3.75162        10     
reco:pandoraShower:LArPandoraShowerCreation         5.15024        15.72        25.157        14.3138       5.94692        10     
reco:pandoracalo:Calorimetry                        3.29712       7.44768       10.7637       7.54434       2.24635        10     
reco:pandoracalonosce:Calorimetry                   3.05789       7.02354       9.90274       6.82936       2.18255        10     
reco:pandorapid:Chi2ParticleID                    0.00885825     0.012778      0.0143452     0.0132681    0.00150156       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.102692      0.125739      0.149902      0.120136      0.0173151       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0090993     0.0123813     0.0143357     0.0126443    0.00148117       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.57341       17.0673       28.2234       16.2866       6.58514        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.73697       16.3956       26.126        15.5372       5.79866        10     
reco:emtrkmichelid:EmTrackMichelId                  312.079       450.112       563.208       450.82        75.4742        10     
reco:crttag:SingleCRTMatchingProducer              0.0227948     0.031401      0.0432141     0.0321123    0.00556571       10     
reco:crtreco:TwoCRTMatchingProducer                0.0168961     0.0252267     0.0328411     0.0257749    0.00416288       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.23742       1.66953       1.92309       1.77142      0.208285        10     
reco:pandora2Track:LArPandoraTrackCreation          19.1366       41.2195       62.0762       39.5103       12.985         10     
reco:pandora2calo:Calorimetry                       9.50192       21.913        32.2159       22.1409       7.32729        10     
reco:pandora2calonosce:Calorimetry                  8.71684       20.5989       29.8144       21.3143       6.12228        10     
reco:pandora2pid:Chi2ParticleID                    0.0196338     0.0329054     0.0440833     0.0326676    0.00740461       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.119894      0.165817      0.219682      0.155372      0.0314102       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0214119     0.0333105     0.0444467     0.0333558     0.0071247       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.23654       13.9986       20.6954       13.5351       4.07992        10     
reco:pandora2Showercalo:ShowerCalorimetry           8.02949       17.8275       27.3026       17.5658       5.57322        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.52234       14.9407       23.647        13.8745       4.5119         10     
[art]:TriggerResults:TriggerResultInserter        4.4223e-05    6.09611e-05   0.000165993   4.84215e-05   3.51974e-05      10     
end_path:out1:RootOutput                           6.953e-06    8.2314e-06    1.4638e-05     7.429e-06    2.17439e-06      10     
end_path:out1:RootOutput(write)                     4.21974       5.8681        7.32068       5.87555      0.929554        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5277.98 MB
  Peak resident set size usage (VmHWM): 4411.58 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.97 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.08 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11266152
-rw-r--r-- 1 dune01 dune01 2618348772 Oct 25 00:04 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2486631137 Oct 25 00:00 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2351283016 Oct 25 02:34 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2040683881 Oct 24 20:27 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1952287458 Oct 24 20:01 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   86679360 Oct 25 02:25 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243163 Oct 25 02:34 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 25 02:34 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 25 02:34 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 25 02:34 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 25 00:04 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 24 20:27 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 25 02:34 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 24 19:50 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 25 02:34 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 25 00:14 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 25 00:04 debugprod.log
justIN time: 2025-11-07 11:56:44 UTC       justIN version: 01.05.01