justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 265071.0@dunegpschedd02.fnal.gov

Jobsub ID265071.0@dunegpschedd02.fnal.gov
Workflow ID11413
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-12-30 06:20:18
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce3
Last heartbeat2025-12-30 09:07:52
From worker nodeHostnamen252.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-12-30 06:29:20
Input filesmonte-carlo-011413-000001
JobscriptExit code0
Real time2h (9414s)
CPU time2h (9181s = 97%)
Max RSS bytes4158844928 (3966 MiB)
Outputting started2025-12-30 09:06:19
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/11413/1/001/reco_stage1.root
Finished2025-12-30 09:07:52
Saved logsjustin-logs:265071.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

Finder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 157783 traces, 0 summary
Retagger: tagging trace set: gauss with 119469 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 119469 traces tagged "gauss"
FrameSaver: q=4.27201e+06 n=2082860 tag=gauss
wclsFrameSaver: saving 157783 traces tagged "wiener"
FrameSaver: q=4.52605e+06 n=1838776 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 36597 XUs and 32809 XVs -> 477 XUVs
C:0 T:1 71684 XUs and 55606 XVs -> 5905 XUVs
C:0 T:2 101326 XUs and 137985 XVs -> 5505 XUVs
C:0 T:3 52823 XUs and 53918 XVs -> 381 XUVs
C:0 T:4 13576 XUs and 14753 XVs -> 102 XUVs
C:0 T:5 135183 XUs and 62005 XVs -> 14602 XUVs
C:0 T:6 149796 XUs and 172592 XVs -> 6972 XUVs
C:0 T:7 57413 XUs and 53771 XVs -> 683 XUVs
C:0 T:8 5106 XUs and 5690 XVs -> 42 XUVs
C:0 T:9 34770 XUs and 47052 XVs -> 4149 XUVs
C:0 T:10 57962 XUs and 50509 XVs -> 4517 XUVs
C:0 T:11 826 XUs and 771 XVs -> 41 XUVs
43376 XUVs total
13644 collection wire objects
39627 potential space points
Neighbour search...
4771509 tests to find 2179628 neighbours
Iterating with no regularization...
Begin: 1.0675e+08
0 8.12936e+07
1 7.98244e+07
2 7.96695e+07
3 7.96259e+07
Now with regularization...
Begin: 6.73534e+07
0 6.72794e+07
1 6.72435e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 411429299
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 411429299
fLifetime: 30000 [us]
30-Dec-2025 09:06:13 GMT  Closed output file "reco_stage1.root"
30-Dec-2025 09:06:13 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          221.169       284.797       350.609       271.681       41.5679        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.000763694   0.00303057    0.00520105    0.00311268    0.00129023       10     
reco:ophit:OpHitFinder                             0.219469      0.267222      0.303086      0.275341      0.027932        10     
reco:opflash:OpFlashFinder                         0.117634      0.172852      0.204779      0.187693      0.0295488       10     
reco:opslicer:OpSlicer                             0.409366      0.594059      0.730356      0.601439      0.0959027       10     
reco:caldata:DataPrepModule                         4.37358       4.5574        4.77039       4.56228      0.115376        10     
reco:wclsdatasp:WireCellToolkit                     30.0725       30.983        31.9066       30.8273      0.515076        10     
reco:gaushit:GausHitFinder                         0.834815        0.981        1.16856      0.940728      0.123644        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00039111    0.000460602   0.000584355   0.000464718   5.45762e-05      10     
reco:reco3d:SpacePointSolver                        8.60766       14.2195       28.0301       13.0652       5.13421        10     
reco:hitpdune:DisambigFromSpacePoints               1.04482       1.4141        1.79797       1.28339      0.272795        10     
reco:pandora:StandardPandora                        17.5671       26.4725       42.118        24.5174       8.11657        10     
reco:pandoraWriter:StandardPandora                 0.296182      0.366985      0.426301      0.359347      0.0449739       10     
reco:pandoraTrack:LArPandoraTrackCreation           1.88621       2.86023       3.69624       2.89428      0.541062        10     
reco:pandoraShower:LArPandoraShowerCreation         1.94822       3.22285       4.92066       3.07685      0.849616        10     
reco:pandoracalo:Calorimetry                        1.23621       1.73926       2.14085       1.7226       0.311976        10     
reco:pandoracalonosce:Calorimetry                   1.11266       1.5926        2.00333       1.61891      0.309572        10     
reco:pandorapid:Chi2ParticleID                    0.00339916    0.00440045    0.00533726    0.00450556    0.000497475      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0379112     0.0444279     0.0529412     0.0431232    0.00404801       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00353975    0.00428868    0.00489839    0.00430912    0.00043599       10     
reco:pandoraShowercalo:ShowerCalorimetry            1.96374       3.68779       5.50045       3.48895      0.989265        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       1.86727       3.57505       5.44458       3.3456       0.980751        10     
reco:emtrkmichelid:EmTrackMichelId                  122.584       152.291       177.306       146.333       17.6711        10     
reco:crttag:SingleCRTMatchingProducer             0.00860059     0.0108038     0.0131808    0.00986913     0.0016218       10     
reco:crtreco:TwoCRTMatchingProducer               0.00554315    0.00714662    0.00819534    0.00720589    0.00079174       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.687132      0.899666       1.07009      0.886315      0.102736        10     
reco:pandora2Track:LArPandoraTrackCreation          5.49257       9.6821        16.2476       8.95215       2.88535        10     
reco:pandora2calo:Calorimetry                       2.83526       5.24017       8.15926       4.91928       1.45437        10     
reco:pandora2calonosce:Calorimetry                  2.68636       4.94698       7.50657       4.67095       1.35394        10     
reco:pandora2pid:Chi2ParticleID                    0.0078607    0.00993073     0.0127785    0.00988374     0.0014177       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0456188     0.0540452     0.0634487     0.0531478    0.00572029       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID         0.00817471    0.00999026     0.0129142    0.00986123    0.00141559       10     
reco:pandora2Shower:LArPandoraShowerCreation        2.23306       3.48775       4.88599       3.31524      0.768239        10     
reco:pandora2Showercalo:ShowerCalorimetry           2.50964       4.54164       6.48661       4.31906       1.13821        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      2.30643       4.19675       5.99232       4.00428       1.03202        10     
[art]:TriggerResults:TriggerResultInserter        4.1408e-05    5.43948e-05   0.000110699   4.6683e-05    1.94181e-05      10     
end_path:out1:RootOutput                           9.638e-06    1.10868e-05   1.5038e-05    1.1051e-05    1.44712e-06      10     
end_path:out1:RootOutput(write)                     1.98412       2.55681       3.08588       2.46775       0.33364        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4960.96 MB
  Peak resident set size usage (VmHWM): 4158.84 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.19 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   2.88 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.03 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 10120400
-rw-r--r--. 1 dune004 dune 2376641607 Dec 30 08:17 detsim_stage2.root
-rw-r--r--. 1 dune004 dune 2255451307 Dec 30 08:15 detsim_stage1.root
-rw-r--r--. 1 dune004 dune 2077495299 Dec 30 09:06 reco_stage1.root
-rw-r--r--. 1 dune004 dune 1826555124 Dec 30 06:45 g4_stage2.root
-rw-r--r--. 1 dune004 dune 1748078422 Dec 30 06:34 g4_stage1.root
-rw-r--r--. 1 dune004 dune   78436400 Dec 30 09:02 Pandora_Events.pndr
-rw-r--r--. 1 dune004 dune     243088 Dec 30 09:06 jobscript.log
-rw-r--r--. 1 dune004 dune     217088 Dec 30 09:06 mem.db
-rw-r--r--. 1 dune004 dune      62376 Dec 30 09:06 Pandora_Geometry.xml
-rw-r--r--. 1 dune004 dune      36864 Dec 30 09:06 time.db
-rw-r--r--. 1 dune004 dune        519 Dec 30 08:17 detsim_single_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 30 06:45 g4_protoDUNE_hist.root
-rw-r--r--. 1 dune004 dune        519 Dec 30 09:06 protoDUNE_reco1_hist.root
-rw-r--r--. 1 dune004 dune         52 Dec 30 06:29 all-input-dids.txt
-rw-r--r--. 1 dune004 dune          7 Dec 30 09:06 justin-processed-pfns.txt
-rw-r--r--. 1 dune004 dune          0 Dec 30 08:21 ProtoDUNE.log
-rw-r--r--. 1 dune004 dune          0 Dec 30 08:17 debugprod.log
justIN time: 2026-02-07 00:50:23 UTC       justIN version: 01.06.00