justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 237055.0@dunegpschedd01.fnal.gov

Jobsub ID237055.0@dunegpschedd01.fnal.gov
Workflow ID9228
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-21 08:08:53
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-10-21 12:32:54
From worker nodeHostnamewn2207290.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-21 08:09:26
Input filesmonte-carlo-009228-000001
JobscriptExit code0
Real time4h (15680s)
CPU time4h (15171s = 96%)
Max RSS bytes4482416640 (4274 MiB)
Outputting started2025-10-21 12:30:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09228/1/001/reco_stage1.root
Finished2025-10-21 12:32:54
Saved logsjustin-logs:237055.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

tool adcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 132436 traces, 0 summary
Retagger: tagging trace set: gauss with 96981 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 96981 traces tagged "gauss"
FrameSaver: q=3.14261e+06 n=1646415 tag=gauss
wclsFrameSaver: saving 132436 traces tagged "wiener"
FrameSaver: q=3.35601e+06 n=1416394 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 32934 XUs and 24998 XVs -> 344 XUVs
C:0 T:1 81411 XUs and 68156 XVs -> 8170 XUVs
C:0 T:2 39879 XUs and 45788 XVs -> 2949 XUVs
C:0 T:3 20639 XUs and 18063 XVs -> 71 XUVs
C:0 T:4 11602 XUs and 10778 XVs -> 181 XUVs
C:0 T:5 61688 XUs and 41752 XVs -> 9106 XUVs
C:0 T:6 113328 XUs and 122057 XVs -> 5154 XUVs
C:0 T:8 9955 XUs and 5715 XVs -> 92 XUVs
C:0 T:9 100004 XUs and 56890 XVs -> 5924 XUVs
C:0 T:10 61687 XUs and 49930 XVs -> 6678 XUVs
C:0 T:11 152 XUs and 149 XVs -> 11 XUVs
38680 XUVs total
13647 collection wire objects
35652 potential space points
Neighbour search...
3541436 tests to find 1782590 neighbours
Iterating with no regularization...
Begin: 3.9527e+07
0 3.06037e+07
1 2.99255e+07
2 2.98162e+07
3 2.97837e+07
4 2.97689e+07
Now with regularization...
Begin: 2.10191e+07
0 2.09755e+07
1 2.09502e+07
2 2.09325e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410021590
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410021590
fLifetime: 30000 [us]
21-Oct-2025 13:30:44 BST  Closed output file "reco_stage1.root"
21-Oct-2025 13:30:44 BST  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          420.703       579.576       750.607       550.625       94.2553        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00170733    0.00721371     0.0177498    0.00634066    0.00436734       10     
reco:ophit:OpHitFinder                             0.372455      0.432094      0.527079       0.43453      0.0495114       10     
reco:opflash:OpFlashFinder                          0.19232       0.26445      0.372458      0.257646      0.0479371       10     
reco:opslicer:OpSlicer                             0.863677       1.02118       1.21568       1.03894      0.117157        10     
reco:caldata:DataPrepModule                         5.98575       7.29628       8.16149       7.42815      0.588824        10     
reco:wclsdatasp:WireCellToolkit                     41.498        44.0656       47.2682       43.6663       1.63147        10     
reco:gaushit:GausHitFinder                          1.27673       1.94923       2.41015       1.96058       0.29804        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000551337   0.00133879    0.00321787    0.00121277    0.000738646      10     
reco:reco3d:SpacePointSolver                        13.5272       28.822        45.2067       26.437        9.48731        10     
reco:hitpdune:DisambigFromSpacePoints               1.81152       2.60012       3.5781        2.25752      0.666073        10     
reco:pandora:StandardPandora                        28.4164       64.2334       109.128       63.365        23.9662        10     
reco:pandoraWriter:StandardPandora                 0.416081      0.701559      0.868137      0.743004       0.12698        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.82958       10.6358       14.3087       10.7635       2.49875        10     
reco:pandoraShower:LArPandoraShowerCreation         7.49682       13.4185       17.0138       14.1334       2.37661        10     
reco:pandoracalo:Calorimetry                        4.16513       6.59412       9.31564       6.09133       1.63385        10     
reco:pandoracalonosce:Calorimetry                   3.97099       6.15779       9.72263       5.75204       1.66587        10     
reco:pandorapid:Chi2ParticleID                    0.00597057    0.00954214     0.0144176    0.00892709    0.00297718       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0498034     0.0805862     0.115916      0.088083      0.0217372       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00544961    0.00907249     0.0142675    0.00899025    0.00266819       10     
reco:pandoraShowercalo:ShowerCalorimetry            7.7282        14.2266       21.9811       13.9678       3.45074        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       7.36153       13.4379       18.2639       12.9001       3.02482        10     
reco:emtrkmichelid:EmTrackMichelId                  213.215       255.887       299.285       256.112       26.8304        10     
reco:crttag:SingleCRTMatchingProducer              0.0195191     0.0286845     0.0397509     0.0278085    0.00524108       10     
reco:crtreco:TwoCRTMatchingProducer                0.0152717     0.0184261     0.0250997     0.0174405    0.00287833       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.812231       1.16264       1.37205       1.21906      0.181191        10     
reco:pandora2Track:LArPandoraTrackCreation          19.0348       31.4397       51.8439       27.9488       10.9289        10     
reco:pandora2calo:Calorimetry                       10.243        17.033        28.4548       15.9056       5.74768        10     
reco:pandora2calonosce:Calorimetry                  9.70775       16.4699       26.5427       14.9201       5.23827        10     
reco:pandora2pid:Chi2ParticleID                    0.0120797     0.0206816     0.0328586     0.0187907    0.00693863       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.056014      0.100794      0.138485      0.106598      0.0278661       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0111896     0.0215657     0.031396      0.0218708    0.00705211       10     
reco:pandora2Shower:LArPandoraShowerCreation        7.69302       11.2337       16.2125       10.5392       2.78287        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.41967       13.5815       19.5567       13.7095       3.41955        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      7.41325       12.1243       17.8418       12.6793       3.25031        10     
[art]:TriggerResults:TriggerResultInserter        6.1976e-05    0.000104021   0.000174148   9.09305e-05   3.15033e-05      10     
end_path:out1:RootOutput                           1.085e-05    1.45872e-05   1.9977e-05    1.40615e-05   2.82209e-06      10     
end_path:out1:RootOutput(write)                     3.53012       4.24563       5.36726       4.13287      0.548861        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5328.33 MB
  Peak resident set size usage (VmHWM): 4482.42 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.15 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.34 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.44 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11311168
-rw-r--r-- 1 dune01 dune01 2621065044 Oct 21 11:51 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2484173430 Oct 21 11:48 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2359026623 Oct 21 13:30 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2063489187 Oct 21 09:33 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1966479071 Oct 21 09:17 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87771330 Oct 21 13:25 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243114 Oct 21 13:30 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 21 13:30 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 21 13:30 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 21 13:30 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 21 11:51 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 21 09:33 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 21 13:30 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 21 09:09 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 21 13:30 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 21 12:00 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 21 11:51 debugprod.log
justIN time: 2025-11-04 01:47:24 UTC       justIN version: 01.05.01