justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 235991.1@dunegpschedd02.fnal.gov

Jobsub ID235991.1@dunegpschedd02.fnal.gov
Workflow ID9508
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-01 09:36:05
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-01 14:33:10
From worker nodeHostnamewn2207301.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-01 09:37:06
Input filesmonte-carlo-009508-000001
JobscriptExit code0
Real time4h (17672s)
CPU time4h (17039s = 96%)
Max RSS bytes4536131584 (4325 MiB)
Outputting started2025-11-01 14:31:39
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09508/1/001/reco_stage1.root
Finished2025-11-01 14:33:10
Saved logsjustin-logs:235991.1-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

dcKeepAllSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 160160 traces, 0 summary
Retagger: tagging trace set: gauss with 126878 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 126878 traces tagged "gauss"
FrameSaver: q=4.17697e+06 n=2250161 tag=gauss
wclsFrameSaver: saving 160160 traces tagged "wiener"
FrameSaver: q=4.38059e+06 n=2002308 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 38400 XUs and 47445 XVs -> 758 XUVs
C:0 T:1 86902 XUs and 92127 XVs -> 10685 XUVs
C:0 T:2 103414 XUs and 122678 XVs -> 3759 XUVs
C:0 T:3 51825 XUs and 52230 XVs -> 375 XUVs
C:0 T:4 19782 XUs and 19480 XVs -> 245 XUVs
C:0 T:5 88821 XUs and 75532 XVs -> 6174 XUVs
C:0 T:6 80671 XUs and 85790 XVs -> 2892 XUVs
C:0 T:7 34502 XUs and 35751 XVs -> 290 XUVs
C:0 T:8 10706 XUs and 10912 XVs -> 114 XUVs
C:0 T:9 61543 XUs and 83232 XVs -> 5350 XUVs
C:0 T:10 31633 XUs and 20929 XVs -> 2839 XUVs
C:0 T:11 139 XUs and 128 XVs -> 7 XUVs
33488 XUVs total
13953 collection wire objects
30439 potential space points
Neighbour search...
1684357 tests to find 887186 neighbours
Iterating with no regularization...
Begin: 7.92899e+07
0 7.02191e+07
1 6.96148e+07
2 6.95337e+07
3 6.95131e+07
Now with regularization...
Begin: 5.81019e+07
0 5.80815e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410245094
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410245094
fLifetime: 30000 [us]
01-Nov-2025 14:31:37 GMT  Closed output file "reco_stage1.root"
01-Nov-2025 14:31:37 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          369.679       520.861       631.119       493.963       89.1303        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00178761     0.0052306    0.00791854    0.00595732    0.00200396       10     
reco:ophit:OpHitFinder                             0.342457      0.458443      0.540551      0.454865      0.0573794       10     
reco:opflash:OpFlashFinder                         0.242468      0.287892      0.388722      0.280993      0.039027        10     
reco:opslicer:OpSlicer                             0.688714      0.982868       1.23177       0.9879       0.131322        10     
reco:caldata:DataPrepModule                         6.04462       7.05333       8.47719       6.87123      0.711527        10     
reco:wclsdatasp:WireCellToolkit                     39.5402       46.1523       56.6669       44.6341       5.02831        10     
reco:gaushit:GausHitFinder                          1.27394       1.91104       2.65177       1.85874      0.375277        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000540484   0.000838074    0.0012293    0.000832437   0.000199546      10     
reco:reco3d:SpacePointSolver                        11.5831       27.6052       40.6615       29.3981       8.6081         10     
reco:hitpdune:DisambigFromSpacePoints               1.79689       2.78827       3.93627       2.73566      0.716026        10     
reco:pandora:StandardPandora                        34.7879       57.1261       102.577       51.4542       19.694         10     
reco:pandoraWriter:StandardPandora                 0.440171      0.693591      0.890852      0.686808      0.128344        10     
reco:pandoraTrack:LArPandoraTrackCreation           4.66967       7.51569       10.3402       7.23746       1.76651        10     
reco:pandoraShower:LArPandoraShowerCreation         4.41917       8.48587       11.4736       9.54821       2.53177        10     
reco:pandoracalo:Calorimetry                        2.93549       4.56749       6.1012        4.79597       1.07878        10     
reco:pandoracalonosce:Calorimetry                   2.48355       4.22676       6.66514       4.5055        1.32063        10     
reco:pandorapid:Chi2ParticleID                    0.00529604    0.00836831     0.0135484    0.00791198    0.00275821       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0467359     0.0804982     0.118786      0.0717832     0.0249546       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00485833    0.00822104     0.012105     0.00864017    0.00214172       10     
reco:pandoraShowercalo:ShowerCalorimetry            5.73526       9.64347       13.8088       10.3122       2.52991        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.69036       9.32478       12.7849       10.207        2.61323        10     
reco:emtrkmichelid:EmTrackMichelId                  196.705       255.374       306.208       251.626       36.1865        10     
reco:crttag:SingleCRTMatchingProducer              0.0142194     0.0208461     0.0294835     0.0205436    0.00420559       10     
reco:crtreco:TwoCRTMatchingProducer               0.00862755     0.0151987     0.0233451     0.0138353    0.00480504       10     
reco:anodepiercerst0:T0RecoAnodePiercers           0.824489       1.14794       1.40156       1.16424      0.159959        10     
reco:pandora2Track:LArPandoraTrackCreation          12.5398       22.5418       32.0368       21.6006       6.1131         10     
reco:pandora2calo:Calorimetry                       7.49158       11.4952       17.9449       10.6223       3.17578        10     
reco:pandora2calonosce:Calorimetry                  7.30689       11.2836       16.9934       10.1441        2.895         10     
reco:pandora2pid:Chi2ParticleID                    0.0113096     0.0209682     0.0280774     0.0218206    0.00508244       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0539929     0.0999991     0.139268      0.0984438     0.025387        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0109546     0.0192022     0.0269523     0.0193959    0.00538797       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.57094       7.98367       12.2741       6.78101       2.40213        10     
reco:pandora2Showercalo:ShowerCalorimetry           5.94069       9.38449       14.2906       8.51091       2.68784        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      6.04494       8.00367       11.9878       7.4071        1.89876        10     
[art]:TriggerResults:TriggerResultInserter        5.9862e-05    9.5923e-05    0.000178805   8.85865e-05   3.43918e-05      10     
end_path:out1:RootOutput                           8.787e-06    1.19635e-05   1.6811e-05    1.1857e-05    2.3343e-06       10     
end_path:out1:RootOutput(write)                     3.41566       4.34972       5.14361        4.248       0.567884        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5359.01 MB
  Peak resident set size usage (VmHWM): 4536.13 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.33 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.05 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.36 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.18 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11386260
-rw-r--r-- 1 dune01 dune01 2629094149 Nov  1 13:02 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2490238913 Nov  1 12:59 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2371176350 Nov  1 14:31 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2078605009 Nov  1 10:05 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2002068642 Nov  1 09:46 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87718860 Nov  1 14:25 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243133 Nov  1 14:31 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  1 14:31 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  1 14:31 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  1 14:31 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  1 13:02 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  1 10:05 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  1 14:31 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  1 09:37 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  1 14:31 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  1 13:10 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  1 13:02 debugprod.log
justIN time: 2025-11-03 18:30:18 UTC       justIN version: 01.05.01