justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242832.0@dunegpschedd01.fnal.gov

Jobsub ID242832.0@dunegpschedd01.fnal.gov
Workflow ID9579
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-04 07:51:49
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-11-04 15:54:15
From worker nodeHostnamewn1205240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 07:53:32
Input filesmonte-carlo-009579-000001
JobscriptExit code0
Real time7h (28698s)
CPU time7h (27554s = 96%)
Max RSS bytes4651982848 (4436 MiB)
Outputting started2025-11-04 15:51:51
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09579/1/001/reco_stage1.root
Finished2025-11-04 15:54:15
Saved logsjustin-logs:242832.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lSignalFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 166416 traces, 0 summary
Retagger: tagging trace set: gauss with 124649 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 124649 traces tagged "gauss"
FrameSaver: q=4.25082e+06 n=2181048 tag=gauss
wclsFrameSaver: saving 166416 traces tagged "wiener"
FrameSaver: q=4.54177e+06 n=1919194 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 25116 XUs and 23820 XVs -> 1259 XUVs
C:0 T:1 40363 XUs and 36513 XVs -> 5316 XUVs
C:0 T:2 143869 XUs and 146242 XVs -> 7484 XUVs
C:0 T:3 59579 XUs and 52594 XVs -> 1218 XUVs
C:0 T:4 24347 XUs and 15539 XVs -> 1854 XUVs
C:0 T:5 45861 XUs and 30754 XVs -> 3978 XUVs
C:0 T:6 226270 XUs and 186562 XVs -> 11388 XUVs
C:0 T:7 77961 XUs and 68611 XVs -> 813 XUVs
C:0 T:8 14677 XUs and 11767 XVs -> 220 XUVs
C:0 T:9 127033 XUs and 125752 XVs -> 13230 XUVs
C:0 T:10 83047 XUs and 17948 XVs -> 8501 XUVs
C:0 T:11 120 XUs and 67 XVs -> 14 XUVs
55275 XUVs total
15250 collection wire objects
50081 potential space points
Neighbour search...
5946213 tests to find 2763780 neighbours
Iterating with no regularization...
Begin: 8.68679e+07
0 7.18502e+07
1 7.04854e+07
2 7.02754e+07
3 7.02177e+07
Now with regularization...
Begin: 5.6697e+07
0 5.66528e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410304038
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410304038
fLifetime: 30000 [us]
04-Nov-2025 15:51:47 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 15:51:47 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          826.95        1060.31       1314.59       1024.62       170.656        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00196304     0.0100065     0.0188024    0.00871287    0.00528928       10     
reco:ophit:OpHitFinder                             0.490779      0.552842       0.62594      0.546561      0.0456249       10     
reco:opflash:OpFlashFinder                         0.325206      0.410098       0.55361      0.398714      0.0661459       10     
reco:opslicer:OpSlicer                              1.30528       1.56585       1.86207       1.57155      0.195042        10     
reco:caldata:DataPrepModule                         8.85418       9.43785       10.295        9.39418      0.359616        10     
reco:wclsdatasp:WireCellToolkit                     57.7983       68.4006       76.3423       69.0965       6.1414         10     
reco:gaushit:GausHitFinder                          2.18261       2.78632       3.45335       2.7624       0.379084        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00102422    0.00170884     0.0022611    0.00185998    0.000404737      10     
reco:reco3d:SpacePointSolver                        23.9236       46.1065       71.4838       48.0434       14.2823        10     
reco:hitpdune:DisambigFromSpacePoints               3.37277       4.91468       6.89578       4.55881        1.111         10     
reco:pandora:StandardPandora                        70.0186       115.062       176.745       110.442       35.1216        10     
reco:pandoraWriter:StandardPandora                 0.952575       1.17243       1.35111       1.16981      0.143892        10     
reco:pandoraTrack:LArPandoraTrackCreation           13.1465       18.0972       23.7785       16.1307       3.60107        10     
reco:pandoraShower:LArPandoraShowerCreation         15.2228       23.2142       32.6985       22.525        6.17065        10     
reco:pandoracalo:Calorimetry                        7.76636       10.4397       14.2097       10.0525       2.01776        10     
reco:pandoracalonosce:Calorimetry                   6.64165       10.1974       13.0906       9.49726       2.0651         10     
reco:pandorapid:Chi2ParticleID                     0.0111623     0.0141022     0.0177979     0.0141317    0.00219721       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.102305      0.138964      0.183616      0.134725      0.0230127       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID           0.0106469      0.01355      0.0153966     0.0138231    0.00139102       10     
reco:pandoraShowercalo:ShowerCalorimetry            18.3975       26.5987       38.644        26.8386       6.53001        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       19.3093       26.4762       37.2033       25.9059       6.38057        10     
reco:emtrkmichelid:EmTrackMichelId                  406.818       489.623       574.711       482.305       59.0064        10     
reco:crttag:SingleCRTMatchingProducer              0.0377934     0.055604      0.0785275     0.0549176     0.0127984       10     
reco:crtreco:TwoCRTMatchingProducer                0.0283343     0.0362757     0.0438335     0.0352952    0.00560887       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.57253       1.93609       2.42402       1.89851      0.233166        10     
reco:pandora2Track:LArPandoraTrackCreation          42.034        58.5005       80.3998       57.513        13.5905        10     
reco:pandora2calo:Calorimetry                       26.0498       34.981        47.7562       34.4279       7.50958        10     
reco:pandora2calonosce:Calorimetry                  22.4318       31.6917       41.9283       30.0528       7.13114        10     
reco:pandora2pid:Chi2ParticleID                    0.027599      0.0353599     0.0467245     0.0323339    0.00641078       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.143912      0.197325      0.327058      0.173164      0.0603583       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0280564     0.0343959     0.0436631     0.0322116    0.00579364       10     
reco:pandora2Shower:LArPandoraShowerCreation        14.1148       19.142        27.0345       18.6762       4.45853        10     
reco:pandora2Showercalo:ShowerCalorimetry           20.8214       27.734        36.3531       26.3813       5.73974        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      17.6124       23.9392       33.7413       20.4397       6.21758        10     
[art]:TriggerResults:TriggerResultInserter        4.5326e-05    6.73903e-05   0.000189037   5.4734e-05    4.09325e-05      10     
end_path:out1:RootOutput                           6.783e-06    8.6041e-06    1.6181e-05     7.704e-06    2.58042e-06      10     
end_path:out1:RootOutput(write)                     5.27462       6.45457       7.86173       6.31431      0.861512        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5516.75 MB
  Peak resident set size usage (VmHWM): 4651.98 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.39 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   6.08 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.23 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 12408540
-rw-r--r-- 1 dune01 dune01 2837940157 Nov  4 12:50 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2691216141 Nov  4 12:45 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2599893877 Nov  4 15:51 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2295682517 Nov  4 08:38 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2187940638 Nov  4 08:06 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   93040440 Nov  4 15:37 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     242986 Nov  4 15:51 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  4 15:51 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  4 15:51 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  4 15:51 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  4 12:50 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  4 08:38 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  4 15:51 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  4 07:53 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  4 15:51 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  4 13:08 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  4 12:50 debugprod.log
justIN time: 2026-02-11 10:43:47 UTC       justIN version: 01.06.00