justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 242834.0@dunegpschedd01.fnal.gov

Jobsub ID242834.0@dunegpschedd01.fnal.gov
Workflow ID9581
Stage ID1
User namepmelas@fnal.gov
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-04 07:51:50
SiteUK_QMUL
EntryDUNE_UK_London_QMUL_arcce02
Last heartbeat2025-11-04 12:27:38
From worker nodeHostnamecn009.htc.esc.qmul
cpuinfoIntel(R) Xeon(R) Gold 6248R CPU @ 3.00GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-04 07:53:57
Input filesmonte-carlo-009581-000001
JobscriptExit code0
Real time4h (16203s)
CPU time4h (15957s = 98%)
Max RSS bytes4462030848 (4255 MiB)
Outputting started2025-11-04 12:24:00
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09581/1/001/reco_stage1.root
Finished2025-11-04 12:27:38
Saved logsjustin-logs:242834.0-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

oAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 171687 traces, 0 summary
Retagger: tagging trace set: gauss with 129533 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 129533 traces tagged "gauss"
FrameSaver: q=4.48356e+06 n=2280412 tag=gauss
wclsFrameSaver: saving 171687 traces tagged "wiener"
FrameSaver: q=4.75083e+06 n=2009467 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 54156 XUs and 63709 XVs -> 828 XUVs
C:0 T:1 143721 XUs and 159382 XVs -> 11536 XUVs
C:0 T:2 239069 XUs and 206948 XVs -> 36254 XUVs
C:0 T:3 62500 XUs and 62261 XVs -> 703 XUVs
C:0 T:4 8299 XUs and 14738 XVs -> 95 XUVs
C:0 T:5 43319 XUs and 74606 XVs -> 6168 XUVs
C:0 T:6 146023 XUs and 115426 XVs -> 13287 XUVs
C:0 T:7 48821 XUs and 38353 XVs -> 432 XUVs
C:0 T:8 9637 XUs and 11814 XVs -> 178 XUVs
C:0 T:9 69146 XUs and 195169 XVs -> 10351 XUVs
C:0 T:10 63562 XUs and 33412 XVs -> 10354 XUVs
C:0 T:11 75 XUs and 43 XVs -> 6 XUVs
90192 XUVs total
15996 collection wire objects
82317 potential space points
Neighbour search...
20803091 tests to find 8337296 neighbours
Iterating with no regularization...
Begin: 1.03532e+08
0 7.64504e+07
1 7.51526e+07
2 7.49775e+07
3 7.49223e+07
Now with regularization...
Begin: 6.272e+07
0 6.26462e+07
1 6.25959e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410304014
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410304014
fLifetime: 30000 [us]
04-Nov-2025 12:23:57 GMT  Closed output file "reco_stage1.root"
04-Nov-2025 12:23:57 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          447.869       737.322       1083.02       727.475       177.514        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00144338     0.0039516    0.00664861    0.00427328    0.00153145       10     
reco:ophit:OpHitFinder                             0.253536      0.395045      0.512799      0.419183      0.0859269       10     
reco:opflash:OpFlashFinder                         0.190734      0.314368       0.41401      0.328237      0.0625561       10     
reco:opslicer:OpSlicer                              0.52838      0.965355       1.22883       1.00268      0.223316        10     
reco:caldata:DataPrepModule                         5.09009       7.50069       9.42051       7.65229       1.39592        10     
reco:wclsdatasp:WireCellToolkit                     49.1578       56.3119       64.6858       56.2258       4.82878        10     
reco:gaushit:GausHitFinder                          1.39658       2.04308       2.97532       2.05166       0.44745        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000767167   0.00116639    0.00170146     0.0010853    0.000305117      10     
reco:reco3d:SpacePointSolver                        10.971        27.983        51.6106       26.4223       11.5057        10     
reco:hitpdune:DisambigFromSpacePoints               1.85284       4.15655       7.53208       3.75749       1.93323        10     
reco:pandora:StandardPandora                        31.0429       83.2351       195.839       63.1469       48.3008        10     
reco:pandoraWriter:StandardPandora                 0.488817      0.759907       1.04373      0.724322      0.226026        10     
reco:pandoraTrack:LArPandoraTrackCreation           7.67557       17.6285       27.1408       14.8404       6.59225        10     
reco:pandoraShower:LArPandoraShowerCreation         5.82641       21.0969       36.1567       17.1867       8.45434        10     
reco:pandoracalo:Calorimetry                        2.78213       8.68599       13.8995       7.99659       3.58046        10     
reco:pandoracalonosce:Calorimetry                   2.58861       8.18391       13.0173       7.72278       3.27474        10     
reco:pandorapid:Chi2ParticleID                    0.00449201    0.00969585     0.0170246    0.00947485    0.00367349       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0458465     0.0947805     0.176044      0.0887502     0.0348476       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00434488    0.00904533     0.0160502    0.00911362    0.00325186       10     
reco:pandoraShowercalo:ShowerCalorimetry            4.67642       21.4001       40.141        18.239        9.74827        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       4.57553       20.6038       38.9033       17.4294       9.65583        10     
reco:emtrkmichelid:EmTrackMichelId                  211.362       277.69        363.687       279.797       43.9632        10     
reco:crttag:SingleCRTMatchingProducer              0.0353262     0.0435191     0.0576162     0.0429566    0.00644888       10     
reco:crtreco:TwoCRTMatchingProducer                0.0295139     0.0355485     0.0482775     0.0345558    0.00556319       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.24143       1.60471       1.94532       1.60704      0.224869        10     
reco:pandora2Track:LArPandoraTrackCreation          25.0445       56.0936       73.7153       59.0101       13.926         10     
reco:pandora2calo:Calorimetry                       10.7241       27.0657       42.1396       27.6692       9.1053         10     
reco:pandora2calonosce:Calorimetry                  10.2281       26.3469       44.6387       26.7335       9.48261        10     
reco:pandora2pid:Chi2ParticleID                    0.0147111     0.0232761     0.0326171     0.0230812    0.00530057       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0792453     0.123296       0.15972      0.121804      0.0248824       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0144444     0.022776      0.030469      0.0219447    0.00510766       10     
reco:pandora2Shower:LArPandoraShowerCreation        8.79219       18.1561       27.4694       18.2024       4.84935        10     
reco:pandora2Showercalo:ShowerCalorimetry           9.34995       23.6714       39.6893       23.1075       7.57283        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      8.46444       20.8383       35.227        20.1325       7.0216         10     
[art]:TriggerResults:TriggerResultInserter        4.5893e-05    8.57694e-05   0.000151431   6.6591e-05    3.34545e-05      10     
end_path:out1:RootOutput                           6.514e-06    1.07079e-05   1.6646e-05     8.962e-06    3.39462e-06      10     
end_path:out1:RootOutput(write)                     2.36808       4.01195       5.94017       3.87624      0.942274        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5267.96 MB
  Peak resident set size usage (VmHWM): 4462.03 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.28 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.76 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.24 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11288860
-rw-r--r-- 1 pildune22 pildune 2612100114 Nov  4 10:18 detsim_stage2.root
-rw-r--r-- 1 pildune22 pildune 2473978745 Nov  4 10:15 detsim_stage1.root
-rw-r--r-- 1 pildune22 pildune 2344129859 Nov  4 12:23 reco_stage1.root
-rw-r--r-- 1 pildune22 pildune 2058919307 Nov  4 08:21 g4_stage2.root
-rw-r--r-- 1 pildune22 pildune 1984133780 Nov  4 08:01 g4_stage1.root
-rw-r--r-- 1 pildune22 pildune   85920800 Nov  4 12:13 Pandora_Events.pndr
-rw-r--r-- 1 pildune22 pildune     243063 Nov  4 12:24 jobscript.log
-rw-r--r-- 1 pildune22 pildune     217088 Nov  4 12:23 mem.db
-rw-r--r-- 1 pildune22 pildune      62376 Nov  4 12:23 Pandora_Geometry.xml
-rw-r--r-- 1 pildune22 pildune      36864 Nov  4 12:23 time.db
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 10:18 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 08:21 g4_protoDUNE_hist.root
-rw-r--r-- 1 pildune22 pildune        519 Nov  4 12:23 protoDUNE_reco1_hist.root
-rw-r--r-- 1 pildune22 pildune         52 Nov  4 07:53 all-input-dids.txt
-rw-r--r-- 1 pildune22 pildune          7 Nov  4 12:24 justin-processed-pfns.txt
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 10:33 ProtoDUNE.log
-rw-r--r-- 1 pildune22 pildune          0 Nov  4 10:18 debugprod.log
justIN time: 2026-02-11 10:44:36 UTC       justIN version: 01.06.00