justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 241997.1@dunegpschedd01.fnal.gov

Jobsub ID241997.1@dunegpschedd01.fnal.gov
Workflow ID9529
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-11-02 14:19:37
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-11-02 21:30:17
From worker nodeHostnamewn1208080.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-11-02 14:20:30
Input filesmonte-carlo-009529-000001
JobscriptExit code0
Real time7h (25662s)
CPU time6h (24717s = 96%)
Max RSS bytes4543959040 (4333 MiB)
Outputting started2025-11-02 21:28:12
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09529/1/001/reco_stage1.root
Finished2025-11-02 21:30:17
Saved logsjustin-logs:241997.1-dunegpschedd01.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

nder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 150371 traces, 0 summary
Retagger: tagging trace set: gauss with 111816 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 111816 traces tagged "gauss"
FrameSaver: q=4.00853e+06 n=1988523 tag=gauss
wclsFrameSaver: saving 150371 traces tagged "wiener"
FrameSaver: q=4.26152e+06 n=1756383 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 31668 XUs and 23534 XVs -> 199 XUVs
C:0 T:1 47357 XUs and 40464 XVs -> 3670 XUVs
C:0 T:2 224953 XUs and 202990 XVs -> 31322 XUVs
C:0 T:3 73149 XUs and 61773 XVs -> 866 XUVs
C:0 T:4 12928 XUs and 12215 XVs -> 151 XUVs
C:0 T:5 44742 XUs and 30431 XVs -> 3982 XUVs
C:0 T:6 187456 XUs and 174413 XVs -> 9377 XUVs
C:0 T:7 67310 XUs and 61474 XVs -> 1304 XUVs
C:0 T:8 9864 XUs and 8976 XVs -> 66 XUVs
C:0 T:9 60186 XUs and 71761 XVs -> 5058 XUVs
C:0 T:10 36946 XUs and 39085 XVs -> 8080 XUVs
C:0 T:11 201 XUs and 172 XVs -> 29 XUVs
64104 XUVs total
14271 collection wire objects
57871 potential space points
Neighbour search...
11247785 tests to find 4797234 neighbours
Iterating with no regularization...
Begin: 8.28693e+07
0 6.83458e+07
1 6.71124e+07
2 6.69398e+07
3 6.68887e+07
Now with regularization...
Begin: 5.47781e+07
0 5.47205e+07
1 5.46801e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410269021
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410269021
fLifetime: 30000 [us]
02-Nov-2025 21:28:08 GMT  Closed output file "reco_stage1.root"
02-Nov-2025 21:28:08 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          609.05        958.321       1210.79       1005.15       173.355        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00231033    0.00751407     0.0167859    0.00680787     0.0040014       10     
reco:ophit:OpHitFinder                             0.444812      0.503905      0.614881      0.488774      0.0486342       10     
reco:opflash:OpFlashFinder                         0.260553      0.333099      0.420653      0.332485      0.0424274       10     
reco:opslicer:OpSlicer                              1.16179       1.4247        1.66778       1.3803       0.164436        10     
reco:caldata:DataPrepModule                         8.43256       8.97364       10.0525       8.97635      0.400683        10     
reco:wclsdatasp:WireCellToolkit                     64.309        68.3552       78.1148       67.6913       3.62424        10     
reco:gaushit:GausHitFinder                          1.84348       2.54362       3.03677       2.63521      0.350207        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00118442    0.00175565    0.00245659    0.00178788    0.000439892      10     
reco:reco3d:SpacePointSolver                        13.9749       37.2523       70.6449       34.6053       15.4627        10     
reco:hitpdune:DisambigFromSpacePoints               2.09307       3.91932       5.75925       4.04923       1.02697        10     
reco:pandora:StandardPandora                        38.4617       101.907       169.952       113.552       38.7651        10     
reco:pandoraWriter:StandardPandora                 0.841569       1.07714       1.32845       1.08183      0.136499        10     
reco:pandoraTrack:LArPandoraTrackCreation           8.96662       17.307        22.4879       18.1265       3.95374        10     
reco:pandoraShower:LArPandoraShowerCreation         10.9334       21.8876       31.2818       22.4265       6.16664        10     
reco:pandoracalo:Calorimetry                        6.09711       10.8604        13.76        11.2947       2.28198        10     
reco:pandoracalonosce:Calorimetry                   5.41191       10.1357       12.8319       10.2268       2.24945        10     
reco:pandorapid:Chi2ParticleID                     0.0107316     0.0144444     0.0239269     0.0142677    0.00361286       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0890999      0.12253      0.164951      0.124857      0.0200879       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00992615     0.0131487     0.0161455     0.0137074    0.00188394       10     
reco:pandoraShowercalo:ShowerCalorimetry            12.4328       25.3697       34.1861       25.6632       6.98874        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       12.2843       24.2123       32.9664       24.7311       6.23349        10     
reco:emtrkmichelid:EmTrackMichelId                  316.471       426.665       523.046       434.913       57.3428        10     
reco:crttag:SingleCRTMatchingProducer              0.031475      0.0455275     0.0719516     0.0405626     0.0127216       10     
reco:crtreco:TwoCRTMatchingProducer                0.0254928     0.0330158     0.0389189     0.0339245    0.00396152       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.36819       1.68918       2.22416       1.69692      0.242565        10     
reco:pandora2Track:LArPandoraTrackCreation          28.562        54.6384       68.3597       55.2825       12.174         10     
reco:pandora2calo:Calorimetry                       15.0289       33.8334       43.0807       36.7729       8.78287        10     
reco:pandora2calonosce:Calorimetry                  14.2477       31.8238       41.793        33.9131       7.64283        10     
reco:pandora2pid:Chi2ParticleID                    0.0220611     0.0319158     0.0394778     0.0331536    0.00489069       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.122365      0.157539      0.185479      0.163926      0.0197666       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0212874     0.0311477     0.0368241     0.0321674    0.00464173       10     
reco:pandora2Shower:LArPandoraShowerCreation        11.2354       17.7883       21.3452       19.1622       3.27199        10     
reco:pandora2Showercalo:ShowerCalorimetry           13.9073       25.8166       34.2254       26.4319       5.42081        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      12.967        23.5301       29.5383       25.8866       4.97757        10     
[art]:TriggerResults:TriggerResultInserter        4.7198e-05    6.81533e-05   0.000198945   5.3055e-05    4.38593e-05      10     
end_path:out1:RootOutput                           7.454e-06    9.0121e-06     1.592e-05      8.2e-06     2.38592e-06      10     
end_path:out1:RootOutput(write)                     4.24869       5.70038       7.02762       5.75042      0.809603        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5401.04 MB
  Peak resident set size usage (VmHWM): 4543.96 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.11 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.38 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   5.71 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.22 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11506240
-rw-r--r-- 1 dune01 dune01 2648314691 Nov  2 18:44 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2509893012 Nov  2 18:40 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2382501846 Nov  2 21:28 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2122497420 Nov  2 15:00 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 2032403931 Nov  2 14:31 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   86149820 Nov  2 21:15 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243032 Nov  2 21:28 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Nov  2 21:28 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Nov  2 21:28 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Nov  2 21:28 time.db
-rw-r--r-- 1 dune01 dune01        519 Nov  2 18:44 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 15:00 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Nov  2 21:28 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Nov  2 14:20 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Nov  2 21:28 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Nov  2 19:01 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Nov  2 18:44 debugprod.log
justIN time: 2025-11-05 04:54:40 UTC       justIN version: 01.05.01