justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 235484.0@dunegpschedd02.fnal.gov

Jobsub ID235484.0@dunegpschedd02.fnal.gov
Workflow ID9486
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-30 18:27:59
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-10-30 23:02:55
From worker nodeHostnamewn2204240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-30 18:29:52
Input filesmonte-carlo-009486-000001
JobscriptExit code0
Real time4h (16281s)
CPU time4h (15533s = 95%)
Max RSS bytes4462817280 (4256 MiB)
Outputting started2025-10-30 23:01:14
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09486/1/001/reco_stage1.root
Finished2025-10-30 23:02:55
Saved logsjustin-logs:235484.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

lFinder
ToolBasedRawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 134223 traces, 0 summary
Retagger: tagging trace set: gauss with 101720 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 101720 traces tagged "gauss"
FrameSaver: q=3.66468e+06 n=1824734 tag=gauss
wclsFrameSaver: saving 134223 traces tagged "wiener"
FrameSaver: q=3.88986e+06 n=1619479 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 39231 XUs and 29238 XVs -> 473 XUVs
C:0 T:1 75754 XUs and 57920 XVs -> 5893 XUVs
C:0 T:2 67471 XUs and 78929 XVs -> 2995 XUVs
C:0 T:3 36495 XUs and 33149 XVs -> 315 XUVs
C:0 T:4 15777 XUs and 17047 XVs -> 452 XUVs
C:0 T:5 147690 XUs and 90962 XVs -> 27947 XUVs
C:0 T:6 90440 XUs and 100182 XVs -> 7104 XUVs
C:0 T:7 35492 XUs and 33762 XVs -> 632 XUVs
C:0 T:8 10903 XUs and 11448 XVs -> 157 XUVs
C:0 T:9 91321 XUs and 118280 XVs -> 7258 XUVs
C:0 T:10 39981 XUs and 21216 XVs -> 3661 XUVs
C:0 T:11 140 XUs and 95 XVs -> 21 XUVs
56908 XUVs total
12288 collection wire objects
53133 potential space points
Neighbour search...
9594945 tests to find 4254648 neighbours
Iterating with no regularization...
Begin: 8.1768e+07
0 6.64823e+07
1 6.51831e+07
2 6.49795e+07
3 6.4923e+07
Now with regularization...
Begin: 5.46721e+07
0 5.46113e+07
1 5.45646e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410212033
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410212033
fLifetime: 30000 [us]
30-Oct-2025 23:01:11 GMT  Closed output file "reco_stage1.root"
30-Oct-2025 23:01:11 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          442.865       549.194       815.075       529.538       99.3093        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00158941    0.00492642    0.00745019    0.00520389     0.0018308       10     
reco:ophit:OpHitFinder                              0.37805      0.418516      0.457643      0.419676      0.0301398       10     
reco:opflash:OpFlashFinder                         0.196515      0.235718      0.285292      0.236366      0.0289462       10     
reco:opslicer:OpSlicer                             0.773872      0.976457       1.12033       1.00394      0.109654        10     
reco:caldata:DataPrepModule                         7.20147       7.58326       7.98519       7.66164      0.275628        10     
reco:wclsdatasp:WireCellToolkit                     42.7439       43.5001       46.5951       42.9481       1.11274        10     
reco:gaushit:GausHitFinder                          1.45318       1.85892       2.36137       1.85778       0.24135        10     
reco:nhitsfilter:NumberOfHitsFilter               0.00071736    0.000863445   0.00108909    0.000810614   0.000129705      10     
reco:reco3d:SpacePointSolver                        21.0794       28.2629       42.2777       27.8094       5.82526        10     
reco:hitpdune:DisambigFromSpacePoints               2.14821       2.6541        4.33829       2.47725      0.599951        10     
reco:pandora:StandardPandora                        32.1095       57.6614       127.771       54.8121       25.0332        10     
reco:pandoraWriter:StandardPandora                 0.633209      0.737485       1.02288      0.699315      0.105042        10     
reco:pandoraTrack:LArPandoraTrackCreation           6.24794       8.91476       13.7054       8.15439       2.00444        10     
reco:pandoraShower:LArPandoraShowerCreation         6.13816       10.2921       18.8304       9.37988       3.37811        10     
reco:pandoracalo:Calorimetry                        3.94107       5.18255       7.48687       4.77955       1.02115        10     
reco:pandoracalonosce:Calorimetry                   3.73258       4.89801       7.0314        4.46338      0.941497        10     
reco:pandorapid:Chi2ParticleID                     0.0079344    0.00940338     0.0121348    0.00925582    0.00121956       10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.0744235     0.142078      0.602469      0.0873527     0.154062        10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00766561    0.00892049     0.0118634    0.00844141    0.00123568       10     
reco:pandoraShowercalo:ShowerCalorimetry            6.81234       10.9343       19.1566       9.98148       3.20126        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       6.59246       10.4913       17.7787       9.41347       2.97142        10     
reco:emtrkmichelid:EmTrackMichelId                  226.984       259.344       343.992       253.747       31.6595        10     
reco:crttag:SingleCRTMatchingProducer              0.0206725     0.0317854     0.0399874     0.0316357     0.0048554       10     
reco:crtreco:TwoCRTMatchingProducer                0.0143202     0.0186536     0.0287905     0.0170711     0.0041459       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.02988       1.19106       1.54289       1.11159      0.157344        10     
reco:pandora2Track:LArPandoraTrackCreation          17.5288       27.5405       45.9242       25.8708       7.77959        10     
reco:pandora2calo:Calorimetry                       10.0383       14.7775       24.0242       13.4493       3.96167        10     
reco:pandora2calonosce:Calorimetry                  9.13641       14.0526       23.158        13.0308       3.88626        10     
reco:pandora2pid:Chi2ParticleID                    0.0188802     0.0220216     0.0327098     0.0194302    0.00442397       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP        0.0943402     0.116134      0.165406      0.106931      0.0229615       10     
reco:pandora2calipidnoxyzt:Chi2ParticleID          0.0168061     0.0211419     0.0298169     0.0190734    0.00389996       10     
reco:pandora2Shower:LArPandoraShowerCreation        6.5564        10.2346       16.2984       9.65506       2.58918        10     
reco:pandora2Showercalo:ShowerCalorimetry           7.28166       11.9588       18.2816       11.3778       2.94413        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.81238       10.6897       17.3587       10.2272       2.84578        10     
[art]:TriggerResults:TriggerResultInserter        6.9671e-05    0.000110671   0.000237798    9.585e-05    4.45922e-05      10     
end_path:out1:RootOutput                           1.573e-05    2.08493e-05    4.242e-05    1.76635e-05   7.76751e-06      10     
end_path:out1:RootOutput(write)                     3.68175       4.2328        5.69095       4.06405      0.525436        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5109.97 MB
  Peak resident set size usage (VmHWM): 4257.51 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.10 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.80 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.20 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.01 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11004456
-rw-r--r-- 1 dune01 dune01 2549645126 Oct 30 21:27 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2417115482 Oct 30 21:23 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2272593626 Oct 30 23:01 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2028033262 Oct 30 18:57 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1916987316 Oct 30 18:37 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   83558000 Oct 30 22:54 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243390 Oct 30 23:01 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 30 23:01 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 30 23:01 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 30 23:01 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 30 21:27 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 30 18:57 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 30 23:01 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 30 18:29 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 30 23:01 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 30 21:34 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 30 21:27 debugprod.log
justIN time: 2025-11-03 18:40:58 UTC       justIN version: 01.05.01