justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 233185.0@dunegpschedd02.fnal.gov

Jobsub ID233185.0@dunegpschedd02.fnal.gov
Workflow ID9369
Stage ID1
User namepmelas@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8388608000 (8000 MiB)
Wall seconds limit86400 (24 hours)
Submitted time2025-10-27 06:10:32
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-10-27 10:52:58
From worker nodeHostnamewn2204240.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7513 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes10485760000 (10000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Started2025-10-27 06:12:07
Input filesmonte-carlo-009369-000001
JobscriptExit code0
Real time4h (16748s)
CPU time4h (16009s = 95%)
Max RSS bytes4352536576 (4150 MiB)
Outputting started2025-10-27 10:51:16
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/pmelas/fnal/09369/1/001/reco_stage1.root
Finished2025-10-27 10:52:58
Saved logsjustin-logs:233185.0-dunegpschedd02.fnal.gov.logs.tgz
List job events     Cached HTCondor job logs

Jobscript log (last 10,000 characters)

RawDigitPrepService:prepare:   Running tool adcScaleKeToAdc
ToolBasedRawDigitPrepService:prepare:   Running tool pdsp_RemoveBadChannels
ToolBasedRawDigitPrepService:endEvent: End processing run 1 event 10 with 7 tools.
CookedFrameSource: got 15360 recob::Wire objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 169263 traces, 0 summary
Retagger: tagging trace set: gauss with 128241 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 128241 traces tagged "gauss"
FrameSaver: q=4.48689e+06 n=2258573 tag=gauss
wclsFrameSaver: saving 169263 traces tagged "wiener"
FrameSaver: q=4.7785e+06 n=1995961 tag=wiener
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 20516 XUs and 24139 XVs -> 161 XUVs
C:0 T:1 42312 XUs and 42469 XVs -> 2689 XUVs
C:0 T:2 136373 XUs and 163195 XVs -> 9550 XUVs
C:0 T:3 51824 XUs and 56310 XVs -> 394 XUVs
C:0 T:4 21795 XUs and 23869 XVs -> 222 XUVs
C:0 T:5 144672 XUs and 126056 XVs -> 20275 XUVs
C:0 T:6 198292 XUs and 217654 XVs -> 9007 XUVs
C:0 T:7 66823 XUs and 67222 XVs -> 751 XUVs
C:0 T:8 6160 XUs and 6379 XVs -> 222 XUVs
C:0 T:9 33367 XUs and 43184 XVs -> 3417 XUVs
C:0 T:10 86032 XUs and 41866 XVs -> 6412 XUVs
C:0 T:11 179 XUs and 124 XVs -> 21 XUVs
53121 XUVs total
16707 collection wire objects
49206 potential space points
Neighbour search...
4840520 tests to find 2400076 neighbours
Iterating with no regularization...
Begin: 9.04571e+07
0 6.76937e+07
1 6.61054e+07
2 6.58306e+07
3 6.57328e+07
4 6.56887e+07
Now with regularization...
Begin: 5.23033e+07
0 5.22017e+07
1 5.21528e+07
run: 1 ; subrun: 0 ; event: 10
evttime: 410141648
fLifetime: 30000 [us]
run: 1 ; subrun: 0 ; event: 10
evttime: 410141648
fLifetime: 30000 [us]
27-Oct-2025 10:51:14 GMT  Closed output file "reco_stage1.root"
27-Oct-2025 10:51:14 GMT  Closed input file "detsim_stage2.root"
DataPrepModule::endJob: # events processed: 10
DataPrepModule::endJob:   # events skipped: 0

====================================================================================================================================
TimeTracker printout (sec)                            Min           Avg           Max         Median          RMS         nEvts   
====================================================================================================================================
Full event                                          437.197       542.898       674.515       545.02        68.0297        10     
------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                            0.00149702    0.00451017    0.00745203    0.00444941    0.00194469       10     
reco:ophit:OpHitFinder                              0.33693      0.421145      0.462577      0.419902      0.0364303       10     
reco:opflash:OpFlashFinder                         0.194873      0.247287      0.288225      0.249995      0.0260782       10     
reco:opslicer:OpSlicer                              0.80787      0.994055       1.24705       1.02398      0.131458        10     
reco:caldata:DataPrepModule                         7.46465       7.7793        8.26856       7.72056      0.248527        10     
reco:wclsdatasp:WireCellToolkit                     40.8027       43.3491       46.672        43.3007       1.37587        10     
reco:gaushit:GausHitFinder                          1.72968       2.01816       2.35831       2.01551      0.206137        10     
reco:nhitsfilter:NumberOfHitsFilter               0.000660843   0.000818359   0.000915531   0.000856151   8.94507e-05      10     
reco:reco3d:SpacePointSolver                        21.2136       29.7147       38.9658       27.3741       6.2157         10     
reco:hitpdune:DisambigFromSpacePoints               2.05814       2.78194       3.76945       2.68518      0.487271        10     
reco:pandora:StandardPandora                        47.5027       65.5206       94.396        65.1252       13.7998        10     
reco:pandoraWriter:StandardPandora                 0.600728      0.749466       0.88751      0.730857      0.0814194       10     
reco:pandoraTrack:LArPandoraTrackCreation           4.39781       6.68073       8.11284       7.12441       1.27776        10     
reco:pandoraShower:LArPandoraShowerCreation         5.02191       7.97646       10.8602       8.46028       1.91214        10     
reco:pandoracalo:Calorimetry                        3.02237       3.77368       4.71934       3.7232       0.540324        10     
reco:pandoracalonosce:Calorimetry                   2.48353       3.4015        4.14148       3.46494      0.529896        10     
reco:pandorapid:Chi2ParticleID                    0.00794824    0.00914852     0.0106388    0.00904015    0.000802443      10     
reco:pandoracalinoxyzt:CalibrationdEdXPDSP         0.081075      0.0897746     0.0971412     0.0906863    0.00542994       10     
reco:pandoracalipidnoxyzt:Chi2ParticleID          0.00768453    0.00867663     0.0104007    0.00856979    0.000763912      10     
reco:pandoraShowercalo:ShowerCalorimetry            5.17105       8.35663       11.648        8.21344       2.11934        10     
reco:pandoraShowercalonosce:ShowerCalorimetry       5.10781       8.08438       11.3585       7.91403       2.04511        10     
reco:emtrkmichelid:EmTrackMichelId                  237.115       275.758       330.493       273.832       28.1389        10     
reco:crttag:SingleCRTMatchingProducer              0.0171155     0.0229923     0.0425784     0.0198252    0.00746884       10     
reco:crtreco:TwoCRTMatchingProducer                0.0125636     0.0141783     0.0171121     0.014163     0.00130882       10     
reco:anodepiercerst0:T0RecoAnodePiercers            1.0815        1.22265       1.38483       1.21832      0.0907714       10     
reco:pandora2Track:LArPandoraTrackCreation          14.2575       21.7921       29.5493       22.5374       4.83305        10     
reco:pandora2calo:Calorimetry                       7.36117       11.1793       14.8695       11.2843       2.37332        10     
reco:pandora2calonosce:Calorimetry                  7.05845       10.6351       14.1044       10.8758       2.20438        10     
reco:pandora2pid:Chi2ParticleID                    0.0181481     0.0239757     0.0274384     0.0253781    0.00302011       10     
reco:pandora2calinoxyzt:CalibrationdEdXPDSP         0.10332      0.128427      0.184216      0.122971      0.022599        10     
reco:pandora2calipidnoxyzt:Chi2ParticleID           0.0187       0.0250665     0.0410496     0.0242462    0.00581799       10     
reco:pandora2Shower:LArPandoraShowerCreation        4.99425       7.69954       10.4644       7.5613        1.82348        10     
reco:pandora2Showercalo:ShowerCalorimetry           6.2733        9.37955       12.3418       9.30302       1.96487        10     
reco:pandora2Showercalonosce:ShowerCalorimetry      5.74622       8.40128       10.8597       8.51962       1.75642        10     
[art]:TriggerResults:TriggerResultInserter        7.1435e-05    9.29259e-05   0.000168006   7.96795e-05   2.8244e-05       10     
end_path:out1:RootOutput                          1.2363e-05    1.45904e-05   2.0439e-05    1.4377e-05    2.08045e-06      10     
end_path:out1:RootOutput(write)                     3.98483       4.5112        5.46565       4.38626      0.488412        10     
====================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5196.42 MB
  Peak resident set size usage (VmHWM): 4352.54 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 10
ToolBasedRawDigitPrepService:dtor:  Call count: 60
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.09 sec/event
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.35 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.06 sec/event
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   4.95 sec/event
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.21 sec/event
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.07 sec/event
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec/event
Art has completed and will exit with status 0.
lar exit code 0
.:
total 11095656
-rw-r--r-- 1 dune01 dune01 2577651769 Oct 27 09:17 detsim_stage2.root
-rw-r--r-- 1 dune01 dune01 2444590622 Oct 27 09:14 detsim_stage1.root
-rw-r--r-- 1 dune01 dune01 2312938041 Oct 27 10:51 reco_stage1.root
-rw-r--r-- 1 dune01 dune01 2011618526 Oct 27 06:41 g4_stage2.root
-rw-r--r-- 1 dune01 dune01 1927022382 Oct 27 06:21 g4_stage1.root
-rw-r--r-- 1 dune01 dune01   87500290 Oct 27 10:43 Pandora_Events.pndr
-rw-r--r-- 1 dune01 dune01     243198 Oct 27 10:51 jobscript.log
-rw-r--r-- 1 dune01 dune01     217088 Oct 27 10:51 mem.db
-rw-r--r-- 1 dune01 dune01      62376 Oct 27 10:51 Pandora_Geometry.xml
-rw-r--r-- 1 dune01 dune01      36864 Oct 27 10:51 time.db
-rw-r--r-- 1 dune01 dune01        519 Oct 27 09:17 detsim_single_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 27 06:41 g4_protoDUNE_hist.root
-rw-r--r-- 1 dune01 dune01        519 Oct 27 10:51 protoDUNE_reco1_hist.root
-rw-r--r-- 1 dune01 dune01         52 Oct 27 06:12 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01          7 Oct 27 10:51 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01          0 Oct 27 09:25 ProtoDUNE.log
-rw-r--r-- 1 dune01 dune01          0 Oct 27 09:17 debugprod.log
justIN time: 2025-11-03 18:40:26 UTC       justIN version: 01.05.01